WO2021148932A1

WO2021148932A1 - Method of authenticating a user through analysis of changes in the external ear canal

Info

Publication number: WO2021148932A1
Application number: PCT/IB2021/050369
Authority: WO
Inventors: Mauro Conti; Stefano CECCONELLO; Piero ROMARE; Mattia CARLUCCI
Original assignee: Universita' Degli Studi Di Padova
Priority date: 2020-01-22
Filing date: 2021-01-19
Publication date: 2021-07-29
Also published as: IT202000001192A1

Abstract

The present invention relates to a method (300; 400; 500; 600) of identifying a user comprising the steps of: recording (401; 601) a deformation of the user's external ear canal while the user performs a predetermined movement of the jaw, said deformation being measured by a sensor positioned at the external ear canal; by using an artificial intelligence system, comparing (403; 603) the recorded deformation with a plurality of identification data relating to deformations of external ear canals records of a plurality of sample users,, wherein each of said deformations have been recorded while each sample user of said plurality carried out said predetermined movement of the jaw, wherein the identification data allows identifying a sample user to which each one of the deformations of external ear canals record is associated; identifying (405; 605) the user in case the recorded deformation is associated by the artificial intelligence system with a predetermined sample user of said plurality of sample users, and activating a function of an external device if the user has been identified, wherein the method further comprises a training phase comprising the steps of: recording (301-303; 501) at least one deformation of the external auditory canal of each sample user of said plurality of sample users; selecting (307; 503-509) a plurality of record portions from each deformation record, and elaborating (311; 513) each selected record portion to identify at least one corresponding identification data, Advantageously, each deformation record comprises a plurality of deformation samples acquired with a predetermined frequency. Further, the step of selecting (307; 503-509) a plurality of record portions from each deformation record comprises: dividing (307) each record into a plurality of record portions, each record portion comprising a same number of deformation samples. Finally, the step of elaborating (311; 513) each selected record portion to identify at least one corresponding identification data comprises: - independently elaborating each selected record portion.

Description

METHOD OF AUTHENTICATING A USER THROUGH ANALYSIS OF CHANGES

IN THE EXTERNAL EAR CANAL

DESCRIPTION

TECHNICAL FIELD

The present invention relates to the field of security. More particularly, the invention relates to a method for identifying (or authenticating) a user based on a biometric parameter. In more detail, the method according to embodiments of the present invention identifies a user by recognising deformations of the external ear canal.

STATE OF THE ART

A variety of identification systems have been developed in the art to allow only authorised individuals to access information stored on a device and/ or restricted areas of a building.

In particular, identification systems based on a user's biometric characteristics have had a great development, as each user has unique biometric characteristics that cannot be lost or transferred from one user to another. Furthermore, the use of biometric characteristics does not require the user to store any code or keyword.

To date, it is known to exploit one or more biometric features associated with the ear to uniquely identify a user.

In particular, the research articles M. Burge and W. Burger, " Ear biometrics in computer vision" , in Proceedings 15th International Conference on Pattern Recognition. ICPR-2000, vol. 2. IEEE, 2000, pp. 822-826, and L. Nanni and A. Lumini, "A multi-matcher for ear authentication", Pattern Recognition Letters, vol. 28, no. 16, pp. 2219-2226, 2007, propose methods configured to identify ear biometric features through image acquisition.

Similarly, in P. Yan and K. W. Bowyer, "Biometric recognition using 3d ear shape", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 8, pp. 1297-1308, 2007, is proposed a method to determine a 3D model of the ear from an image of the ear. The 3D model is used for the authentication of a user.

Methods based on image capture are intended to be implemented by surveillance devices, such as cameras or other image capture devices, for example, to identify a user before allowing her/him access to an area. However, such methods cannot be effectively implemented in devices that can be worn by a user.

M. Derawi, P. Bours, and R. Chen, " Biometric acoustic ear recognition" , in Biometric Security and Privacy. Springer, pp. 71-120, 2017 proposes, instead, to identify acoustic features of the ear using a headset equipped with a microphone.

This method is quite invasive, as it involves emitting acoustic waves into the external ear canal and then detecting an eardrum response to them. This procedure can be annoying and/ or distracting to the user, particularly if configured to identify a user continuously or repeatedly - for example, periodically.

In contrast, the use of variations in biometric characteristics over time, identified by the expression 'behavioural biometrics', has been proposed for the recognition of actions taken by a user.

For example, Taniguchi, Kazuhiro et al. "A Novel Earphone Type Sensor for Measuring Mealtime: Consideration of the Method to Distinguish between Running and Meals", Sensors (Basel, Switzerland) vol. 17(2):252, 2017, proposes a device configured to determine an activity undertaken by a user, including strenuous physical activity and eating a meal, by analysing changes in the shape of the external ear canal.

Additionally, US 2010/ 0308999 proposes a device to be attached to an ear and a portion of skin adjacent to the ear, which can be configured to allow the user to control a machine, interface with a predetermined object, and monitor conditions of the user wearing the device. This device has a sensor that can be configured to identify characteristics of a user and detect the removal of the device from the ear. In addition, US 2010/ 0308999 indicates how the device can be used to identify the user based on the conformation of the ear or the response to a sound by the external ear canal.

Finally, US 8,994,647 proposes a device configured to detect variations in the shape of a user's natural orifice, in particular of the external ear canal, which can be used to identify commands provided by the user to control another device.

PURPOSES AND SUMMARY OF THE INVENTION

The purpose of the present invention is to overcome the drawbacks of the known art.

In particular, it is an object of the present invention to present an identification method, based on a behavioural biometric feature, that is accurate and non-invasive and suitable for implementation in portable devices.

It is a further object of the present invention to present a method capable of identifying a user based on the analysis of deformations of the external ear canal, in particular, due to a movement of the user's jaw.

It is a further object of the present invention to present a method capable of rapidly identifying a user based on the analysis of the deformation of the external ear canal.

Furthermore, it is an object of the present invention to present a method for identifying a chewing action performed by the user.

Another object of the present invention is to present a method for identifying a food chewed by a user.

In addition, it is an object of the present invention to present a method for identifying a swallowing action and/ or an eyelid movement.

Finally, it is an object of the present invention to present a method for identifying the articulation of a sound or a word, and possibly identifying one or more sounds and/ or articulated words or a specific sequence of sounds and/or words (e.g. a specific sentence).

These and other objects of the present invention are achieved by means of a device incorporating the features of the appended claims, which form an integral part of the present description.

One aspect of the present invention relates to a method of identifying a user comprising the steps of: recording a deformation of the user's external ear canal while the user performs a predetermined movement of the jaw, named deformation as being measured by a sensor placed at the external ear canal; by means of an artificial intelligence system, comparing the recorded deformation with a plurality of identification data relating to external ear canal deformation records of a plurality of sample users, each of said deformations being recorded while a respective sample user of said plurality performs said predetermined jaw movement, wherein said identification data enables identification of a sample user with which each external ear canal deformation record is associated; identifying the user in the event that the recorded deformation is associated by the artificial intelligence system with a predetermined sample user of said plurality of sample users, and activating a function of an external device if the user has been identified.

The method further comprises a learning phase that includes the steps of: recording at least one deformation of the external ear canal of each sample user of said plurality of sample users; selecting a plurality of record portions of each recording of deformation, and processing each selected record portion to identify that plurality of identifying data.

Advantageously, each deformation comprises a plurality of deformation samples acquired at a predetermined frequency.

In addition, the step of selecting a plurality of record portions from each record of deformation involves: dividing the deformation record into a plurality of record portions comprising the same number of samples.

Finally, the step of processing each selected record portion to identify involves: processing each selected record portioning independently.

Thanks to this solution, it is possible to identify a user safely and reliably. In particular, deformations of the external ear canal have specific characteristics that differ from individual to individual. The use of artificial intelligence makes it possible to identify these deformations effectively and reliably.

Thanks to this solution, it is also possible to substantially expand the number of samples through which artificial intelligence can be trained, significantly improving the reliability and effectiveness with which a specific user is identified, while at the same time substantially limiting the number of samples to be acquired to train artificial intelligence.

Furthermore, splitting the deformation record into a plurality of record portions allows the duration of the recording of the external ear canal deformation necessary to identify the user to be limited. In particular, the minimum duration of the recording of the external ear canal deformation necessary to identify the user is equal to the duration of one of said record portions.

In an embodiment, the number of samples of each of the record portions is defined as the number of samples that allows obtaining record portions that minimise a False Acceptance Rate (FAR) and a False Rejection Rate (FRR).

The Applicant has determined that through the selection of specific record portion sizes, optimised performance of the artificial intelligence system can be achieved.

Preferably, each of said portions of the record comprises an equal number of samples of said plurality of deformation samples, wherein said number of samples is between 100 and 550, preferably the number of samples of each record portion is 250. This subdivision makes it possible to obtain a large number of record portions and, at the same time, contains sufficient information to ensure correct training of the artificial intelligence.

In an embodiment, where the step process each record portion to identify at least one corresponding piece of identification data comprises: determining identification data based on a kurtosis analysis, an asymmetry index and a variance between samples of the same record portion.

In an embodiment, the identification data comprises one or more statistical values selected from quantile 0.2, quantile 0.4, quantile 0.6, quantile 0.8, sigma 1, sigma 2, sigma 3 and variance of the record portions.

Preferably, the identification data are efficiently ordered by means of a K-best algorithm. In particular, the identification data ordered by significance are: quantile 0.8, quantile 0.2, sigma 3, quantile 0.6, sigma 1, variance, sigma 2, and quantile 0.4.

Studies carried out by the Applicant have determined that such values represent the identifying data which allow the recording of a deformation of the external ear canal of a user to be more reliably compared with deformation records of sample users and reliably identified matches with them.

In an embodiment, selecting a plurality of record portions from each deformation record instead comprises: identifying a plurality of positive peaks, i.e. relative maxima, in that deformation record, and acquiring a record portion at each relative maximum.

Preferably, acquiring a record portion at each relative maximum comprises selecting a size for each record portion that avoids overlap between portions of the record acquired at adjacent relative maxima.

In this way, it is possible to ensure the acquisition of a plurality of unique portions of the recording all comprising information relating to a salient part of the predetermined jaw movement.

In an embodiment, recording at least one deformation of the external ear canal of each sample user of said plurality of sample users comprises: recording in parallel at least one deformation of both external ear canals of each sample user of said plurality of sample users.

Furthermore, identifying a plurality of relative maxima, i.e. positive peaks, in said deformation record preferably involves filtering in a predefined passband both external ear canal deformations recorded in parallel, summing together said parallel filtered external ear canal deformation records, identifying positive peaks of the sum of parallel filtered external ear canal recordings.

Finally, acquiring a record portion at each relative maximum preferably involves: acquiring portions of each external ear canal deformation record recorded in parallel in a neighbourhood of a time instant associated with an identified relative maximum.

Thanks to this alternative solution, it is possible to obtain a reduced number of record portions through which it is possible to train the artificial intelligence, significantly improving the reliability and effectiveness with which a specific user is identified, without, however, losing any relevant information to ensure the correct identification of the user.

In an embodiment, the method further comprises the steps of: selecting a group of identification data, preferably by means of a feature selection algorithm, preferably an algorithm based on Recursive Features Elimination.

In an embodiment, said artificial intelligence implements a machine learning model, and in which the method further comprises the step of: training said artificial intelligence to recognise said sample user using at least part of the identification data obtained by processing said plurality of record portions.

Preferably, the artificial intelligence implements a machine learning model comprised among:

Random Forest Classifier;

Logic Regression;

K-nearest neighbours, and C-Support Vector.

These machine learning models make it possible to identify the user effectively and, in particular, the Random Forest Classifier offers the best performance in user identification.

In an embodiment, the step of recording at least one deformation of the external ear canal of each sample user of said plurality of sample users comprises: recording a primary sample deformation while the user performs a chewing movement with the oral cavity empty for an initial predetermined time interval, and recording at least one secondary sample deformation while the user chews an object for a second time interval.

In addition or alternatively, the step of recording at least one deformation of the user's external ear canal while the user performs a predetermined jaw movement comprises: recording a deformation of the user's external ear canal while the user performs a chewing movement with the oral cavity empty, or recording at least one secondary sample deformation while the user chews an object.

Both simulated and real chewing is associated with a large number of deformations of the external ear canal that are unique to each user and therefore allows for recordings of deformations of the external ear canal that include a particularly suitable amount of information to identify the user.

In alternative embodiments, instead of recording deformations associated with chewing, it is possible to record deformations of the external ear canal - due to mandibular movement - associated with the articulation of one or more sounds, one or more specific words and/ or specific sequences of sounds and/ or words.

In such a case, the step of recording at least one deformation of the external ear canal of each sample user of said plurality of sample users comprises: recording at least one sample deformation while the user articulates a sound and/ or a word, or recording at least one sample deformation while the user articulates a predetermined sequence of sounds and/or predetermined words and/or for a predetermined period of time.

Furthermore, the step of recording at least one deformation of the user's external ear canal while the user performs a predetermined jaw movement comprises: recording a deformation of the user's external ear canal while the user articulates a predetermined sound and/ or word, or recording at least one secondary sample deformation while the user articulates a predetermined sequence of sounds and/ or words.

This makes it possible to identify the user by identifying deformations of the external ear canal associated with the articulation of predetermined sounds, words or sequences of sounds and/ or words. Furthermore, this makes it possible to identify specific words, for example, that can be used as voice commands to control a corresponding device.

In other embodiments, deformations of the external ear canal associated with swallowing action and/ or eyelid movement (e.g. blinking) may be recorded.

In such embodiments, the step of recording at least one deformation of the external ear canal of each sample user of said plurality of sample users comprises: recording at least one sample deformation while the user performs a swallowing action, or recording at least one sample deformation while performing an eyelid movement. Similar ly, the step of recording at least one deformation of the user's external ear canal while the user performs a predetermined jaw movement involves: recording a deformation of the user's external ear canal while the user performs a swallowing action, or record at least one secondary sample deformation while the user performs an eyelid movement.

In one embodiment, the steps of the method are repeated in parallel for the deformations of both of the user's external ear canals and the user is only identified if both deformations are associated with the same predetermined sample user.

Using pairs of deformation records relative to both external ear canals increases the effectiveness and reliability of user identification.

In an alternative embodiment, the deformations acquired from the external ear canals are instead analysed independently of each other.

This essentially doubles the number of deformation records obtained in the same time interval.

A different aspect of the present invention concerns an electronic device comprising: a processing module, a memory module, and at least one sensor insertable into, or facing the external ear canal of a user and configured to measure a deformation of the external ear canal. Advantageously, the electronic device is configured to implement the method according to any one of the embodiments considered above.

Preferably, the electronic device is integrated into at least one pair of earphones or headset of a pair of earphones that can be connected to an additional electronic device - for example, a smartphone.

In an alternative embodiment, the electronic device is integrated into a smartphone, mobile phone or other similar device, preferably at the loudspeaker to be placed next to the ear during a telephone conversation.

Further features and purposes of the present invention will become clearer from the following description.

BRIEF DESCRIPTION OF THE DRAWS

The invention will be described herein by reference to certain examples, provided for explanatory and non-limiting purposes, and illustrated in the accompanying drawings. These drawings illustrate different aspects and embodiments of the present invention and, where appropriate, reference numerals illustrating structures, components, materials and/ or similar elements in different figures are indicated by similar reference numerals.

Figure 1 schematically illustrates a device suitable for implementing the method according to one embodiment of the present invention;

Figure 2 is a qualitative graph of the deformation of the external ear canal as a function of time;

Figure 3 is a flow chart relating to a procedure for training an artificial intelligence in accordance with a form of embodiment of the present invention;

Figure 4 is a flow chart of an identification procedure according to a form of embodiment of the present invention;

Figure 5 is a flow chart relating to a procedure for training an artificial intelligence in accordance with an alternative embodiment of the present invention;

Figure 6 is a flow chart of an identification procedure according to an alternative embodiment of the present invention, and

Figure 7 is a qualitative graph of the deformation trend of a pair of a user's external ear canals and a sum of these trends as a function of time.

DETAILED DESCRIPTION OF THE INVENTION

While the invention is susceptible to various modifications and alternative constructions, certain preferred embodiments are shown in the drawings and will be described in detail below. It should be understood, however, that there is no intention to limit the invention to the specific embodiment shown, but, on the contrary, the invention is intended to cover all modifications, alternative constructions, and equivalents falling within the scope of the invention as defined in the claims.

The use of "for example", "etc", "or" indicates non-exclusive alternatives without limitation unless otherwise indicated. The use of "includes" means "includes, but not limited to" unless otherwise indicated.

Figure 1 shows a system 1 in which it is possible to implement a method according to a form of embodiment of the present invention.

In the example considered, the system 1 comprises a sensor device 10 and a user device 20. The sensor device 10 comprises an ear portion 11 and a body 13.

The ear portion 11 is configured to be at least partially inserted within the external ear canal 31 of a user 30. The ear portion 11 includes a detection assembly configured to measure a deformation of the external ear canal. For example, the ear portion includes an LED diode 111 configured to emit an electromagnetic radiation in the infrared and a phototransistor 113 configured to detect electromagnetic radiation in the infrared. In particular, the LED diode 111 is arranged in the ear portion 11 so as to radiate electromagnetic radiation (represented by a dashed arrow in Figure 1) towards a wall of the external ear canal 31, when the ear portion is inserted - at least partially - into the external ear canal 31. Similarly, the phototransistor 113 is arranged in the ear portion 11 so as to absorb a portion of electromagnetic radiation (represented by a dashed arrow in Figure 1) emitted by the LED diode 111 and reflected by the wall of the external auditory conduit 31.

The body 13 includes a processing module 131 and a memory module 133 and a transceiver module 135. The processing module 131 is configured to manage the operation of the entire sensor device 10, thus it is connected to the other modules of the sensor device 10. For this purpose, the processing module 131 may comprise one or more of a processing element - such as a processor, a microprocessor, a microcontroller, an ASIC, an FPGA, a DSP, etc. - and one or more ancillary circuits - such as a synchronisation signal generation circuit (clock), ADC and/or DAC converters, amplifiers for input/ output signals, etc. Advantageously, the processing module 131 is configured to implement operating procedures, stored in the memory module 133, for example, in the form of software applications or in hardware components, for example, in the form of firmware. The memory module 133 preferably comprises at least one non-volatile memory unit and at least one volatile memory unit configured to permanently and temporarily store, respectively, data typically in binary format. The transceiver module 135 comprises elements necessary to exchange data via a wired connection 15 - for example, by means of a two-wire cable - or wirelessly - for example, Bluetooth - with the user device 20.

In addition, the sensor device 10 may comprise one or more additional modules (not shown), such as a power supply module configured to provide electrical power necessary for the operation of the sensor device 10.

In embodiments of the present invention, the sensor device 10 is configured to execute an artificial intelligence software application, in short, artificial intelligence AI, as described in greater detail below in this description.

Of course, the sensor device 10 may comprise a second body and a second ear portion (not illustrated) entirely corresponding to the body 13 and the ear portion 11 just described, and configured to be associated with the other external ear conduit (not illustrated) of the user 30. The user device 20 comprises a processing module 21, a memory module 23 and a transceiver module 25 having similar functionality to the corresponding modules 131, 133 and 135 described above, with the processing module 21 connected to the remaining modules 23 and 25 to control their operation. The user device 20 may also comprise one or more additional modules (not shown) - such as an interface module a power supply module, etc. - and necessary ancillary circuitry. For example, the user device 20 may substantially consist of a smartphone, a personal computer, a building/ room security system, a home automation system, etc.

The system 1 is configured to implement an identification method according to an embodiment of the present invention.

Preferably, the identification method comprises a training procedure 300 (of which a flowchart is illustrated in Figure 3) which is configured to train the artificial intelligence AI to recognise one or more users wearing the sensor device 10.

In detail, the training procedure 300 comprises a data acquisition phase, in which the deformation of the external ear canal 31 of the user 30 is recorded, while the latter performs at least one movement of the jaw, by means of the sensor device 10 worn by the user (block 301).

In the embodiment considered, the jaw movement performed by the user corresponds to a simulated chewing action - i.e., the user chews 'empty' - or the user chews an object, generally a food. Therefore, in the following the external ear canal deformation records 31, are referred to as deformation records Cr(t) for simplicity.

The term "deformation" is used in this description to identify a change in the physical characteristics - i.e. one or more of the volume, diameter, conformation, etc. - of the external ear canal. In particular, a deformation involves a change, of an elastic type, in the geometric shape of the external ear canal, which disappears when the stress that caused it ceases. Each, deformation record Cr(t) (of which a qualitative example is illustrated in Figure 2) thus obtained has a time-varying duration and extends from a first chewing action - or 'chewing' - at the moment when the user 30 swallows - bolus in the case of chewing food or saliva in the case of simulated chewing. Preferably, the user 30 may be asked to perform each chewing action for a predetermined duration, for a duration greater than a minimum duration and/ or for a duration less than a maximum duration.

In the example considered, each deformation record Cr(t) produced by the sensor 10 is a time-varying electrical signal - for example, a signal with a variable voltage value - proportional to the deformation of the external ear canal 31 during the execution of a corresponding movement of the jaw mentioned above.

In the preferred embodiment, a plurality of sample deformation records Cr(t) referring to different chews performed by the user 30 are acquired. For example, a user is required to perform simulated chewing and chewing of at least one or more different foods, for example four different foods of different textures. Further, it is contemplated that two or more, e.g. three, cycles of acquiring deformation records Cr(t) are repeated so as to obtain two or more deformation records Cr(t) associated with each type of chewing.

After that, the data acquisition step involves populating a dataset - i.e., a collection of data - with the acquired deformation records Cr(t) and a plurality of other sample deformation records Dr(t) do not refer to the user 30 (block 303).

For example, the other deformation records Dr(t) may comprise deformation records referred to users of other devices 1 (not illustrated). Preferably, the sample deformation records Dr(t) are performed under conditions similar to those under which the deformation records Cr(t) are performed. In other words, the sampled deformation records Dr(t) are acquired during jaw movements of other users in a manner similar to that described above.

Such other deformation records Dr(t) are stored in the memory module 133 of the sensor device 10 or may be acquired from a database (not shown) external to the user device 10 which the latter accesses via the transceiver module 135.

After that, procedure 300 includes a feature extraction phase, referred below as 'identification data Y'. As is well known, the features or identification data Y identified during a feature extraction phase are a reduced set of data compared to the totality of available data, but which are considered to contain the information necessary to allow a desired analysis. In the case under consideration, the identification data Y extracted from the Cr(t) and Dr(t) records as described below allow the Cr(t) and Dr(t) records to be classified according to the user with whom they are associated. At this stage there is a pre-processing of each of the Cr(t) and Dr(t) records in the dataset (block 305).

Preferably, although not limitatively, for each of the deformation records Cr(t) and Dr(t) in the dataset, the pre-processing step comprises: a) identifying and subtract the respective average value; b) identifying and eliminate any contribution due to movement of the sensor device 10 during recording, and c) identifying and normalising outlier values - i.e., abnormal values - which may be caused by abrupt movements of the sensor device 10 or user actions such as swallowing, head movements, etc.

Each deformation record Cr(t) and Dr(t) is then divided into a plurality of record portions ACr(t) and ADr(t), respectively (block 307).

In particular, each deformation record Cr(t) and Dr(t) consists substantially of a signal sampled at a predetermined sampling frequency. For example, each deformation record Cr(t) and Dr(t) is a signal sampled at a substantially constant frequency F between 10 Hz and 100 Hz (10 Hz < F < 100 Hz). Tests carried out by the applicant have determined that reliable user identification 30 can be ensured by splitting each deformation record Cr(t) and Dr(t) into a number of record portions ACr(t) and ADr(t) each comprising between 100 and 500 consecutive samples s of the respective deformation record Cr(t) and Dr(t) - as schematically shown in Figure 2. Preferably, it is envisaged to split each of the deformation records Cr(t) and Dr(t) into portions of the recordings ACr(t) and ADr(t) comprising the same number of samples.

In particular, splitting the deformation records Cr(t) and Dr(t) into portions of the record ACr(t) and ADr(t) all comprising 250 samples allows for better accuracy - measured in terms of Fl-score, False Acceptance Rate (FAR), and/ or False Rejection Rate (FRR).

In particular, the size of the record portions ACr(t) and ADr(t) comprising 250 samples appears to provide a particularly optimal relationship between the size of the record portions ACr(t) and ADr(t) - and thus, the volume of physical memory occupied by them - and the reliability of the user identification 30 provided by the system.

For example, the size of the record portions ACr(t) and ADr(t) can be defined by exploiting a trade-off between the two performance indicators FAR and FRR. Specifically, the applicant determined that the optimal number of samples can be defined by combining the minimization of both performance indicators. In other words, the size of the record portions ACr(t) and ADr(t) is selected so that the Equal Rejection Rate (EER) of the artificial intelligence performing the user identification is minimised.

Preferably, it is contemplated to discard any record portions ACr(t) and ADr(t) comprising a number of samples different from a predetermined number of samples - for example 250 - in order to have homogeneous portions of the recordings ACr(t) and ADr(t). In fact, due to the variability of the durations of the deformation records Cr(t) and Dr(t) it is possible that some portions of the recordings ACr(t) and ADr(t) - in particular, the final record portions - have a different, in particular lower, number of samples than the predetermined number.

In particular, the record portions ACr(t) and ADr(t) of the same deformation record Cr(t) and Dr(t) are subsequently considered to be completely independent deformation records.

The Applicant has determined that the record portions ACr(t) and ADr(t) having the optimal number of samples ensure that they comprise at least one salient phase of chewing that allows the user to be reliably identified, for example, a contraction of the masticatory muscles leading to a compression between the teeth of the mandible and the jaw or, conversely, a contraction of the masticatory muscles leading to a maximum distance of the mandible from the jaw during chewing. In other words, each record portions ACr(t) and ADr(t) refers to a corresponding single chewing action (i.e., a sequence of approaching and receding of the mandible from the jaw) of the total recorded chewing.

After that, the moving average of each record portion ACr(t) and ADr(t) is calculated (block 309). Indeed, the moving average of the record portions ACr(t) and ADr(t) - as well as of the deformation records Cr(t) and Dr(t) - allows to emphasise chewing characteristics specific to the considered user, thus allowing to determine the most effective identification data Y.

In detail, a plurality of identification data Y are extracted for each record portion ACr(t) and ADr(t) (block 311). In a preferred embodiment, it is contemplated to select a predefined number of identification data Y. Studies carried out by the Applicant, have made it possible to determine how a number of identification data Y between five and fifteen, preferably ten, allows reliable results to be obtained at a low computational cost.

For example, the identification data Y is extracted by statistically analysing the set of samples included in each record portion ACr(t) and ADr(t). Additionally or alternatively, the identification data Y may be determined based on a comparison of one or more portions of the record ACr(t) and ADr(t).

In a non-limiting way, the identification data Y can be determined by means of a kurtosis, an index of asymmetry(skewness), a variance between samples of the same record portion ACr(t) and ADr(t), etc. In one embodiment, a select K-Best algorithm is used configured to identify and order the identification data Y by importance according to a predetermined criterion. For example, the order of the identification data Y is defined based on the variance of each identification data Y.

Such a statistical analysis enables the identification data Y useful for user identification to be identified. In one embodiment, the selected identification data Y comprises statistical values of the selected parameters, for example one or more of quantile 0.2, quantile 0.4, quantile 0.6, quantile 0.8, sigma 1, sigma 2, sigma 3 and variance. In particular, Applicant has determined that according to the K-best algorithm, the most significant data are in order: quantile 0.8, quantile 0.2, sigma 3, quantile 0.6, sigma 1, variance, sigma 2, and quantile 0.4.

In summary, the identification data Y are the result of the segmentation, moving average calculation and extraction operations described above and comprise a set of features obtained by processing the record portions ACr(t) and ADr(t), i.e. features associated with a measurement of the deformation of the external ear canal 31 of the user 30 acquired via the sensor 10 and of the sample users.

Subsequently, an artificial intelligence AI learning phase is performed, which initially involves defining a subset of the identification data Y' from among the identification data Y for each record portion ACr(t) and ADr(t) (block 313). In particular, the most relevant identification data Y are identified and selected, while the remaining parameters are discarded as less relevant. In the preferred embodiment, the subset of identifying data Y' is identified using a Recursive Feature Elimination (RFE) based algorithm.

Finally, artificial intelligence AI is trained using the identification data subsets Y' associated with the record portions ACr(t) and ADr(t) (block 315). In particular, such subsets of identification data Y' are used to train a supervised Machine Learning (ML) model, or machine learning that essentially constitutes artificial intelligence AI. At the end of the training, the Machine Learning model is configured to discriminate between the identification data Y' associated with one of the record portions ACr(t) relating to the user 30 and the identification data Y' associated with one of the record portions ADr(t) not relating to the user 30 to be identified. Studies carried out by the Applicant have identified Machine Learning models based on:

- Linear Regression;

- K-Nearest Neighbours (KNN);

- Random Forest, and

- Support Vector Machine.

In particular, the Applicant has determined that Machine Learning models based on Random Forest achieve better performance.

At the end of the procedure 300 the device 10 is then able to correctly identify the user 30 based on deformations of the external ear canal 31 of the user during chewing.

Accordingly, the system 1 may be configured to implement an identification procedure 400 (a flowchart of which is illustrated in Figure 4) designed to allow the use and/ or operation of the user device 20 once the user 30 has been identified.

In detail, the user 30 initially wears the device 10 - in particular, it inserts at least part of the ear portion 11 into the external ear canal 31 - and the deformation of the external ear canal 31 is recorded during a chewing performed by the user 30 for a predetermined period of time (block 401) - i.e. a deformation record to be identified.

Thanks to the subdivision into record portions ACr(t) and ADr(t) of the deformation records Cr(t) and Dr(t) used during the training of the artificial intelligence AI, it is possible to identify the user 30 by means of a short external ear canal deformation record 31. In particular, the minimum duration of the external ear canal deformation record 31 is equal to the duration of one record portioning - that is, equal to the inverse of the sampling frequency multiplied by the number of samples included in the record portions ACr(t) and ADr(t) used during training. For example, in the case of a sampling rate of 100 Hz and a number of samples per record portion of 250, the minimum duration of a deformation record useful for user recognition is 2.5 seconds.

Alternatively, the period of time for which the recording of ear canal deformation is acquired may depend on the object chewed and/ or the user's own chewing habits 30.

Chewing may include empty chewing and/or chewing of any food used during the training procedure 300.

Preferably, the user 30 may be guided in performing the chewing by means of information provided through a user interface of the user device 20 and/ or a further device (not shown) connected to the sensor device 10.

The chewing recording is processed by the sensor device 10 (decision block 403) in order to compare the chewing recording with the identification data subsets Y' and identify whether the recorded chewing was performed by the user 30. In other words, the artificial intelligence AI identifies or does not identify the user based on the training procedure 300 it has undergone.

For example, it is contemplated to extract and select a subset of identification data Y' from the chewing recording - in a manner substantially corresponding to the above described pre-processing, moving average, extraction and selection steps described in relation to the training procedure 300 (blocks 305, 309, 311, 313) - and then base identification on such identification data Y'.

In the event that the identification is successful (output branch Y of block 403), the device 10 communicates to the user device 20 the success of the identification, thereby allowing the user device 20 (block 405) to be used and/ or put into operation. For example, if the user device 20 is a smartphone, successful identification may enable an 'unlocking' of the user interface, may enable activation of a software application installed on the same, and/or may enable a payment or other transaction performed through a software application running on the user device 20. Conversely, if the user device 20 corresponds to, or is part of, a security system, successful identification may enable a lock to be unlocked or an alarm system to be disabled.

On the contrary, if the identification is unsuccessful (output branch N of block 403), the device 10 communicates to the user device 20 the failure of the identification thereby preventing the use and/ or commissioning of the user device 20 (block 407).

Optionally, the procedure may comprise a antitamper and/ or alarm step (dashed block 409) in which the user device 20 enters a blocking mode that prevents interaction attempts by the unidentified user 30 and/ or enters an alarm mode that causes an alarm signal to be issued and/ or an alarm communication to be transmitted to another device.

The invention thus conceived is susceptible to numerous modifications and variants, all of which fall within the scope of the present invention as reflected in the appended claims.

For example, deformation records Cr(t) and Dr(t) can be acquired by variable frequency sampling.

In particular, when the sensor device 10 comprises a second body 13 and a second ear portion 11, the method according to embodiments of the present invention comprises acquiring Crl(t) and Cr2(t) chewing recordings from both external ear canals 31 of the user 30, in parallel. The deformation records are then processed - preferably, in parallel - to extract respective feature subsets YG and Y2' which are used to train the artificial intelligence AI to identify the user 30 based on a pair of recordings of the same chewing carried out in parallel in the two external ear canals 31 of the user 30.

In this case, a correlation can be determined between portions of the recording made simultaneously in the user's right and left external ear canal.

Furthermore, in an embodiment (not illustrated), the training procedure comprises a preliminary step in which a resting condition - or baseline - of the external ear canal 31 of the user 30 is determined. For example, a recording of deformations of the external ear canal 31 is made while the user 30 assumes a predefined posture. This allows compensation for specific morphological particularities of the external ear canal 31 of the user 30. In an alternative embodiment, it is planned to associate different weights to the features of subsets Y' associated with record portions ACr(t) and features of subsets Y' associated with record portions ADr(t) respectively. In this way it is possible to train artificial intelligence AI more effectively.

In an alternative embodiment (referred to in Figures 5 - 7), it is planned to select sample record portions of the ACr'(t) and ADr'(t) of each acquired Cr(t) and Dr(t) deformation record, according to a criterion described below.

During an alternative training procedure 500 (of which a flowchart is illustrated in Figure 5), it is contemplated to acquire pairs of samples of deformation records Cr,dx(t) and Cr,sx(t), and Dr,dx(t) and Dr,sx(t) in parallel from the right external ear canal and the left external ear canal of the user 30 (block 501). In other words, a "stereo" acquisition of each sample chew is performed.

Each deformation record Cr,dx(t) and Cr,sx(t), and Dr,dx(t) and Dr,sx(t) is subjected to filtering in a predetermined ABW bandwidth, in particular between 0 Hz and 5Hz (0 Hz < ABW < 5 Hz), more preferably between 0.5 Hz and 3 Hz (0.5 Hz < ABW < 3 Hz) (block 503).

Next, the deformation records Cr,dx(t) and Cr,sx(t), and Dr,dx(t) and Dr,sx(t) acquired in parallel are summed together to obtain respective recording sums åC(t) and åD(t) (block 505) as can be appreciated in the graph of Figure 7, in which two deformation records Cr,dx(t) and Cr,sx(t) acquired in parallel and a corresponding recording sum åC(t) are qualitatively illustrated. In other words, a "mono" recording of the sample chewing is generated from the "stereo" recordings of the same.

Each recording sum åC(t) and åD(t) is analysed to identify P-peaks of positive value (block 507) - as appreciable in the graph in Figure 7.

As will be obvious to a skilled person in the art, the term "peak" denotes a local maximum of the deformation records Cr,dx(t), Cr,sx(t), and the corresponding recording sum åC(t). In particular, the applicant found that each peak is associated with a contraction of the masticatory muscles leading to a maximum distancing of the mandible from the jaw.

Advantageously, each recording sum åC(t) and åD(t) is analysed by means of one or more suitable algorithms, e.g. one or more signal analysis algorithms included in the Python 3 'signal' library.

Advantageously, the extraction of local maxima can be efficiently performed by applying prior band-pass filtering to the deformation records åC(t) and åD(t) - preferably, in a bandwidth between 0.5 Hz and 1.5 Hz. In this way, the peaks due to chewing are easily discriminated and, therefore, processable.

The identified peaks are used to identify the chewing actions in the deformation records Cr,dx(t) and Cr,sx(t), and Dr,dx(t) and Dr,sx(t), i.e., the compressive action of the mandible against the user's jaw 30 or, in a dual manner, the removal of the mandible from the user's jaw 30. Specifically, from each deformation record Cr,dx(t) and Cr,sx(t), and Dr,dx(t) and Dr,sx(t) corresponding record portions AC'r(t) and AD'r(t) are acquired in a surrounding of a time instant At associated with an identified peak P (block 509) - as illustrated schematically in the graph in Figure 7. Preferably, each acquired record portion AC'r(t) and AD'r(t) comprises a predetermined number of samples of the corresponding deformation record Cr,dx(t), Cr,sx(t), Dr,dx(t) or Dr,sx(t), e.g. 70 samples, more preferably 110 samples, centred around the corresponding peak P. Advantageously, the Applicant has determined that such a number of samples minimises the likelihood that the record portions AC'r(t) and AD'r(t) comprise more than one chewing action or that there is an overlap between record portions AC'r(t) and AD'r(t) centred around different peaks P. Again, the size of the record portions AC'r(t) and AD'r(t) is selected so that the Equal Rejection Rate (EER) of the artificial intelligence performing the user identification is minimised.

In other words, we obtain a number of record portions AC'r(t) and AD'r(t) equal to the number of peaks P identified for each deformation record Cr,dx(t), Cr,sx(t), Dr,dx(t) and Dr,sx(t). As will be evident, each record portions AC'r(t) and AD'r(t) refers to a corresponding single chewing action (i.e., a sequence of approaching and receding of the mandible from the jaw) of the total recorded chewing.

Subsequently, in steps 511-517 the record portions AC'r(t) and AD'r(t) are subjected to the same operations described above in steps 309 - 315 in relation to the procedure 300 in order to extract subsets of identification data Y' and use the latter to train the artificial intelligence AI in order to identify the user 30. It will be apparent to the skilled person that record portions AC'r(t) and AD'r(t) may be considered independently of each other or, alternatively, record portions AC'r(t) and AD'r(t) of acquired deformation records Cr,dx(t) and Cr,sx(t), and Dr,dx(t) and Dr,sx(t), acquired in parallel and associated with the same peak P may be considered in parallel to identify the user 30.

Similarly, an alternative identification procedure 600 (of which a flowchart is illustrated in Figure 6), wherein steps 601 - 609 substantially correspond to steps 401-409 of the above-described procedure 400, except that at step 601 deformation records of both external ear canals of the user 30 are acquired in parallel, which (at step 603) are processed to extract subsets of identification data Y" in a manner analogous to that just described in procedure 500 so as to identify user 30 by comparing said subsets of identification data Y" with subsets of identification data Y' (at step 605).

There is nothing to prevent reiteration of the identification procedure 400 or 600 after a failure to identify the user, in particular a variant of the identification procedure provides for reiteration of the identification procedure 400 or 600 for a maximum number of consecutive unsuccessful iterations (e.g., three iterations) before denying permission to use the user device 20 permanently.

Furthermore, it is possible to envisage training the artificial intelligence to recognise a type of object, in particular the type of food, chewed by the user. In this way, it is possible to add a further level of security, as it is possible to allow identification of the user 30 only when a predetermined food is chewed. Additionally or alternatively, it is possible to configure the sensor device 10 to identify foods assimilated by the user 30 in order to monitor their diet and/or provide dietary guidance - for example, by training the artificial intelligence with sample chewing recordings associated with one or more foods to be identified.

In alternative embodiments (not illustrated), the recognition of one or more foods assimilated by the user 30 is integrated into a system configured to automatically determine a payment to be made to a caterer who has provided such foods to the user 30.

In different embodiments (not illustrated) in accordance with the present invention, instead of recording deformations associated with chewing, it is possible to record deformations of the external ear canal associated with the articulation of one or more sounds, one or more specific words and/ or specific sequences of sounds and/ or words.

In such a case, the step (at block 301) of recording at least one deformation of the external ear canal of each sample user of said plurality of sample users comprises recording at least one sample deformation while the user performs a mandibular movement associated with the articulation of a sound and/or a word, or recording at least one sample deformation while the user articulates a predetermined sequence of sounds and/ or words and/ or for a predetermined period of time. The same variation can clearly be applied to step 501 of alternative procedure 500 mutatis mutandis.

Similarly, the step (at block 401) of recording at least one deformation of the user's external ear canal while the user performs a predetermined jaw movement includes recording a deformation of the user's external ear canal while the user articulates a predetermined sound and/or word, or recording at least one secondary sample deformation while the user articulates a predetermined sequence of sounds and/or words. The same variation can clearly be applied to step 601 of alternative procedure 600 mutatis mutandis.

In this variant, the user is then identified based on deformations of the external ear canal associated with the articulation of predetermined sounds, words or sequences of sounds and/ or words rather than chewing.

In alternative embodiments, it is contemplated to configure the method to identify one or more sounds and/ or words associated with control commands of a corresponding device connected to the sensor device.

Finally, there is nothing to prevent the artificial intelligence from being trained to recognise a swallowing action, an eye movement and/or the blinking of eyelids, in addition to or as an alternative to the recognition of chewing and/or the articulation of sounds or words associated with a jaw movement.

In this case, the steps of recording at least one deformation of the external ear canal of each sample user of said plurality of sample users (block 301) and recording at least one deformation of the user's external ear canal while the user performs a predetermined jaw movement (block 401) comprise recording at least one sample or identification deformation, respectively, while the user performs a swallowing action, or recording at least one sample or identification deformation, respectively, while performing an eyelid movement.

The same variant can clearly be applied to steps 501 and 601 of the alternative procedures 500 and 600 mutatis mutandis.

Of course, all details can be replaced by other technically equivalent elements.

For example, in an alternative embodiment (not illustrated), the system comprises an alternative sensor device having only the detection assembly - namely, the LED diode 111, the phototransistor 113, and possibly ancillary circuitry. The alternative sensor device is configured to be connected directly - wired or wirelessly - to the user device 20 or a portable device - e.g., a smartphone or tablet - so as to implement the identification method by taking advantage of the processing module, the memory module and, possibly, the transceiver module of the said user device or portable device. In addition, there is nothing to prevent the device from being integrated into earphones or headphones for sound reproduction. In this case, it is also possible to provide acoustic indications to the user to guide him through the learning and identification procedures.

Again, instead of storing sample deformation records Dr(t), it is possible to simply store corresponding subsets of identification data Y' of the pluralities of identification data Y' for each sample record portion ADr(t). This makes it possible to reduce the processing time and computational load required of the processing module, and, possibly, an overall volume of non-volatile memory of the memory module required to implement the method according to the present invention.

It will be apparent to a person skilled in the art how the method and system described herein can be used in an authentication procedure, wherein a user's identity is verified and certified by a certification entity based on whether or not a match is detected between sample records of ear canal deformation of a user whose identity can be authenticated and a record of ear canal deformation of the user currently to be authenticated.

In conclusion, all details can be replaced by other technically equivalent elements. In particular, one or more steps of the method described above may be performed in parallel with each other instead of in series or vice versa. Furthermore, nothing prohibits combining the steps of the procedures 300 and 400 and/ or 500 and 600 of the method to obtain an alternative procedure, just as one or more optional steps can be added to and/ or removed from the procedures 300 and 400 and/ or 500 and 600 described above according to specific implementation needs without thereby going outside the scope of protection of the following claims.

In particular, it will be apparent to the person skilled in the art that the procedure for selecting portions of the deformation record around the peaks identified in the deformation record described in relation to procedure 500 is also applicable to procedure 300 mutatis mutandis. Conversely, the sampling of the external ear canal deformation and the criteria for selecting the number of samples in each portion of the deformation record described in relation to procedure 300 are also applicable to procedure 500 mutatis mutandis.

Claims

1. Method (300; 400; 500; 600) for identifying a user including the steps of: recording (401; 601) a deformation of the user's external ear canal while the user performs a predetermined movement of the jaw, said deformation being measured by a sensor positioned at the external ear canal; by using an artificial intelligence system, comparing (403; 603) the recorded deformation with a plurality of identification data relating to deformations of external ear canals records of a plurality of sample users, , wherein each of said deformations have been recorded while each sample user of said plurality carried out said predetermined movement of the jaw, wherein the identification data allows identifying a sample user to which each one of the deformations of external ear canals record is associated; identifying (405; 605) the user in case the recorded deformation is associated by the artificial intelligence system with a predetermined sample user of said plurality of sample users, and activating a function of an external device if the user has been identified, wherein the method further comprises a training phase comprising the steps of: recording (301-303; 501) at least one deformation of the external auditory canal of each sample user of said plurality of sample users; selecting (307; 503-509) a plurality of record portions from each deformation record, and elaborating (311; 513) each selected record portion to identify at least one corresponding identification data, characterized in that each deformation record comprises a plurality of deformation samples acquired with a predetermined frequency, and in that the step of selecting (307; 503-509) a plurality of record portions from each deformation record comprises: dividing (307) each record into a plurality of record portions, each record portion comprising a same number of deformation samples, and in that the step of elaborating (311; 513) each selected record portion to identify at least one corresponding identification data comprises:

- independently elaborating each selected record portion.

2. Identification method (300; 400; 500; 600) according to claim 1, wherein the number of deformation samples in each record portion is defined as the number of deformation samples that allows to minimize the False Acceptance Rate and a False Rejection Rate.

3. Identification method (300; 400; 500; 600) according to claim 1 or 2, wherein the number of deformation samples in each of said record portion is comprised between 100 and 550, preferably the number of deformation samples in each of said record portion is equal to 250.

4. Identification method (300; 400; 500; 600) according to any one of the preceding claims, wherein the step of elaborating (311; 513) each selected record portion to identify at least one corresponding identification data comprises:

- determine the identification data based on a kurtosis, skewness, or a variance analysis of the deformation samples of a same record portion.

5. Identification method (300; 400; 500; 600) according to any one of the preceding claims, wherein the identification data comprise one or more statistical values selected among 0.2 quantile, 0.4 quantile, 0.6 quantile, 0.8 quantile, sigma 1, sigma 2, sigma 3 and variance of the record portions.

6. Identification method (300; 400; 500; 600) according to any one of the preceding claims, wherein the step of selecting (307; 503-509) a plurality of registration portions from each deformation record comprises: identifying (503-507) a plurality of relative maxima in said deformation record, and acquiring (509) a record portion in correspondence of each relative maxima.

7. Identification method (300; 400; 500; 600) according to claim 6, wherein the step of acquiring (509) a record portion in correspondence of each relative maxima comprises: selecting a size of each record portion that avoids a superposition between record portions acquired at adjacent relative maxima.

8. Identification method (300; 400; 500; 600) according to claim 6 or 7, wherein the step of recording (301-303; 501-503) at least one deformation of the external auditory canal of each sample user of said plurality of sample users comprises: recording (501) in parallel at least one deformation of both external ear canals of each sample user of said plurality of sample users, and wherein the step of identifying (503-507) a plurality of positive peaks in said deformation record comprises: filtering (503) in a predefined passband both deformations of the external auditory canals recorded in parallel, adding (505) together said filtered deformation records of the external auditory canals recorded in parallel, identifying (507) positive peaks of said sum of filtered records of the external auditory canals, and wherein the stop of acquiring (509) a record portion in correspondence of each positive peak comprises: acquiring (509) record portions of each deformation record of the external auditory canals recorded in parallel in a neighborhood of a temporal instant associated with an identified peak.

9. Identification method (300; 400; 500; 600) according to any one of the preceding claims, wherein said artificial intelligence implements a machine learning model, and in which the method further comprises the step of: training (315; 517) said artificial intelligence to identify said predetermined sample user using at least part of the identification data obtained by processing said plurality of record portions.

10. Identification method (300; 400; 500; 600) according to claim 8, wherein said machine learning model is selected among:

Random Forest Identifier;

Logic Regression;

K-nearest neighbors, and C-Support Vector.

11. Identification method (300; 400; 500; 600) according to any one of the preceding claims, wherein the step of recording (301-303; 501-503) at least one deformation of the external auditory canal of each sample user of said plurality of sample users comprises: recording a primary sample deformation while the user performs a chewing movement with the oral cavity empty for a first predetermined time interval, and record at least one secondary sample deformation while the user chews an object for a second predetermined time interval.

12. Identification method (300; 400; 500; 600) according to claim 11, wherein the step of recording (401; 601) at least one deformation of the user external auditory canal while the user performs a predetermined movement of the jaw comprises: recording a deformation of the user external auditory canal while the user performs a chewing movement with the oral cavity empty, or recording at least one secondary sample deformation while the user chews an object.

13 Identification method (300; 400; 500; 600) according to any one of the preceding claims, wherein the step of recording (301-303; 501-503) at least one deformation of the external auditory canal of each sample user of said plurality of sample users comprises: record at least one sample deformation while the user articulates a sound and/ or a word, or record at least one sample deformation while the user articulates a predetermined sequence of sounds and/ or words and/ or for a predetermined period of time.

14. Identification method (300; 400; 500; 600) according to claim 13, wherein the step of recording (401; 601) at least one deformation of the user external auditory canal while the user performs a predetermined movement of the jaw comprises: recording a deformation of the user external ear canal while the user articulates a predetermined sound and/ or word, or record at least one secondary sample deformation while the user articulates a predetermined sequence of sounds and/ or words.

15. Electronic device (10) comprising: a processing module (131), a memory module (133), and at least one sensor (111,113) which can be inserted in, or positioned in front of, an external ear canal of a user and configured to measure a deformation of the external ear canal, wherein the electronic device (10) is configured to implement the method according to any one of the preceding claims.