WO2018115543A1

WO2018115543A1 - Method and device for biometric authentication by means of blink recognition

Info

Publication number: WO2018115543A1
Application number: PCT/ES2017/070148
Authority: WO
Inventors: Julián ESPINOSA TOMÁS; Begoña DOMENECH AMIGOT; Carmen VAZQUEZ FERRI; Jorge PÉREZ RODRIGUEZ; David MAS CANDELA
Original assignee: Universidad De Alicante
Priority date: 2016-12-20
Filing date: 2017-03-16
Publication date: 2018-06-28
Also published as: ES2610196A1

Abstract

The invention relates to a method and device for biometric authentication by means of the recognition of the blink of a subject, the method comprising the steps of: (a) recording, with a digital video camera, at least one blink sequence of at least one subject; and (b) analysing changes in the intensity of the light diffused by at least one eye, said changes being captured by the digital video camera, and the corresponding blink of said eye, in at least one subject. The method also comprises a step of cinematic and dynamic blink characterisation, said characterisation being configured to identify at least one subject by means of a classification algorithm.

Description

METHOD AND DEVICE OF BIOMETRIC AUTHENTICATION THROUGH THE

BLINKING RECOGNITION

Object of the invention

The object of the present invention is a method and a device for the authentication of the identity of a human being from the recognition of the blink of said human being in a previously recorded video sequence. State of the art

Biometric authentication is the automatic study for the unique recognition of humans based on one or more biometric identifiers that are classified as behavioral traits or physiological traits [Jain, Añil K .; Ross, Arun (2008). "Introduction to Biometrics." In Jain, AK; Flynn; Ross, A. Handbook of Biometrics. Springer pp. 1-22. ISBN 978-0-387-71040-2].

The physiological features are related to intrinsic physical characteristics of the body, such as the fingerprint, veins or fingerprint of the palm, face, DNA, iris, retina, electroencephalogram (EEG) or electrocardiogram (ECG). On the other hand, behavioral traits are related to a person's pattern of behavior, such as the rhythm of writing, signature or voice.

The eye movements [M. Juhola, Y. Zhang, J. Rasku, "Biometric verification of a subject through eye movements," Computers in Biology and Medicine 43, pp 42-50 (2013)] and flickering [M. Abo-Zahhad, Sabah M. Ahmed, Sherif N. Abbas, "A novel biometric approach for human Identification and verification using eye blinking signal," IEEE signal processing letters, 22, No. 7 pp 876-880 (2015)] have been Recently used as physiological traits suitable for human biometric authentication. In both cases, the bioelectric signals from which the parameters that characterize and identify each individual are derived are obtained from the recording of an electrooculogram (EOG) derived from the EEG, although in the case of eye movements, they can also be obtained from camcorders (videograph). The fact that electrodes glued to the skin are needed to measure the EOG, makes it impractical for biometric use.

Traditionally, flickering has been evaluated primarily through contact techniques that require the use of electrodes to measure the EOG [D. Denney and C. Denney, "The eye blink electro-oculogram.," Br J Ophthalmol 68, pp. 225-228 (1984)] or the electromyogram [BWOD Visser and C. Goor, "Electromyographic and reflex study in idiopathic and symptomatic trigeminal neuralgia: latency of the jaw and blink reflexes," J Neurol Neurosurg Psychiatry 37, pp. 1225-1230 (1974)], or the application of a magnetic coil [J. Schlag, B. Merker, and M. Schlag-Rey, "Comparison of EOG and search coil techniques in long-term measurements of eye position in alert monkey and cat," Vision Research 23, pp. 1025-1030 (1983)]. However, it is also possible to use contactless registration procedures, such as photography or video that allow a quantitative evaluation of eye movement during blinking without interfering with the subject [SH Choi, KS Park, MW Sung, and KH Kim , "Dynamic and quantitative evaluation of eyelid motion using image analysis," Med Biol Eng Comput 41, pp. 146-150 (2003)].

The characteristics of the blink that have been studied the most are the frequency and duration due to their relationship with mental states such as fatigue, attention spans and stress. The determination of the beginning and end of the blink is usually addressed by defining precalibrated thresholds. In fact, some method that accurately calculates or determines the end of the blinking is unknown [F. VanderWerf, P. Brassinga, D. Reits, M. Aramideh, and B. O. de Visser, "Eyelid Movements: Behavioral Studies of Blinking in Humans Under Different Stimulus Conditions," Journal of Neurophysiology, 89, pp. 2784-2796 (2003)].

Explanation of the invention It is an object of the present invention to identify and authenticate human beings from a video sequence of their flickering. For this, the present invention analyzes the intensity changes captured by a digital camera of the light diffused by the eye and its corresponding eyelid when blinking. These changes are directly related to eyelid displacement. From the variation of the eyelid position over time, it is an object of the present invention to calculate a plurality of physical parameters that characterize the kinematics and dynamics of the flickering. These parameters are used to identify each subject in a classification process.

The blinking is a temporary closure of both eyes and involves the movement of the upper and lower eyelids. From a physiological point of view, the blinking keeps the eye hydrated, which allows the distribution of the tear film over the ocular surface, protecting it against external objects. Eyelid movements require simple neural commands and few active forces. Flickering represents a normal phenomenon easily observable and accessible, which reflects the processes of central nervous system activation without voluntary manipulation. Thus, its analysis allows to find any abnormality and if it is derived from a muscular or neuronal abnormality.

Flickering is one of the fastest human reflexes (300-400 ms) therefore, to obtain parameters that properly characterize it from a video, a camera with a capture rate according to that duration is necessary. This camera, in a particular embodiment is a commercial camera, and in another particular embodiment is a camera of a portable electronic device, such as a mobile phone or equivalent, as long as the capture rate is at least 150 frames per second (fps). The intensity of the light diffused by the frontally illuminated eyelid varies depending on its position, being maximum when the eyelid is closed and minimal when it is open. Thus, in a recorded video of a blinking subject, this variation will be reflected as changes in the intensity of the recorded light. In each frame of the flickering sequence, the intensity of the light diffused by the eyelid can be estimated by adding the gray levels of the pixels of the area of interest around each eye. Blinks will appear as peaks in the intensity profile [D. Mas, B. Domenech, J. Espinosa, J. Pérez, C. Hernández, and C. Illueca, "Noninvasive measurement of eye retraction during blinking," Optics Letters 35, 1884 (2010)]. Using a peak detection algorithm, each blink is isolated and adjusted to a smoothed spline curve to eliminate the effect of noise. The first and second derivative with respect to the time of this curve are related, respectively, to the speed (first derivative) and the acceleration (second derivative) and its product is proportional to the power developed by the muscles responsible for the blinking. These curves are used to determine different kinematic and dynamic parameters that characterize the blink of each individual.

The kinematic and dynamic parameters obtained are suitable for the biometric authentication of a human being through classification algorithms, since they describe physiological features related to flickering.

Classification is a type of supervised machine learning based on a set of training data containing observations whose membership in a category is previously established. The classification algorithms take advantage of the discriminant information of that training set and learn to classify a new observation in one of the categories or classes. In the present invention, the technical problem that is solved at this stage of the method is the assignment of a blink, ie a new observation, to a class selected from a plurality of classes, ie the subjects, that is, the technical problem that is Solve is a multi-class classification.

The extraction of a set of characteristic parameters that are capable of preserving the discriminant information of each subject allows a new observation of a blink to be assigned to one of the subjects that make up the training set by classification.

In a particular embodiment of the invention, different classification algorithms selected from: a linear and quadratic discriminant analysis (LDA and QDA) have been evaluated; K-Nearest Neighbors (KNN), Classification Tree (CT) and Normalized Cross Correlation (NCC). However, in a preferred embodiment of the invention a linear discriminant analysis LDA has been used where a correct identification rate of up to 99% has been obtained.

Throughout the description and the claims the word "comprises" and its variants are not intended to exclude other technical characteristics, additives, components or steps. For those skilled in the art, other objects, advantages and features of the invention will be derived partly from the description and partly from the practice of the invention. The following examples and drawings are provided by way of illustration, and are not intended to restrict the present invention. In addition, the present invention covers all possible combinations of particular and preferred embodiments indicated herein.

BRIEF DESCRIPTION OF THE FIGURES Next, a series of drawings that help to better understand the invention and that expressly relate to an embodiment of said invention which is presented as a non-limiting example thereof is described very briefly.

FIG. 1 shows a scheme of the elements involved in capturing a scene.

FIG. 2 shows a scheme detailing the selection of a region of interest in each frame and how intensity varies over time in a blink in that region of interest.

FIG. 3 shows the normalized power curve obtained for an example blink. The intersections with zero, the local maximums and minimums and the areas under the curve between intersections are used to define characteristic parameters of the blinking.

FIG. 4 shows the normalized velocity and acceleration curves for an example blink. Local highs and lows and the areas under the curve between intersections are used to define characteristic parameters of the flickering.

FIG. 5 shows the eyelid displacement curve for an example blink.

Detailed statement of an embodiment of the invention

As can be seen in detail in the attached figures, the first stage of the method object of the present invention consists in the recording of different sequences by means of a video camera with a capture speed greater than 150 fps, and where the subjects that they form the set of classes to identify blink. A scheme of the capture system configuration is shown in FIG. 1. Thus, given a subject (1), a camera (2) with a capture rate greater than 150 fps records a sequence (3) of undetermined duration in which the subject (1) flashes. In each frame the eyelid appears in a different position. The measurement is carried out without contact with the subject (1) and can even be done unconsciously for the subject, which facilitates the use of the invention in applications where it is not necessary to require the collaboration of the subject (1).

To accurately assess the flickering it is necessary to record a high-speed video, i.e. with a capture speed such that it allows capturing the eyelid position difference in a blink whose duration is between 300 and 400 ms. If the video is not high speed, the difference in eyelid position between frames is too large to accurately track the eyelid.

Once the capture has been carried out, it is processed as shown in FIG. 2. Given the sequence (3) a region of rectangular interest (4) is selected in the first frame of this around each eye. This selection can be made manually or automated using an eye detection algorithm. The same region of interest is select in the rest of the frames of the sequence and its function is to cut the frames to lighten the processing.

The ocular detection algorithm is based on the difference in light absorption between the eyelid and the open eye (the pupil, the iris and the sclera). Visible light, as well as infrared radiation, is absorbed by the pupil and iris considerably more than is absorbed by the eyelid, as described in [M. Durkin, L. Prescott, C. J. Jonet, E. Frank, M. Niggel, and D. A. Powell, "Photoresistive Measurement of the Pavlovian Conditioned Eyelid Response in Human Subjects," Psychophysiology 27, pp. 599-603 (1990)]. As a result, the energy in the region of interest of the "open eye" image is lower than in the same region of interest of the "closed eye" image (5).

The energy contained in each region of interest of each frame is obtained by adding the value of gray levels of each pixel of it. The amount of intensity reflected by the eye is almost constant when the eyelid is open. Blinks appear as rapid increases and decreases in intensity: when the eyelid closes, the light diffused by the eyelid changes and the same occurs with the intensity recorded by the camera. The intensity peaks represent the moment when the eyelid is completely closed. Using a peak detection algorithm, each blink is cut from the sequence from 0.25 seconds before the peak to 0.46 seconds after. This involves conditioning the interval between flashes to be greater than 0.67 seconds and that the duration of the blinking is less than that value. This selection covers the entire range of flicker duration for normal subjects (50-500 ms) as indicated in [P. P. Caffier, U. Erdmann, and P. Ullsperger, "Experimental evaluation of eye-blink parameters as a drowsiness measure," Eur J Appl Physiol 89, 319-325 (2003)]) and incomplete and / or double blinks are ruled out .

The intensity curve obtained, directly related to the displacement of the eyelid (15) is smoothed by means of spiines (16) to then calculate the first and second temporal derivative and their product. These magnitudes are proportional to the speed (10) and acceleration of the eyelid (1 1) and to the power developed by the muscles in the process (9) respectively.

FIG. 3 shows the normalized power curve (9) obtained for a sample flicker. The standardized power curve (9) allows to clearly define the beginning at the first moment when it ceases to be zero and the end at the last, which returns to zero. Likewise, it is possible to locate the instants in which local maximums and minimums (8) occur, as well as the zero intersections (7). All these, together with the values of the local maximums and minimums (8) provide information on the flicker and are used as characteristics to describe it.

Chronologically, a few hundredths of a second after the blink has begun, the total power developed by the muscles is maximum at the time t _{1 P} in the closing phase. Then, at t ₂ p, the eyelid muscles stop working, the power is zero and the eyelid achieves a maximum closing speed. Then, the eyelid starts braking and the power develops with the opposite sign. There is a moment (t _3P ), when the curve reaches the minimum, which corresponds to the maximum power developed to stop the eyelid closure. Then, the power decreases in absolute value, until it returns to zero. This moment (t _4P ) corresponds to the closed eye, when the closing phase ends and the opening phase begins.

The shape of the curve in the opening phase is similar to the closing. The total power developed by the muscles reaches a local maximum at t _5P that occurs when the eyelid is in the ascending phase. Then, the power decreases until it is zero at t ₆ p and the eyelid reaches a maximum speed. After that, the sign of the force changes when the eyelid is braking and the curve reaches a local minimum at t _7P . At that time, the eye is not yet fully open. Finally, the power decreases in absolute value to zero (t ₈ p), when the eyelid retracts again, the muscle forces are compensated, the eye is open and the blinking ends. The area under the standardized power curve (9) W¡ ^ is related to the work done by the muscles in a period of time between t _c and t _d . Said area between zero intersections is calculated, according to figure 3, obtaining four more parameters. In figure 4, the normalized speed (10) and acceleration (1 1) are represented together with the previously defined instants. It can be seen that the zeroes and the local maximums and minimums of the velocity have already been characterized, while the local maximums and minimums of the acceleration provide new instants of time (12) not yet defined.

Chronologically, t _1a is the time after starting the blinking when the eyelid is in the closing phase and reaches a maximum in acceleration. Then, after the maximum in the developed power, a maximum is reached in the speed and a zero in acceleration, after which the total force of the muscles that intervene in the blinking brakes to the eyelid (there is a change in the sign of acceleration). This braking force peaks at t _2a , before completely closing the eye. Then, in the opening phase, the dynamics are similar. The force accelerates the eyelid to a maximum in t _3a . Later, the force decreases and probably reaches a local minimum that corresponds to the time when the acceleration of braking of the eyelid opening in the ascending phase is maximum, just before stopping it. However, contrary to what happens in the power curve, this braking phase does not clearly appear in the acceleration graphs, so that local minimum cannot be defined. Proceeding with an analysis similar to that of the power, the absolute values of the local peaks of both acceleration and velocity (13), and the areas under the acceleration curve (14) have been obtained. The area under the acceleration curve in a time interval is denoted by j and represents the mechanical impulse per unit of mass developed by the muscles in that period of time between t _c and t _d . Three parameters related to the impulse between the intersections of the curve with zero are defined.

Finally, in Figure 5, the curve adjusted from the displacement data (16) is analyzed. Having defined S = e ₁ / e ₂ , the ratio between average speeds of the closing and opening processes (17), and the half-height width of the curve, w, (18).

The characteristics that describe the dynamics and kinematics of the flicker are grouped into a vector in the order indicated in table 1.

Vector Description Phase

U Start (1 ^at power ≠ zero)

UP Local maximum power

Up Zero power cross Close

Up Local minimum power

Weather(

UP Closed

s)

UP Local maximum power

UP Zero power crossing Opening

UP Minimum local power

UP End (Zero Power)

P (t _1P )

Closing

(t _3P ) \

Normalized absolute power

Opening

p (t _7P ) |

w ₀ ^tip Work From 0 to _2P

_W t4P Close

(ua) Of t ₂ pa

Table 1 . Flashing parameters

The proposed technique has been evaluated with videos obtained with a sports camera (type GOPRO®) recording at 240 fps on 26 subjects. 3251 blinks were obtained from the recorded video sequences, ranging from 74 to 191 blinks per subject. The difference in the number of blinks per subject is due to losses in the processing of signals due to overlapping of blinks or incomplete blinks. However, at least 74 blinks were obtained from all subjects, so 74 blinks of each participant were randomly selected to obtain a set with the same number of data for each subject. Thus, the number of blinks is reduced to 1924. With this set the classification for biometric authentication has been carried out.

The performance of each classifier (LDA, QDA, KNN, CT and NCC) is evaluated through cross-validation of 10 iterations. The data set has been distributed proportionally into 10 disjoint subsets. Nine subsets are used for training and the last subset is evaluated. The process is repeated 10 times, each time leaving a different subset for evaluation. Five sets of data have been evaluated through cross-validation of 10 iterations: the original set of 1924 eyelids and four additional sets obtained from the original. The definition of the additional sets consists of the generation of 100 flashes for each participant. Each blink is constructed with the arithmetic mean of β blinks randomly selected from 74 trials of each participant, with β = 3, 5, 10 and 25 for each set (named β-media). Biometric classifiers were compared to through the correct identification rate in table 2.

Table 2. Correct identification rate (%)

Claims

1. - A method for biometric authentication by recognizing the blink of a subject, where said method comprises the steps of:

(a) record with a digital video camera at least one flickering sequence of at least one subject;

(a.1) and where the flickering sequence comprises a plurality of frames;

(b) analyze the intensity changes captured by the digital video camera of the light diffused by at least one eye and its corresponding eyelid of at least one subject;

and that is characterized because

it comprises a stage of kinematic and dynamic characterization of the blinking; said characterization being configured to identify at least one subject by a classification algorithm.

2. - The method according to claim 1 wherein for each frame the intensity of light diffused by the eyelid is estimated by adding the gray levels of some pixels within an area of interest around each eye.

3. The method according to claim 2 wherein each blink is defined as a peak in an intensity profile and where by means of a peak detection algorithm each blink is isolated and adjusted to a smoothed blink curve and eliminating noise .

4. The method of claims 1 to 3 wherein the kinematic and dynamic characterization of the blinking comprises the calculation of: (i) the eyelid velocity by first derivative of the flickering curve; (ii) acceleration of the eyelid by the second derivative of the flickering curve; and where the product of the speed and acceleration of the blinking is proportional to the power developed by the muscles responsible for the blinking.

5. The method according to claims 1 to 4 wherein the classification algorithm assigns a blink to a subject; and where said classification algorithm is one selected from: a linear and quadratic discriminant analysis (LDA and QDA); K-Nearest Neighbors (KNN), Classification Tree (CT) and Normalized Cross Correlation (NCC).

6. The method according to claim 5 wherein the classification algorithm is preferably a linear discriminant LDA analysis.

7. A biometric authentication device by recognizing the flashing of a subject characterized in that it comprises means for executing the method according to any of claims 1 to 6.