Classifications

• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
  • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
  • A61B3/117—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for examining the anterior chamber or the anterior chamber angle, e.g. gonioscopes
  • A61B3/1173—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for examining the anterior chamber or the anterior chamber angle, e.g. gonioscopes for examining the eye lens
  • A61B3/1176—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for examining the anterior chamber or the anterior chamber angle, e.g. gonioscopes for examining the eye lens for determining lens opacity, e.g. cataract
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B5/00—Measuring for diagnostic purposes; Identification of persons
  • A61B5/117—Identification of persons
  • A61B5/1171—Identification of persons based on the shapes or appearances of their bodies or parts thereof
  • A61B5/1176—Recognition of faces
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
  • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
  • A61B3/107—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for determining the shape or measuring the curvature of the cornea
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
  • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
  • A61B3/14—Arrangements specially adapted for eye photography
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
  • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
  • A61B3/14—Arrangements specially adapted for eye photography
  • A61B3/15—Arrangements specially adapted for eye photography with means for aligning, spacing or blocking spurious reflection ; with means for relaxing
  • A61B3/152—Arrangements specially adapted for eye photography with means for aligning, spacing or blocking spurious reflection ; with means for relaxing for aligning
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
  • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
  • G06V40/18—Eye characteristics, e.g. of the iris
  • G06V40/19—Sensors therefor
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
  • G06V40/40—Spoof detection, e.g. liveness detection
  • G06V40/45—Detection of the body part being alive
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V10/00—Arrangements for image or video recognition or understanding
  • G06V10/10—Image acquisition
  • G06V10/12—Details of acquisition arrangements; Constructional details thereof
  • G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
  • G06V10/141—Control of illumination

Definitions

• This disclosure generally relates to computer-based authentication.
• Implementations of the present disclosure include computer-implemented methods for detecting facial liveliness.
• methods include actions of processing first and second facial images of a subject to determine first and second corneal reflections of an object, the first and second facial images being captured at first and second sequential time points; determining a corneal reflection change of the object based on the determined first and second corneal reflections; comparing the determined corneal reflection change of the object to a known change associated with the first and second time points; and determining facial liveliness of the subject based on a result of the comparison.
• the known change can be a motion of a camera device or an illumination change of an illuminator.
• Implementations of the present discl osure can also include computer- implemented methods for detecting facial liveliness that include actions of processing first and second facial images of a subject to determine first and second corneal reflections of an object, the first and second facial images being captured at first and second sequential time points; determining a corneal reflection change of the object based on the determined first and second corneal reflections; comparing the determined corneal reflection change of the object to a motion associated with the first and second time points; and determining facial liveliness of the subject based on a result of the comparison.
• comparing the determined corneal reflection change of the object to a motion includes correlating the determined corneal reflection change of the object to the motion: scoring a matching quality based on a result of the correlation; and comparing the scored matching quality to a predetermined threshold.
• Determining facial liveliness of the subject includes determining that the face of the subject is live in response to determining that the scored matching quality is beyond the predetermined threshold.
• the actions further includes determining an expected corneal reflection of the object at the second time point based on the determined first corneal reflection and the motion; and assessing the facial liveliness of the subject by determining likelihood between the expected corneal reflection and the determined second corneal reflection.
• the corneal reflection change of the object includes a position change of the corneal refection of the object.
• the motion can be associated with a movement of the object between first and second positions, and the first facial image can be captured at the first time point when the object is at the first position and the second facial image can be captured at the second time point when the object is at the second position.
• the actions can further include receiving information of the movement of the object measured by a sensor; and determining the motion based on the recei v ed information of the mo vement of the object.
• the actions include prompting the subject to move the object.
• the movement of the object is associated with a natural motion of the subject.
• the actions are executed by one or more processors that can be included in a computing device.
• the computing device can include the object and the sensor, and the object can be a camera device and the sensor can include at least one of an accelerometer, a gyroscope, or a global positioning system (GPS).
• the actions can further include
• the motion is a predetermined motion for the object, and the object is moved from the first position to the second position based on the predetermined motion.
• the first and second facial images include a second object that is static during the movement of the object, and the actions further include determining the motion based on a position change of the second object in the first and second facial images.
• the actions can include transmitting a command to a controller coupled to the object, the command indicating the controller to move the object.
• the command can include a predetermined motion for the object, and wherein the predetermined motion is different from a previous predetermined motion for the object.
• the actions include transmitting a command to a camera device to capture facial images of the subject at sequential time points including the first and second time points: and receiving the captured facial images from the camera device.
• Receiving the captured facial images from the camera device can include receiving a video stream feed from the camera device, the video stream feed comprising the captured facial images.
• the camera device is moved from a first position to a second position between the first and second time points. The first facial image of the subject is captured at the first time point when the camera device is at the first position, and the second facial image of the subject is captured at the second time point when the camera device is at the second position, and the motion is based on a movement of the camera device between the first and second positions.
• the actions include transmitting a second command to a controller coupled to the object, the second command indicating the controller to move the object at the first and second time points, and the motion is based on the movement of the camera device and the movement of the object.
• the object can include one of a camera device, an illumination device, or an object brighter than ambient environment.
• the corneal reflection change of the object can include a first illumination change on the object in the first and second corneal reflections.
• the actions further include obtaining a second illumination change on the object for the first and second time points: and determining a matching quality between the obtained second illumination change on the object and the first illumination change in the first and second corneal reflections.
• Determining facial liveliness of the subject can include determining the facial liveliness of the subject based on the determined matching quality and the result of the comparison.
• the actions can include determining that the subject chooses to use liveliness verification for biometric authentication.
• the actions can also include transmitting, to a computing device at an identity provider, an authentication request of the subject for accessing an account managed by a relying party different from the identity provider, the authentication request including biometric data of the subject and the determined facial liveliness of the subject.
• Implementations of the present disclosure can also include computer- implemented methods for detecting facial liveliness that include actions of processing first and second facial images of a subject to determine first and second poses of a face of the subject, the first and second facial images being captured at first and second sequential time points; determining a change in pose of the face based on the determined first and second poses; comparing the determined change in pose of the face to a motion associated with the fi rst and second time points; and determining facial liveliness of the subject based on a result of the comparison.
• comparing the determined change in pose of the face to a motion includes correlating the determined change in pose of the face to the motion; scoring a matching quality based on a result of the correlation; and comparing the scored matching quality to a predetermined threshold.
• Determining facial liveliness of the subject includes determining that the face of the subject is live in response to determining that the scored matching quality is beyond the predetermined threshold.
• Determining first and second poses of the face can include determining at least one facial landmark of the face.
• the actions include calculating a pose of the face at the second time point based on the determined first pose and the motion; and assessing facial liveliness by determining likelihood between the calculated pose of the face to the determined second pose of the face from the second facial image.
• the motion can be associated with a relative movement between the face and a camera device configured to capture facial images of the subject.
• the actions include prompting the subject to move the camera device relative to the face of the subject.
• the actions include receiving information of the movement of the camera device measured by a sensor; and determining the motion based on the received information of the movement of the camera device.
• the actions are executed by one or more processors that can be included in a computing device, and the computing device can include the camera device and the sensor, and the sensor includes at least one of an accelerometer, a gyroscope, or a global positioning system (GPS).
• the computing device can include the camera device and the sensor, and the sensor includes at least one of an accelerometer, a gyroscope, or a global positioning system (GPS).
• GPS global positioning system
• Implementations of the present disclosure can also provide computer- implemented methods for detecting facial liveliness that include actions of processing a facial image of a subject to determine a corneal reflection of an illuminator adjacent to the subject, the facial image being captured at a time point, determining an expected corneal reflection of the illuminator based on an illumination of the illuminator at the time point, comparing the determined corneal reflection of the illuminator to the expected corneal reflection of the illuminator to obtain a comparison result, and determining facial liveliness of the subject based on the comparison result,
• comparing the determined corneal reflection of the illuminator to the expected corneal reflection of the illuminator can include: correlating the determined corneal reflection of the illuminator to the expected corneal reflection of the illuminator; scoring a matching quality based on a result of the correlation; and comparing the scored matching quality to a predetermined threshold.
• Determining facial liveliness of the subject can include determining that the face of the subject is live in response to determining that the scored matching quality is beyond the predetermined threshold.
• Determining a corneal reflection of the illuminator can include determining the corneal reflection of the illuminator in at least one of: color, shape, position, or pattern.
• the actions can include activating the illuminator to illuminate based on an illumination pattern. In some examples, the actions include generating the illumination pattern for the illuminator. In some examples, activating the illuminator comprises:
• the illuminator can include two or more illumination sources with respective colors and at respective positions, respectively.
• the illumination of the illuminator is based on a temporal illumination pattern of the two or more illumination sources.
• the two or more illumination sources can generate illumination in bursts each with respective temporal durations. At least one of the two or more illumination sources can have different illumination shapes for different temporal durations.
• the illumination of the illuminator is based on a spatial illumination pattern of the two or more illumination sources. Each of the two or more illumination sources can have an illumination on or off state.
• the illuminator includes one or more physical light sources each at respective light wavelengths.
• the illuminator includes one or more digital illumination sources displayed on a screen to provide respective illumination colors.
• the actions can include generating the digital illumination sources.
• the illuminator can be positioned such that illumination from the illuminator is incident on an eye of the subject at an angle.
• the actions can include prompting the subject to change a relative position of the illuminator and the eye of the subject.
• the actions can include receiving at least a portion of a video stream feed from a camera device, the portion of the video stream feed including facial images of the subject captured by the camera device at sequential time points.
• the action can further include: processing a second facial image of the subject to determine a second corneal reflection of the illuminator, the second facial image being captured at a second time point sequential to the time point for the first facial image; determining a second expected corneal reflection of the illuminator based on a second illumination of the illuminator at the second time point, the second illumination being different from the first illumination; comparing the determined second corneal reflection of the illuminator to the expected second corneal reflection of the illuminator to obtain a second comparison result; and determining facial liveliness of the subject based on the comparison result and the second comparison result.
• the actions can include transmitting the determined facial liveliness of the subject to a computing device at identity provider.
• the actions can also include transmitting, to a computing device at identity provider, an authentication request of the subject for accessing an account managed by a relying party different from the identity provider, the authentication request including biometric data, of the subject and the determined facial liveliness of the subject.
• Implementations of the present disclosure can also provide computer- implemented methods for detecting facial liveliness that include actions of processing first and second facial images of a subject to determine first and second corneal reflections of an illuminator adjacent to the subject, the first and second facial images being captured at first and second sequential time points, respectively; determining a corneal reflection change of the illuminator based on the determined first and second corneal reflections; comparing the determined corneal reflection change of the illuminator to an illumination change of the illuminator associated with the first and second sequential time points; and determining facial liveliness of the subject based on a result of the comparison.
• the actions can include activating the illuminator to change illumination of the illuminator based on the illumination change.
• the illumination change of the illuminator can include at least one of: an illumination color change, an illumination shape change, an illumination position change, an illumination on/off state change, or an illumination temporal duration change.
• Determining a corneal reflection change of the illuminator can include determining a change of the corneal reflection of the illuminator in at least one of: color, shape, position, on/off state, or temporal duration.
• the illuminator can include at least one of: one or more physical light sources each at respective light wavelengths, or one or more digital illumination sources displayed on a screen to provide respective illumination colors.
• the illuminator includes first and second illumination sources with first and second colors and at first and second positions, respectively.
• the illumination change of the illuminator includes a change of a temporal illumination pattern of the first and second illumination sources, and the first and second illumination sources can generate illumination in bursts each with respective temporal durations. At least one of the first illumination source or the second illumination source can change an illumination shape for different temporal durations.
• the illumination change of the illuminator includes a change of a spatial illumination pattern of the first and second illumination sources over time, and at least one of the first illumination source or the second illumination source can change an illumination on/off state over time.
• the present disclosure also provides one or more non-transitory computer- readable storage media coupled to one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations in accordance with implementations of the methods provided herein.
• the present disclosure further provides a system for implementing the methods provided herein.
• the system includes one or more processors, and a computer- readable storage medium coupled to the one or more processors having instructions stored the eon which, when executed by the one or more processors, cause the one or more processors to perform operations in accordance with implementations of the methods provided herein.
• FIG. 1 A is a diagram showing an example web site capable of authenticating a user based on a biometric identity of the user according to some implementations.
• Fig. IB is a diagram showing an example interaction among a financial institution customer, a financial institution, and a third-part ⁇ - biometric verification service to authenticate a user based on a biometric identity of the user according to some
• FIG. 2A is a timing diagram showing an example interaction among a consumer, a relying party, and an identity provider in authenticating the consumer when the consumer has enrolled at the identity provider according to some implementations.
• Fig. 2B is a timing diagram showing an example interaction among a consumer, a relying party, and an identity provider in authenticating the consumer when the consumer has not yet enrolled at the identity provider according to some implementations.
• Figs. 3A and 3B illustrate the changed positions of the corneal reflection of an object.
• Fig. 4 illustrates an object that has changed position by virtue of ami adjustments.
• Fig. 5 is a diagram showing an example verification based on corneal reflection from illumination sources.
• biometric data submitted on-line can be verified for liveliness to deter spoofing attempts.
• the liveliness verification can be leverage analysis of real-time corneal reflection and/or real-time face pose to increase confidence in the biometric data being submitted on-line for authentication. Implementations disclosed herein may be extended to enterprise entities other than financial institutions. This application is directed to systems and methods to detect liveliness of a facial representation during, for example, an online session between two parties over a network. Note that the terms “liveliness” and “liveness” can be used interchangeably here.
• Fig. 1A is a diagram showing an example web site capable of authenticating a user based on a biometric identity of the user according to some implementations.
• a user name and password may be required for logging into an account administered by the server hosting the web-site.
• the server may run an implementation of a hypertext transmission protocol secure (https).
• https hypertext transmission protocol secure
• a consumer user may have accounts at dozens of more web-sites, each with disparate requirements of user names and passwords.
• the consumer user may have a hard time keeping track of user names and passwords at each of the web-sites. Losing track of the registered usemame or password can hinder on-line access convenience.
• the consumer user may store the usemames and passwords for each account at a central file. But, access to the central file may be breached, leading to comprises in the stored usemames and passwords.
• simple on-line identities including user name and password may be subject to identity theft and identity fraud.
• identity theft in the United States rose to a three-year high in 2012, with more than 5 percent of the adult population , or 12.6 million people, falling victim to such crimes. The numbers are up from 4.9 percent in 2011 and 4.35 percent in 2010. The incidence of identity theft is only expected to rise.
• biometric data may include, but are not limited to, facial pattern, finger print, palm print, retina scan, iris scan, DNA pattern, voice characteristics, gait analysis.
• biometric data submitted on-line may be in a digital form to facilitate electronic verification.
• biometric login window 1 2 may be provided by a third-party, different from, for example, the entity that runs bigbank.com.
• the login window 102 may present the captured facial biometric of the user as the user attempts to log in based on the user's biometric.
• the biometric data may be encrypted in accordance with industry standard including, for example, data encryption standard (DES), triple-DES, advanced encryption standard (AES), Rivest-Shamir-Adlema (RSA), open pretty good privacy (PGP), etc.
• the encryptions may be performed on a session-by-session basis in which the session encryption and decryption keys are constantly updated or refreshed to thwart attacks.
• Fig. IB is a diagram showing an example interaction among a financial institution customer, a financial institution, and a third-party biometric verification sen/ice to authenticate a user based on a biometric identit 7 of the user according to some implementations.
• the financial institution may include a bank, and may also be known as the relying party.
• the third-party service provider may also be known as the identity provider,
• customer 104 may be greeted with the biometric authentication window 102.
• the financial institution customer 104 may choose to use the biometric authentication.
• the server running the financial institution website 106 may verify the user's authentication request using biometnc data at biometric verification sen/ice 108 (1 10B).
• biometric verification service 108 may engage customer 104 to have the customer enrolled (1 IOC).
• the enrollment process may generally include asking customer 104 to provide a biometric data as the biometric identity data stored/managed by biometric verification service 108.
• the enrollment process may become part of the vetting process in which biometric verification service further verifies the identity of the customer 104 at other authorities, including, for example, Department of Motor Vehicles (DMV) at the state level and the State Department at the federal level.
• the authorities may include a credit verification agency, such as, for example, Equifax, Experian, LexisNexis.
• the credit verification agency may serve as a surrogate of a government-administered authority.
• biometric verification se dee 108 After the customer 104 obtains an enrolled biometric identity at the biometric verification service, customer 104 may interact with biometric verification se dee 108 to conduct routine management of the biometric identity (1 IOC). Routine management may include, for example, replacement of a facial pattern, revocation of an existing facial pattern, etc. Hence, the first issue of a vetting process to register an enrolled biometric data of customer 104 may be addressed.
• the biometric login window 102 may authenticate customer 104 by- analyzing real-time corneal reflection of an object (e.g., a camera device or an illumination source) and/or analyzing real-time face pose from customer 104.
• a financial institution may be generalized as a relying party (RP) who may rely on a third party to authenticate that a customer is who the customer purports to be and that the customer is presenting a live facial biometric.
• the third party known as the biometric verification service 108 in Fig. IB, may be generalized as an identity provider (MP).
• Fig. 2A is a timing diagram showing an example interaction among a consumer 202, a relying party (RP) 204, and an identity provider (IdP) 206 in authenticating the consumer when the consumer has enrolled at the identity provider 206 according to some implementations.
• the process may initiate with consumer 202 send an authentication request to access an account managed by relying party 204 (208).
• the submission of the authentication request may correspond to consumer 202 choosing biometric authentication 102 to access an account managed by, for example, a financial institution.
• the consumer may also choose a verified liveliness login for biometric authentication (210).
• the verified liveliness login may address the liveliness of biometric data being submitted online, e.g., by verifying facial liveliness of the consumer when the consumer submits the biometric data.
• SAML Security Assertion Markup Language
• OAuth Open standard for Authentication
• SAML addresses web browser single sign-on (SSO).
• Single sign-on solutions may be implemented at the intranet level using, for example, cookies. Extending such single sign-on solutions beyond the intranet has been problematic and has led to the proliferation of non-mteroperabie proprietary technologies.
• the user also known as the principal
• the service provider requests and obtains an identitv 7 assertion from the identity provider. On the basis of this identity assertion, the service provider can make an access control decision -deciding whether to perform some service for the connected principal.
• the web application may specifically request a limited access OAuth Token (also known as the valet key) to access the Application Program Interfaces (APIs) on behalf of the user.
• OAuth Token also known as the valet key
• the valet token may explicitly name the particular rights requested, and the user may not be required to enter a credential. If the user can grant that access, the web application can retrieve the unique identifier for establishing the profile (identity) using the APIs.
• authentication standards may generally provide a single sign-on so that a consumer can use, for example, the consumer's Facebook or Google login credentials to access other accounts (for example, an amazon account, a linked-in account, etc.), typically of the same consumer.
• Some authentication methodologies as disclosed herein may provide a single authentication proof of identity by using biometric data, for example, using facial biometric.
• a sen/er at the relying party may redirect consumer to proceed with verified liveliness (212). Thereafter, consumer 202 may be redirected to a server at the identity provider 2,06. For example, consumer 202, may submit a request to use verified liveliness by using a credential token in the form ofconsumer@relying_party.com (214). As illustrated in Fig. 2A, the request may be submitted at a server at identity provider 206.
• the server at identity provider 206 may perform verified liveliness authentication to verify the biometric submitted by the consumer (216).
• the verification may include that (i) the biometric submitted by the consumer provides affirmative proof of identity for the consumer: and (ii) the biometric submitted by the consumer on-line is fresh and live and not is being forged, for example, in a spoofing attempt.
• the server at identity provider 206 may retrieve, for example, a facial biometric from the data submitted by consumer 202.
• the retrieved facial biometric may be sent, from a computing device of consumer 202 to the server at identity provider 206, in an encrypted format based on an encryption algorithm.
• the computing device may include a video input component to provide a video stream feed.
• the video input component includes a webcam. Images in the video feed may be captured at a frame rate of, for example, 24 frames per second. The frames generally are digitized for digital processing.
• the images may be at a spatial resolution of, for example, 800x600 pixels, 1024x768 pixels, 1152x864 pixels, or 1280x1024 pixels.
• Hie computing device may include an imaging device (e.g., a camera device) that captures images of consumer 202.
• the captured images include facial biometric of consumer 202, and can be transmitted by the computing device to the server at identity provider 206.
• the retrieved facial biometric is sent from a computing device associated with the relying party 204 or the identity provider 206.
• the computing device can internally include or be externally coupled to a camera device that is local to consumer 202 when consumer 202 is seeking authentication.
• the camera device can be adjacent to an automatic teller machine (ATM) facing consumer 202 or within a registration office of a bank.
• ATM or the office may be a property of the relying party 204 with whom consumer 202 seeks to perform a transaction .
• the camera device can capture images or a video feed that includes facial biometric of consumer 202 and transmit to the computing device.
• the computing device can process the captured images or video feeds and transmit the captured images or video feeds and/or the processed data to the server at identity provider 206.
• the server at identity provider 206 may compare the retrieved facial biometric with biometric identity data of consumer 202.
• the biometric identity data of consumer 202 may be pre-stored at the server at identity provider 206. Hie comparison may be conducted m a manner consistent with the context-dependent rigidity requirement of the authentication. For example, if the amount in transaction is less than a particular threshold, such as $50, then the degree of matching between the facial biometric as submitted by consumer 202 on-line and the pre-stored facial biometric at the server at the identity provider 206 may be reduced.
• the degree of matching between the facial biometric as submitted by consumer 202 on-line and the pre-stored facial biometric at the server at the identity provider 205 may be heightened.
• the degree of matching may also be heightened.
• a scoring mechanism may be implemented to generate a score to quantify the degree of resemblance between the facial biometric as submitted by consumer 202 on-line and the pre-stored facial biometric at the server of identity provider 206.
• the score of resemblance may be used by the server at identity provider 206 to determine whether the retrieved facial biometric as submitted by consumer 202 on-line is sufficient to prove that the submitting consumer 202 is the person whose biometric data is stored at the server of identity provider 206.
• the server at identity provider 206 may determine whether the retrieved facial biometric being submitted by consumer 202 on-line is a live facial biometric.
• One risk associated with on-line authentication has been the prospect of a man-in-the-middle attack.
• a third party may impersonate consumer 202 by replaying an authentication proof as submitted by consumer 202 earlier and was obtained by the third party.
• Some encryption mechanism may counter the man-in-the-middle threat by refreshing or updating the encryption key used to encrypt the stream of communication sessions between consumer 202 and identity provider 206. Generally, however, such encryption mechanism may not address the staleness of the biometnc data itself.
• some encryption mechanism may be inadequate if a third party obtains an instance of the biometric data and attempts to replay the instance of biometric data to gain access to the account of consumer 202.
• Such risks may be amplified with the combination of username and password login because the text strings for username and password may not be refreshed at the granularity of each login session. Specifically, refreshing username and password for each login session (or at a sub-session level at, for example, every few seconds) may not be practical.
• Some implementations as disclosed herein may verify the livelin ess of the biometric data as submitted by consumer 202, for example, by performing verification of facial liveliness of consumer 202.
• the server at identity provider 206 performs liveliness verification based on the submitted images or video feeds.
• a computing device associated with consumer 202 performs liveliness verification based on the captured images or video feeds and submits a result of liveliness verification to the server at identity provider 206, e.g., in an authentication request including the biometric data of consumer 202.
• the server at identity provider 206 can then seamlessly determine that the biometric data submitted by consumer 202 is live.
• a computing device associated with relying party 204 or identity provider 206 that obtains images or video feeds from a camera device local to consumer 202, performs liveliness verification based on the captured images or video feeds and submits a result of liveliness verification to the server at identity provider 206.
• liveliness verification may include analysis of a corneal reflection from consumer 202. Initially, the face of consumer 202 may be detected. Referring to Figs. 3 A and 3B, the eye 301 of consumer 202 may then be detected in a video feed or images. Generally, such detections include segmentation techniques that leverage face or eye template data.
• Such template data may be morphed through various shape transforms to identify face or eye area in the video feed or images.
• cornea region 302 may then be determined in the video feed or images. Reflections of the corneal region in the video frames or images may then be analyzed.
• an object at the location of consumer 202 may cause corneal reflection from consumer 202.
• the object may include an illumination source such as a light source, a camera device such as a webcam, or a device or subject (e ,g., a window or a table) brighter than the ambient environment.
• the object is a camera in a mobile device of consumer 202.
• Consumer 202 uses the camera to capture facial images or biometric of consumer 202.
• the object may be in a place local to consumer 202, The place may be associated with relying party 204 with whom consumer 202 seeks to perform a transaction.
• the place includes an automatic teller machine (ATM) facing consumer 202.
• ATM automatic teller machine
• the object may be fixed or positioned to the locale where consumer 202 is seeking authentication.
• the object is readily visible under normal illumination condition and can cause specular reflections from consumer 202 on-site.
• the specular reflection off the cornea of consumer 202 may be leveraged to determine liveliness of the authentication.
• the specular reflections of such object in the corneal of consumer 202 are expected to move accordingly. If the specular reflection of such object moves in concert with the movement of the object on-site, then consumer 202 can be authenticated as live. In some cases, the movement of the object is performed by consumer 202. For example, consumer 202 is prompted to move a mobile phone including a camera device, and the corneal reflection of the came a device is moved accordingly. The movement of the camera device on-site can be measured by an on -board accelerometer or gyroscope in the mobile device. In some cases, the object is moved in a manner unpredictable by consumer 202 (or would-be hackers as man in the middle).
• the object can be moved from position 1 to position 2.
• a first image is captured.
• the location, shape, and color/wavelength of the reflected object may be detected by analyzing the corneal reflections in the captured first image. The detection can leverage segmentation techniques to automatically zero in on the corneal reflection 302A of the object.
• a second image is captured.
• the location, shape, and color/wavelength of the reflected object may be detected anew as corneal reflection 302B by analyzing the captured second image.
• the trajectory of the motion of object may be tracked by monitoring the reflected object.
• changes in the location of the detected object in corneal reflections 302A and 302B may be compared to the changes of object locations, e.g., programmed or measured location changes of the object.
• some implementations may register the position of the reflected object relative to reflections of other structures adjacent to consumer 202 in a room or in front of an ATM where consumer 202 is seeking authentication.
• Some implementations may register the position of the reflected object relation to other bodily structures in or around the eye of consumer 202.
• Fig. 4 illustrates an example object camera 403 that has been changed position by virtue of arm adjustments of consumer 202.
• Camera 403 is held on arm 402, of consumer 202, and faces head 401 of consumer 202.
• arm 402 is more extended, for example, to be closer to the face of consumer 202.
• arm 402 is more recoiled, for example, to move further away from, the face of consumer 202.
• Such positional changes of camera 403 are expected to cause changes in the reflected camera from cornea of consumer 202.
• the object may rely on a mechanism that leads a changed position of the object.
• the mechanism can be pan, zoom, tilt (PZT), or any other suitable mechanism.
• Motions of an object can cause more interesting changed location and shape of the reflected object in corneal reflections.
• the object may be programmed to move by virtue of a controller such as on-board accelerorneter or gyro motors.
• the object is moved in a predetermmed (or preprogrammed) motion.
• the predetermined motion can be previously embedded in the object.
• the predetermined motion can be also transmitted to the controller, e.g., from the server at identity provider 206 or a computing device coupled to the controller. Each time the predetermined motion can be different from previous
• the object is moved randomly.
• the actual movement of the object can be measured or detected by sensors such as on-board accelerorneter or gyro motors.
• the changes of the object may not be limited to mere positional changes manifested as translational or rotational adjustments.
• the variety of such changes in position and change can be leveraged to provide an enriched analysis of corneal reflections.
• the object can even change to a position to cause the wavelength of corneal reflection to change.
• the object can be translated to deflect
• illumination from a different illumination source facing consumer 202 e.g., a flashlight of a mobile device of consumer 202.
• the object moves, in a pattern known to the identity provider but unknown to consumer 202 (or would-be hackers as man in the middle) beforehand, to cause changes in cornea reflection of the object.
• the object can be moved from a first position to a second position in relatively fast speed compared to the movement of the head or face of consumer 202, such that the movem ent of the head or face of consumer 202 can be considered as relatively static.
• a camera device can capture a first facial image of consumer 202 when the object is at the first position at a first time point and capture a second facial image of consumer 202 when the object is at the second position at a second time point.
• An interval between the first and second time points can be short, e.g., less than 1 second.
• the movement of the head or face of consumer 202 can be leveraged by analysis of movement of static or unmoved subjects (e.g., a window or a table adjacent to consumer 202) in the captured images. If consumer 202 is prompted to move the object, consumer 202 can be notified not to move the head or face of consumer 202.
• a camera device is configured to capture the images and/or video feeds of consumer 202.
• the camera device can be positioned facing a face of consumer 202.
• the camera device can be internally included in a mobile device of consumer 202.
• Consumer 202 can hold the mobile device such that the camera device faces the face of consumer 202 to capture facial images.
• the camera device can be positioned such that the camera device is at a tilted angle to the eye of consumer 202 or opposite to the eye of consumer 202.
• consumer 202 is seeking authentication before an ATM.
• the camera device can be fixed on the ATM or adjacent to the ATM.
• the camera device can capture the images and/or video feeds for verifying facial liveliness of consumer 202, e.g., without interaction with consumer 202 or without notice to consumer 202.
• the movement of the camera device can be associated with a natural motion of consumer 202.
• a sensor e.g., an accelerometer, a gyroscope, or a global positioning system (GPS) on a mobile device including the camera device, can measure the movement of the camera device.
• the mobile device can determine that the camera device is moving by using the sensor and then transmit a request to the camera device to capture facial images of consumer 202.
• GPS global positioning system
• the server at identity provider 206 may transmit a request to the camera device, e.g., directly or through a computing device coupled to camera device and in communication with the server.
• the camera device can automatically capture facial images of consumer 202 when the object is moved between positions.
• the object can be moved based on the request.
• a computing device in communication with the server receives the request and transmits a command to a controller of the object for moving the object.
• the object can be moved between positions in synchronization with the camera device capturing facial images of consumer 202.
• the movement of the object can be associated with respective time points.
• the object is at a first position; at a second time point, the object is moved to a second position.
• the camera device can capture facial images or video feeds of consumer 202 during the movement of the object and/or before and after the movement of the object.
• the captured facial images are also associated with respective time points.
• the time points can have a high accuracy, e.g., less than 1 millisecond.
• the camera device captures a first facial image when the object is at the first position at the first time point, and a second facial image when the object is at the second position at the second time point.
• the captured facial images of consumer 202 can be associated with the positions of the object.
• the corneal reflection of the object in the captured images or video feeds can be associated with the movement of the object, which can be used to verify facial liveliness of consumer 202.
• the camera device can also capture images and/or video feeds of consumer
• biometnc data submitted by consumer 202 to the server at identity provider 206 for authentication, e.g., by interacting with consumer 202 to obtain the biometric data.
• an additional imaging device or a video input component is used to capture images and/or video feeds that include biometric data of consumer 202 for authentication.
• the object is the camera device.
• the camera device can be moved by consumer 202 or by a mechanism in a controlled manner or randomly.
• the motion (and/or orientation and/or relative position) of the camera device can be measured, e.g., by on-board accelerometer or gyroscope.
• the detected position changes in the corneal reflections of the camera device may be correlated to the predetermined or measured motion of the camera device.
• the object is a static subject adjacent to consumer
• the camera device is moved between positions and captures facial images of consumer 202 at these positions.
• the facial images include corneal reflections of the object.
• the detected position changes in the corneal reflections of the object may be correlated to a predetermined or measured motion of the camera device.
• the object is a subject configured to be moved between positions in a motion for liveliness verification, while the camera device is static and captures facial images of consumer 202 when the object is at different positions.
• the facial images include corneal reflections of the object.
• the detected position changes in the corneal reflections of the object may be correlated to a predetermined or measured motion of the object.
• the object is a subject configured to be moved between positions in a first motion pattern, while the camera device is also moved between positions in a second motion pattern. Moving both the object and the camera device for liveliness verification can increase security against hackers.
• the camera device captures facial images of consumer 202 when the object is at different positions and the camera device is also at different positions.
• the facial images include corneal reflections of the object.
• the detected position changes in the corneal reflections of the object may be correlated to a motion pattern based on the first motion pattern (predetermined or measured) of the object and the second motion pattern
• the motion pattern can be a relative motion pattern for the object and the camera device.
• the object is static, and the camera device is also static.
• An illumination on the object from an illuminator can change at different time points, e.g., changing illumination colors or patterns, and the corneal reflection of the object can change correspondingly.
• the detected illumination change on the object in the corneal reflections of the object may be correlated to a known illumination change of the illuminator.
• the object is an illuminator, e.g., a light source and/or a digital illumination source displayed on a screen, e.g., a liquid crystal display (LCD) of a mobile device.
• the camera device can capture facial images of consumer 202 that include corneal reflections of the object.
• the detected illuminator in the corneal reflections of the object may be correlated to a predetermined or measured illumination change of the illuminator. Using illumination for liveliness verification is described with respect to Fig. 5 below.
• the camera device is static and/or the object is also static.
• a computing device coupled to the camera device can prompt consumer 202 to move the head or the face in a motion pattern, e.g., turning from left to right.
• the camera device can capture facial images of consumer 202 during the movement of consumer 202.
• the detected position changes in the corneal reflections of the object may be correlated to a motion of consumer 202.
• the object and/or the camera device can be moved in a motion pattern, and an illumination on the object can also change at different time points, e.g., changing illumination colors or patterns.
• Verification of facial liveliness of consumer 202 can be based on: 1) the correlation between the detected position changes in the corneal reflections of the object and the motion pattern of the object and/or the camera de vice, and 2) the correlation between the detected illumination change on the object in the corneal reflections of the object and the predetermined or measured illumination change.
• the images during the movement of the object and/or the camera device may be captured in a video stream at a full frame rate (for example, as determined by National Television System Committee— NTSC, or as determined by the video communication standard H.261).
• a full frame rate for example, as determined by National Television System Committee— NTSC, or as determined by the video communication standard H.261.
• the video feed may be digitalized and transmitted to the server at identity provider 206 at a rate faster than a floor-level rate (when consumer 202 remains stationary).
• the video feed or images are transmitted to the server at identity provider 206 by the camera device.
• a computing device coupled to the camera device receives the images or video feed from the camera device and transmits the images or video feed to the server at identity provider 206.
• the server at identity provider 206 can perform liveliness verification based on the transmitted images or video feed.
• the server can analyze the images or video feed to determine corneal reflections of the object. Based on the determined corneal reflections, the server can determine a position change in the corneal reflections of the object.
• the server can compare the position change in the corneal reflections of the object to a motion pattern and determine facial liveliness of consumer 202 based on a result of the comparison.
• the motion pattern is a predetermined motion pattern of the object or the camera device or both when moved between positions.
• the server can pre-store the predetermined motion pattern.
• the motion pattern is a measured motion pattern of the object or the camera device or both when moved between positions.
• the motion pattern can be transmitted to the server together with the captured images or video feed.
• the server at identity provider 206 can analyze the received images and/or video feeds to determine the motion pattern, e.g., by analyzing a positon change of a static subject in the captured images and/or video feeds.
• a computing device coupled to the camera device performs liveliness verification based on the captured images or video feeds and submits a result of liveliness verification to the server at identity provider 206, e.g., with or without transmitting the captured images or video feeds.
• the computing device receives biometric data of consumer 202 for authentication and transmits the result of liveliness verification together with the biometric data of consumer 202 to the server at identity provider 206.
• the computing device is associated with relying party 204 or identity provider 206.
• consumer 202 is seeking authentication using an ATM associated with relying part ⁇ ' 204.
• the computing device can be local to the ATM and coupled to the camera device that faces consumer 202.
• the computing device can receive from the server at identity provider 206 a request to verify liveliness of consumer 202.
• the computing device can transmit commands to the camera device and/or the object to ask the camera device to capture images and/or video feeds when the camera device and/or the object is moved between positions in a motion pattern, e.g., unknown to consumer 202 and without input from consumer 202.
• the computing device predetermines the motion pattern, and transmits the motion pattern to the camera device and/or the object.
• the computing device can change the motion pattern each time when performing liveliness verification.
• the motion pattern is a preprogrammed or predetermined motion pattern.
• the computing device can pre-store the motion pattern.
• the motion pattern is measured by sensors when the camera device and/or the object is moved with positions. The computing device can receive the measured motion pattern from the sensors.
• the computing device is associated with consumer 202, e.g., a smartphone or a computer.
• consumer 202 is seeking authentication at home.
• the computing device can intemally include a camera device or externally be coupled to a camera device.
• an application (or an App when the computing device is a smartphone) provided by identity provider 204 in installed in the computing device.
• identity provider 204 Upon receiving a request from the server at identity provider 204 or determining consumer 202 chooses to use liveliness verification for biometric authentication, the application can solicit consumer 202 to move the camera device in relative to the face of consumer 202.
• a sensor e.g., an acceleromeier or gyroscope, the computing device can determine a movement of the camera device.
• the computing device can analyze the captured images and/or video feeds to determine the motion of the camera device, e.g., by analyzing a positon change of a static subject in the captured images and/or video feeds.
• the images of the static subject can be corneal reflections or outside of the corneal area.
• the computing device can perform liveliness verification based on the motion pattern, and the corneal reflections of the camera device in captured images and/or video feeds.
• the application in the computing device controls an illumination source such as a flashlight of the camera device to change illumination color or pattern unknown to consumer 202.
• the application can perform liveliness verifi cation based on the changed color or pattern of the illumination source and the corneal reflections of the camera device in captured images and/or video feeds.
• a scoring mechanism may be implemented to quantify the degree of match between the response corneal reflection of the object and the expected change (e.g., a motion pattern or an illumination change).
• the scoring mechanism may generate a numerical value as a measure for determining liveliness of the face of consumer 202.
• the degree of match may depend on the context and hence can vary from application to application. For example, the degree of match may depend on the nature of the underlying transaction or the dollar amount involved in the underlying transaction.
• corneal reflections of the object in sequential facial images of consumer 202 are detected.
• the sequential facial images are captured at sequential time points by a camera device.
• An expected corneal reflection of the object at a particular time point can be determined based on a determined corneal reflection in a previous time point and a motion between the previous time point and the particular time point.
• the facial liveliness of consumer 202 can be assessed by determining likelihood between the expected corneal reflection and the determined corneal reflection from the captured facial images at the particular time point.
• Corneal reflection may proffer benefits as non-intrusive and capable of handling a wide band of operations (for example, from infra-red to visible light).
• corneal reflection tends to be more specular than skin reflections, which tend to be more scattered.
• Implementations disclosed herein may not include verbal directives to prompt consumer 202 to change facial gestures or other bodily positions.
• pose of the face of consumer 202 may be estimated, for example, usmg an automatic algorithm .
• the automatic algorithm can detect the pose of the face relative to the camera device in each capture image, e.g., from a video feed.
• an expected pose or change in pose of a face may be calculated based on the measured or known positions of the camera device, e.g., by on-board accelerometer or gyro motors or from analyzing the corneal reflections of the camera device.
• consumer 202 is prompted to move the camera device in relative to the face of consumer 202. Such calculation can be performed for each additional image captured from the video feed.
• the expected pose or changes in pose, as calculated from the measured locations of the camera device and the detected pose of the face in the capture image may be correlated in a quantitative manner using a scoring mechanism.
• the numerical value as determined from the correlation can be used to gauge the liveliness of the face.
• pose of the face of consumer 202 is detected or estimated by analyzing one or more facial landmarks of consumer 202.
• the facial landmark features can include the nose, the forehead, the eye brows, the eyes, the eye comers, the lips, and/or the mouths.
• Poses of the face or facial portraits from a live session reflect the face of the subject at various angles. Due to slight changes in viewing angles of the pose or facial portrait, the apparent orientation and size of each facial landmark varies. Orientation and size of one or more facial landmark features can be expected to van,'.
• Each landmark feature may be associated with one commensurate change in orientation and size.
• the threshold of expected change may be a combination of all expected changes for the landmark features. As noted above, the threshold may be application-specific.
• the comparison may be less sophisticated in order to determine that the session is live and the submitted facial biometric is genuine. Otherwise, when the transaction involves a paramount identification document, for example, renewal of passport, the comparison may involve a more comprehensively determined threshold.
• verification of facial liveliness of consumer 202 is based on both the corneal reflection of the object and the pose change of the face.
• a scoring mechanism may be implemented to quantify the degree of match between the response corneal reflection of the object and the known motion pattern to determine a first numerical value.
• the scorning mechanism can be also implemented to quantify the degree of match between the expected pose or changes in pose (e.g., calculated from the measured locations of the camera de v ice) and the detected pose in a capture image to determine a second numerical value. Then an overall score can be determined based on the first and second matching values, e.g., with a weighting factor.
• the weighting factors of the first and second matching values can be, for example, 50% and 50% or 60% and 40%.
• the additional verification may increase a confidence level that consumer 202 is indeed alive and is the person whose biometric is stored at the server of identity provider 206.
• liveliness verification includes analysis of a corneal reflection of an illuminator from consumer 202.
• the face of consumer 202 may be detected.
• an eye 501 of consumer 202 may then be detected in a video feed (or an image).
• detections include segmentation techniques that leverage face or eye template data.
• Such template data may be morphed through various shape transforms to identify face or eye area in a video feed.
• cornea region 502 may then be determined in the video feed. Reflections of illuminations on the corneal region in the video frame may then be analyzed.
• illuminator 506 adjacent to consumer 202 may be activated.
• illuminator 506 can be activated in response to determining that consumer 2,02 chooses a verified liveliness login for biometric authentication, e.g., 210 of FIG. 2A.
• illuminator 506 in Fig. 5 includes illumination source 506A and illumination source 506B with two distinct colors (e.g., blue or red).
• An illumination source can be a physical light source or a digital illumination source.
• illuminator 506 includes physical light sources, e.g., light-emitting diodes (LEDs) or light lamps.
• LEDs light-emitting diodes
• Illuminator 506 may include one or more light sources at distinct light wavelengths thus with one or more distinct colors.
• illuminator 306 includes one or more digital illumination sources displayed on a screen, e.g., on a laser-powered phosphor display (LPD), a liquid-crystal display (LCD) screen, a light-emitting diode (LED) screen such as an active matrix organic LED screen, or any suitable display screen.
• a digital illumination source can be a digital color pattern displayed on a screen, e.g., a LCD screen of a mobile device.
• the digital illumination sources can be provided by an application (or an App) installed on a computing device of consumer 202 or provided by the server at identity provider 206, e.g., in a user interface displayed on the computing device of consumer 202.
• illuminator 306 includes one or more physically light sources and one or more digital illumination sources.
• Illuminator 506 may be included or displayed on a consumer electronic device, such as a smartphone, a mobile phone, a mobile computing de vice, or a desktop computer, from which consumer 202 attempts to authenticate his or her identity and liveliness.
• Illuminator 506 may be included on the console of an automatic teller machine (ATM) of a relying part ⁇ ' or adjacent to the ATM.
• ATM automatic teller machine
• a computing device associated with the relying party or identity provider 206 may be configured to control illuminator 506.
• Consumer 202 may attempt to authenticate his or her identity and liveliness through the computing device.
• illuminator 506 may illuminate consumer 202.
• the illumination from illuminator 506 can be straight to the eye of consumer 202 or to the eye of consumer 202 with an incident angle.
• illuminator 506 is positioned so that the illumination from illuminator 506 is incident on the eye of consumer 202 with an angle.
• consumer 202 can be prompted to move illuminator 506 (e.g., a flashlight on his/her smartphone) to a side of the face (or eyes) or move his/her face (or eyes) so that the illumination from illuminator 506 is incident on the eye of consumer 202 with an angle.
• illuminator 506 e.g., a flashlight on his/her smartphone
• Information of the illumination can be known or predetermined.
• a computing device of consumer 202 performs liveliness verification.
• the information of illumination can be known to the computing device but unknown to consumer 202 (or would- be hackers as man in the middle) beforehand.
• the information of illumination can be generated by the computing device or provided by the server at identity provider 206.
• the corneal reflection immediately following the activation of illuminator 506 may be captured, e.g., by a camera device adjacent to consumer 202.
• the camera device can keep capturing facial images of consumer 202 before, during, and after the activation of illuminator 506.
• the captured image frame may be analyzed to ascertain whether the corneal reflection is in agreement with the expected (known or predetermined) illumination.
• the position, shape, and color/wavelength of the reflected illumination in the cornea of consumer 202 may be detected and compared to the expectation illumination.
• the position of the corneal reflection for example, reflection spot 504A or 504B may be correlated with the position of illumination sources 506 A or 506B.
• the shape of the reflection spot 504A or 504B may be compared to the shape of illumination sources 506A or 506B.
• the wavelength (or color) of reflected light from reflection spot 504A or 504B may be compared to the wavelengths (or colors) of illumination sources 506A or 506B.
• illumination pattern by illuminator 506 may be modulated over time to form a temporal illumination pattern.
• illumination source 506A may generate illumination in bursts, each burst with a same or different temporal duration.
• a first temporal duration can be 1 second and a second temporal duration can be 2 seconds.
• the temporal illumination pattern can also include a change in illumination shape, e.g., varying between a square shape, a triangle shape, and/or a circle shape, and/or a change in illumination color, during different temporal durations or at different time points.
• illuminator 506 provides a red square at a first temporal duration, a red circle at a second temporal duration, no illumination at a third temporal duration, and a blue triangle at a fourth temporal duration.
• the corneal reflection in captured sequential images may be analyzed to determine whether the corneal reflection contains the same temporal pattern.
• the corneal reflection in captured sequential images can be also individually analyzed to determine whether the corneal reflection in the captured sequential images matches the individually expected corneal reflections of the illuminator 506. Determination of facial liveliness of consumer 202 can be based on the match results of the captured sequential images.
• illumination pattern by two or more illumination sources are two or more illumination sources.
• illumination sources 506A and 506B may be modulated over time to form a temporal illumination pattern .
• the illumination sources can illuminate same or different colors.
• illumination sources 506A and 506B may generate illumination in bursts, each with a same or distinct temporal duration and/or each with a same or different distinct illumination shape during the temporal duration.
• illumination source 506A illuminates a red square and illumination source 506B illuminates a blue square.
• illumination source 506A illuminates a red circle and illumination source 506B illuminates a blue square.
• illumination source 506 illuminates a red circle and illumination source 506 illuminates a blue circle.
• illumination source 506 illuminates a red rectangle and illumination source 506 illuminates a red circle.
• the corneal reflection in sequential images may be analyzed to determine whether the corneal reflection contains the same temporal pattern.
• the illumination pattern by two or more illumination sources may be modulated spatially to form a spatial illumination pattern.
• illumination sources 506A and 506B may be activated to illuminate cornea 502 in an interleaved manner.
• the illumination pattern from illumination sources 506A and 506B may be varied by turning on or off illumination source 506A, illumination source 506A, or the combination of illumination sources 506A and 506B at different time points.
• the corneal reflection may be analyzed to determine whe ther the reflection contains the same spatial pattern of illumination.
• the temporal and spatial aspects of illumination may be combined to further buttress the variety of matches that improves the confidence of a liveliness authentication session.
• the images in response to activated illumination may be captured in a video stream at a full frame rate.
• the server at identity provider 206 performs liveliness verification based on the captured video stream. For efficiency of communication, only a portion of the captured video stream may be transmitted. When corneal reflections are expected to change, the video stream may be digitalized and transmitted to the server at identity provider 206 at a rate faster than a floor-level rate (when consumer 202 remains stationary).
• the transmitted video stream may be analyzed by the server at identity provider 206 to determine whether the changes in corneal reflections of the video stream from consumer 202 matches the known changes in illumination of illuminator 506 or whether the corneal reflections of video frames at particular time points matches the expected corneal reflections based on known illuminations of illuminator 506 at the particular time points.
• a scoring mechanism may be implemented to quantify the degree of match between the response corneal reflection being received at the server of the identity provider 206 in the captured images and the expected illumination (e.g., position, shape, and wavelength or color, or temporal and/or spatial pattern).
• the degree of match may depend on the context and hence can vary from application to application. For example, the degree of match may depend on the nature of the underlying transaction or the dollar amount involved in the underlying transaction.
• a computing device receives the captured video stream, e.g., by a camera device included in or coupled to the computing device, and performs liveliness verification based on the captured video stream.
• the computing device can be a computing device of consumer 202 from which consumer 202 attempts to authenticate his or her identity and liveliness.
• the computing device can be a smartphone which includes a flashlight as illuminator 506 and a camera as the camera device.
• a mobile app installed on the smartphone can also provide one or more digital illumination sources as illuminator 506.
• the flashlight and the digital illumination sources can be combined to be used as illuminator 506.
• the computing device can control illuminator 506 to illuminate based on a predetermined or known illumination pattern.
• the illumination pattern can be generated by the computing device or transmitted from the server at identity provider 206.
• the illumination of illuminator 506 can be also measured by a detector that provides the measured illumination to the computing device.
• the known illumination of illuminator 506 can be stored in the computing device.
• the computing device can also control the camera device to capture a video stream during activation of the illumination pattern of illuminator 506.
• the computing device is associated with relying party or identity provider 206.
• the transmission of the captured video stream (e.g., a portion) can be fast.
• the computing device can analyze the transmitted video stream to determine the illumination (or illumination changes) in corneal reflections of the video stream (e.g., sequential video frames) matches the known illumination (or illumination changes) of illuminator 506,
• the computing device can use a scoring mechanism to quantify the degree of match between the response corneal reflection in the captured images and the expected illumination. The degree of match may depend on the context and hence can vary from application to application . Based on the scored match, the computing device can verity facial liveliness of consumer 202.
• the computing device can then submit the result of the verification to the server at identity provider 206, e.g., together with biometric data of consumer 202. Then the server at identity provider 206 can seamlessly determine whether the submitted biometric data is a live facial biometric of consumer 202 based on the submitted result of the verification from the computing device.
• Implementations disclosed herein detect corneal reflections in response to illuminations, including illumination patterns temporally and/or spatially.
• Corneal reflection may proffer benefits as non-intrusive and capable of handling a wide band of operations (for example, from infra-red to visible light).
• corneal reflection tends to be more specular than skin reflections, which tend to be more scattered.
• Implementations disclosed herein may not include verbal directives to prompt consumer 202 to change facial gestures or other bodily positions.
• Some implementations disclosed herein can detect liveliness seamlessly and without user awareness and input. For example, activation of an illumination pattern and capturing sequential images can be performed without user's notice, e.g., when facial biometric of a consumer is captured.
• the liveliness verification can be also completed during capturing facial biometric, e.g., by a computing device of consumer 202.
• Some implementations may additionally include a biometric recognition feature to authenticate consumer 202.
• server at identity provider 206 may conduct a facial recognition to compare the facial biometric of an enrolled consumer with the facial biometric of consumer 202 submitting the authentication request. If the comparison yields a matching result, the facial recognition may prove that consumer 202 is the person whose facial biometric is stored at the server at identity provider 206.
• the biometric recognition may include voice recognition, speaker recognition, gait analysis, etc.
• Some implementations disclosed herein can detected liveliness seamlessly and without user awareness (e.g., directing the user to make any specific moves during the image capturing process).
• the stealth feature can be leveraged to further isolate liveliness detection from an on-line identit 7 management session.
• server at identity provider 206 may provide a signal that authentication is successful. If, however, consumer 202 cannot be verified as alive or as the person whose biometric has been stored at identity provider 206 in association with the account at relying part ⁇ ' 204, server at identity provider 206 may provide a signal that authentication has failed. The signal may be embedded in a message to consumer 202 to redirect consumer 202 back to relying party 204 (218).
• Consumer 202 may then return to the relying party 204 (220). If the embedded signal indicates that the verification is successful, consumer 202 may proceed to access the account at relying part ⁇ ' 204. In returning to relying party 204, consumer 202 may attach a success signal backed up by a credential of identity provider 206.
• the credential of the identity provider 206 may include, for example, a digital private key of identity provider 206, a digital watermark of identity provider 206. If, however, the embedded signal indicates that the verification has failed, consumer 202 may not access the account at relying party 204. Moreover, consumer 202 may not even be redirected back to the relying party 204.
• Fig. 2B is a timing diagram showing an example interaction among a consumer, a relying party, and an identity provider in authenticating the consumer when the consumer has not yet enrolled at the identity provider according to some implementations.
• the process may initiate with consumer 202 send an authentication request to access an account managed by relying party 204 (208).
• the submission of the authentication request may correspond to consumer 202 choosing biometric authentication 102 to access an account managed by, for example, a financial institution.
• the consumer may also choose to a verified liveliness login for biometric authentication (210).
• a server at the relying party may redirect consumer to proceed with verified liveliness (212). Thereafter, consumer 202 may be redirected to a server at the identity provider 206. For example, consumer 202 may submit a request to use verified liveliness by using a credential token in the form of consumer@retying_party.com (214). As illustrated in Fig. 2B, the request may be submitted at a server at identity provider 206.
• the server at identity provider 206 may notice that consumer 202 has not yet registered for the verified liveliness service at identity provider 206.
• the server at identity provider 206 may inquire the server at the relying party 204 whether the identity provider 206 may enroll consumer 202 in the verified liveliness service, as requested by consumer 202.
• Some relying parties may financially compensate identity provider 206 for the verified liveliness.
• these relying parties may stratify consumers into various classes with corresponding account benefits. By way of illustration, consumers with holdings above a threshold value of, for example, $50k may have the verified liveliness service as a complimentary sen-ice. Consumers with less account value or shorter membership history may need to pay a fee to relying party 204 in order to have the verified liveliness service.
• server at relying party 204 may indicate to the server at identity provider 206 that the enrollment for consumer 202 may proceed (224). If, however, server at relying party 204 determines that consumer 202 may not enroll in the verified liveliness service, server at relying party 204 may inform the server at identity provider 206 that consumer 202 does not have permission to enroll in the requested sen-ice and the sen d er at identity provider 206 may- drop the enrollment request initiated from consumer 202. Particularly, server at identity provider 206 may direct consumer 202 back to relying party 204 with the flag that enrollment has failed (224).
• server at identity provider 206 may administer an enrollment procedure for consumer 202 (226). Specifically, identity provider 206 may request consumer 202 to register a biometric.
• the biometric may include, for example, a facial biometric, a finger-print, a palm-print, an iris scan, a retina scan.
• server at identity provider 206 may request consumer 202 to pose in front of, for example, a web camera, a video camera, or a smart-phone camera.
• the camera device may reside on, for example, a computing device of consumer 202, or an ATM or in registration office of a bank.
• the camera device or a computing device coupled to the camera device may initially transmit a capture frame to server at identity provider 206.
• This captured frame may be stored at identity provider 206 as biometric information identifying consumer 202.
• server at identit 7 provider 206 or the computing device may verify the liveliness of the enrollment session as discussed above in association with Figs. 2A, 3 A, 3B, 4 and 5.
• server at identity provider 206 may also initiate illumination patterns on consumer 202 to increase the confidence level that a live consumer is attempting to enroll in the service.
• illumination patterns for different and separate spatial and temporal modulations may obviate the ambiguities associated with a single illumination response. For example, when consumer 202 may not be responding to the fullest extent as expected from one single illumination, or when sampling error during video transmission may have missed the frame that contains the exact response reflection, a single response illumination may be insufficient to determine with confidence that consumer 202 is a live person attempting to enroll in the service.
• the server at identity provider 206 may redirect consumer 202 back to relying party 204 with a signal that the authentication is successful (228). If consumer 202 has been successfully enrolled in the program, the server at identity provider 206 may provide a signal that the authentication has succeeded. If, however, consumer 202 cannot be verified as alive or as the person and cannot be enrolled in the verified sen/ice program, the server at identity provider 206 may provide a signal that authentication has failed. The signal may be embedded in a message to consumer 202 to redirect consumer 202 back to relying party 204 (218).
• Consumer 202 may then return to the relying party 204 (230). If the embedded signal indicates that the authentication has succeeded, consumer 202 may proceed to access the account at relying party 204. In returning to relying party 204, consumer 202 may attach a success signal backed up by a credential of identity provider 206.
• the credential of the identity provider 206 may include, for example, a digital private key of identity provider 206, a digital watermark of identity provider 206. If, however, the embedded signal indicates that the authentication has failed, consumer 202 may not access the account at relying party 204, Moreover, consumer 202 may not even be redirected back to the relying party 204.
• a first-time enrollee may be sent to the server at identity parry 206 after the first-time enrollee has successfully logged into an account of the first-time enrollee at the relying party 204,
• the enrollment may provi de consumers with additional security features to guard against spoofing attacks.
• the security features may also include biometric recognition, as detailed above. In fact, some
• configurations may use the biometric recognition as a form of password authentication.
• the biometric recognition may be used as a surrogate for the password login.
• identity provider 206 may obtain a database for a number of consumers attempting to enroll in a verified liveliness program.
• the server at identity provider 206 may collect a biometric from a consumer attempting to enroll. After verifying that the consumer is a live person making a legitimate enrollment request, the server at identity provider 206 may enroll the consumer.
• the server at identity provider 206 may have a copy of a digital biometric of consumer 202 when consumer 202 has been redirected from relying party 204. When a copy of the digital biometric of consumer 202 is available, the server at identity provider 206 may only need to determine that consumer 202 is a live person.
• Implementations of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly- implemented computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equi valents, or in combinations of one or more of them.
• Implementations of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non transitory program carrier for execution by, or to control the operation of, data processing apparatus.
• the computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.
• data processing apparatus refers to data processing hardware and encompasses all kinds of apparatus, devices, and machines for processing data, including, by ⁇ way of example, a programmable processor, a computer, or multiple processors or computers.
• the apparatus can also be or further include special purpose logic circuitry, e.g., a central processing unit (CPU), a FPGA (field programmable gate array), or an ASIC (application specific integrated circuit).
• special purpose logic circuitry e.g., a central processing unit (CPU), a FPGA (field programmable gate array), or an ASIC (application specific integrated circuit).
• the data processing apparatus and/or special purpose logic circuitry may be hardware-based and/or software-based.
• the apparatus can optionally include code that creates an execution environment for computer programs, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.
• code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.
• the present disclosure contemplates the use of data processing apparatuses with or without conventional operating systems, for example Linux, UNIX, Windows, Mac OS, Android, iOS or any other suitable conventional operating system.
• a computer program which may also be referred to or described as a program, software, a software application, a module, a software module, a script, or code, can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.
• a computer program may, but need not, correspond to a file in a file system.
• a program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub programs, or portions of code.
• a computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network. While portions of the programs illustrated in the various figures are shown as individual modules that implement the various features and functionality through various objects, methods, or other processes, the programs may instead include a number of sub-modules, third part ⁇ ' services, components, libraries, and such, as appropriate. Conversely, the features and functionality of various components can be combined into single components as appropriate.
• the processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output.
• the processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., a central processing unit (CPU), a FPGA (field programmable gate array), or an ASIC (application specific integrated circuit).
• CPU central processing unit
• FPGA field programmable gate array
• ASIC application specific integrated circuit
• Computers suitable for the execution of a computer program include, by way of example, can be based on general or special purpose microprocessors or both, or any other kind of central processing unit.
• a central processing unit will receive instructions and data from a read only memor - or a random access memory or both.
• the essential elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data.
• a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks.
• mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks.
• a computer need not have such devices.
• a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few.
• PDA personal digital assistant
• GPS Global Positioning System
• USB universal serial bus
• Computer readable media suitable for storing computer program instructions and data include all forms of non volatile memosy, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks.
• the memory may store various objects or data, including caches, classes, frameworks, applications, backup data, jobs, web pages, web page templates, database tables, repositories storing business and/or dynamic information, and any oilier appropriate information including any parameters, variables, algorithms, instructions, rules, constraints, or references thereto.
• the memor ' may include any other appropriate data, such as logs, policies, security or access data, reporting files, as well as others.
• the processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.
• implementations of the subject matter described in this specification can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube), LCD (liquid crystal display), or plasma monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer.
• a display device e.g., a CRT (cathode ray tube), LCD (liquid crystal display), or plasma monitor
• a keyboard and a pointing device e.g., a mouse or a trackball
• Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input.
• a computer can interact with a user by sending documents to and receiving documents from a
• GUI graphical user interface
• GUI may be used in the singular or the plural to describe one or more graphical user interfaces and each of the displays of a particular graphical user interface. Therefore, a GUI may represent any graphical user interface, including but not limited to, a web browser, a touch screen, or a command line interface (CLI) that processes information and efficiently presents the information results to the user.
• a GUI may include a plurality of user interface (UI) elements, some or all associated with a web browser, such as interactive fields, pull-down lists, and buttons operable by the business suite user. These and other UI elements may be related to or represent the functions of the web browser.
• UI user interface
• Implementations of the subject matter described in this specification can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back end, middleware, or front end components.
• the components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (LAN), a wide area network (WAN), e.g., the Internet, and a wireless local area network (WLAN).
• LAN local area network
• WAN wide area network
• WLAN wireless local area network
• the computing system can include clients and servers.
• a client and server are generally remote from each other and typically interact through a communication network.
• the relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

Landscapes

• Health & Medical Sciences (AREA)
• Life Sciences & Earth Sciences (AREA)
• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Health & Medical Sciences (AREA)
• Biophysics (AREA)
• Medical Informatics (AREA)
• Veterinary Medicine (AREA)
• Public Health (AREA)
• Animal Behavior & Ethology (AREA)
• Ophthalmology & Optometry (AREA)
• Surgery (AREA)
• Biomedical Technology (AREA)
• Heart & Thoracic Surgery (AREA)
• Molecular Biology (AREA)
• General Physics & Mathematics (AREA)
• Human Computer Interaction (AREA)
• Theoretical Computer Science (AREA)
• Multimedia (AREA)
• Pathology (AREA)
• Collating Specific Patterns (AREA)
• Image Input (AREA)
• Image Analysis (AREA)

Priority Applications (5)

Applications Claiming Priority (4)

Publications (1)

Family

ID=56285092

Family Applications (1)

Country Status (6)

Cited By (9)

Families Citing this family (7)

Citations (3)

Family Cites Families (13)

• 2015
  • 2015-12-31 WO PCT/US2015/068357 patent/WO2016109841A1/en not_active Ceased
  • 2015-12-31 EP EP15876387.0A patent/EP3240467A4/en not_active Withdrawn
  • 2015-12-31 CA CA2972821A patent/CA2972821A1/en not_active Abandoned
  • 2015-12-31 JP JP2017535830A patent/JP2018504703A/ja active Pending
  • 2015-12-31 AU AU2015373894A patent/AU2015373894B2/en not_active Expired - Fee Related
  • 2015-12-31 KR KR1020177021245A patent/KR20170126444A/ko not_active Withdrawn

Patent Citations (3)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events