WO2012104312A1 - Method and apparatus for gesture authentication - Google Patents

Method and apparatus for gesture authentication Download PDF

Info

Publication number
WO2012104312A1
WO2012104312A1 PCT/EP2012/051585 EP2012051585W WO2012104312A1 WO 2012104312 A1 WO2012104312 A1 WO 2012104312A1 EP 2012051585 W EP2012051585 W EP 2012051585W WO 2012104312 A1 WO2012104312 A1 WO 2012104312A1
Authority
WO
WIPO (PCT)
Prior art keywords
gesture
computing device
captured
user
motion
Prior art date
Application number
PCT/EP2012/051585
Other languages
French (fr)
Inventor
Wolfgang Michael Theimer
Pascal Wißmann
Roberto Avanzi
Original Assignee
Research In Motion Deutschland Gmbh
RUHR-UNIVERSITäT BOCHUM
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Research In Motion Deutschland Gmbh, RUHR-UNIVERSITäT BOCHUM filed Critical Research In Motion Deutschland Gmbh
Publication of WO2012104312A1 publication Critical patent/WO2012104312A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/30Writer recognition; Reading and verifying signatures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20Movements or behaviour, e.g. gesture recognition
    • G06V40/28Recognition of hand or arm movements, e.g. recognition of deaf sign language

Definitions

  • This application relates to computing device security.
  • this application relates to a method and apparatus for accepting and authenticating a gesture input by a user.
  • Computing device authentication commonly utilizes input of a sequence of characters by a user using a keyboard user input interface. This method of authentication is typically referred to as password security.
  • the captured sequence of characters, or password may be matched to a reference password by the computing device to authenticate the user.
  • the requirement that a user use a keyboard user input interface limits the applicability of the method to devices that include keyboards, whether physical or virtual, and further limits the security of input passwords to the number of key sequences entered by the user as well as the possible key combinations presented by the keyboard user input interface.
  • One alternate method for authenticating a user replaces key entry from a keyboard input interface with user input locations captured by a user input interface such as a touchpad, touchscreen, mice, camera, etc...
  • the sequence of captured user input locations may be combined to capture the motion or gesture created by the user.
  • the captured gesture may comprise a gesture password that may be compared with a reference gesture password to authenticate the user.
  • Some techniques may use known pattern recognition methods to confirm a match between the captured gesture and the reference gesture password. Typical methods include pattern recognition classification methods, Hidden Markov Models (HMM) and Kalman filter techniques.
  • Other techniques may divide the user input interface into input areas according to a pre-determined grid and compare the captured gesture with the input areas to generate a sequence of areas that have been traced by the gesture. The traced sequence of areas may be compared with a stored reference sequence of areas to validate the user. Multiple gestures may satisfy the comparison, provided they all traverse the same sequence of areas.
  • FIG. 1 is a block diagram of an embodiment of a mobile device.
  • FIG. 2 is a block diagram of an embodiment of a communication subsystem component of the mobile device of FIG. 1.
  • FIG. 3 is an example of a sampled gesture.
  • FIG. 4a is an example of a sampled gesture.
  • FIG. 4b is the example of FIG. 4a with a predicted most probable next point.
  • FIG. 4c illustrates the example of FIG. 4a with exemplary uncertainty areas.
  • FIG. 4d illustrates the example of FIG. 4b with exemplary uncertainty areas.
  • FIG. 5 illustrates example sampled gestures.
  • FIGS. 6a, 6b & 6c illustrate exemplary probability density functions for a gesture motion on a two-dimensional input interface.
  • FIG. 7 illustrates an example gesture with exemplary uncertainty areas.
  • a computing device implemented method for authenticating a user including the computing device capturing a gesture of the user with a sensor of the computing device, the captured gesture comprising a sequence of captured sensor readings sampled in time; comparing each captured sensor reading with a corresponding reference sensor reading of a reference gesture to determine whether the captured sensor reading falls within a reference uncertainty range from the corresponding reference sensor reading, and when the captured sensor reading falls within the reference uncertainty range, including an information content value associated with that reference sensor reading in a threshold calculation; and, authenticating the user when a sum of the included information content values exceeds a threshold value.
  • the reference uncertainty range for each of the reference sensor readings may be generated, prior to authenticating the user, by a reference computing device capturing a plurality of repetitions of the gesture from the user on a reference sensor, each captured repetition comprising a sequence of repeated reference sensor readings sampled in time; evaluating average coordinate values at each time sample for the plurality of repetitions to generate a reference gesture; and evaluating an uncertainty range for each average coordinate value at each time sample by comparing the coordinate values of each repetition at each time sample.
  • the information content values may be generated by the reference computing device calculating a probability for the uncertainty range for each time sample by integrating a probability density function defined for the reference sensor over the uncertainty range.
  • the probability density function may be provided to the reference computing device, the probability density function may be generated by capturing motion data comprising a plurality of different gestures from a plurality of users, each captured different gesture comprising a sequence of motion sensor readings sampled in time, each of the sequence of motion sensor readings comprising n- coordinate motion values; determining a likelihood for a next motion sensor reading by, for each of the motion sensor readings from the motion data, evaluating a change in value of the n-coordinate motion values of the next motion sensor reading from two or more preceding motion sensor readings, and evaluating a number of the next motion sensor readings for each of the change in value of the coordinate motion values; and, fitting the number of the next motion sensor readings for each of the change in value of the n-coordinate motion values to an analytical distribution.
  • Fitting the number of next motion sensor readings may comprise individually fitting the number of the next motion sensor readings for each of the change in value of a coordinate of the n-coordinate motion values to a statistically independent analytical distribution.
  • the integrating a probability density function defined for the sensor over the uncertainty range in may comprise: approximating the integrating by calculating a product of n one-dimensional probability functions, each of the n one-dimensional probability functions corresponding to one of the n-coordinates and evaluated over the uncertainty range for that coordinate.
  • the gesture may be drawn by the user in n'- dimensions and wherein the captured gesture comprises a sequence of n- dimension captured sensor readings sampled in time, and, wherein n' does not equal and is less than or greater than n.
  • a computing device may be provided for carrying out the methods above.
  • the embodiments described herein may be implemented on a computing device, such as the communication device that illustrated in FIGS. 1 and 2 and described below.
  • the computing device may exemplarily communicate with other devices over a wireless communication system or enterprise system.
  • the computing device may be a mobile device with two-way communication and advanced data communication capabilities including the capability to
  • the communication device 100 can also have voice communication capabilities.
  • FIG. 1 is a block diagram of an exemplary embodiment of a
  • the communication device 100 includes a number of components such as a main processor 102 that controls the overall operation of the communication device 100. Communication functions, including data and voice communications, are performed through a communication subsystem 104. Data received by the communication device 100 can be decompressed and decrypted by decoder 103, operating according to any suitable decompression techniques, and encryption/decryption techniques according to various standards, such as Data Encryption Standard (DES), Triple DES, or Advanced Encryption Standard (AES)). Image data is typically compressed and decompressed in accordance with appropriate standards, such as JPEG, while video data is typically compressed and decompressed in accordance with appropriate standards, such as H.26x and MPEG-x series standards.
  • DES Data Encryption Standard
  • AES Advanced Encryption Standard
  • the communication subsystem 104 receives messages from and sends messages to a wireless network 200.
  • the communication subsystem 104 is configured in accordance with one or more of Global System for Mobile Communication (GSM), General Packet Radio Services (GPRS) standards, Enhanced Data GSM Environment (EDGE) and Universal Mobile Telecommunications Service (UMTS).
  • GSM Global System for Mobile Communication
  • GPRS General Packet Radio Services
  • EDGE Enhanced Data GSM Environment
  • UMTS Universal Mobile Telecommunications Service
  • the wireless link connecting the communication subsystem 104 with the wireless network 200 represents one or more different Radio Frequency (RF) channels, operating according to defined protocols specified for GSM, GPRS, EDGE, or UMTS, and optionally other network communications. With newer network protocols, these channels are capable of supporting both circuit switched voice communications and packet switched data communications.
  • RF Radio Frequency
  • wireless networks can also be associated with the communication device 100 in variant implementations.
  • the different types of wireless networks that can be employed include, for example, data-centric wireless networks, voice- centric wireless networks, and dual-mode networks that can support both voice and data communications over the same physical base stations.
  • Combined dual- mode networks include, but are not limited to, Code Division Multiple Access (CDMA) or CDMA2000 networks, GSM/GPRS networks, third-generation (3G) networks like EDGE, HSPA, HSPA+, EVDO and UMTS, or fourth- generation (4G) networks such as LTE and LTE Advanced.
  • CDMA Code Division Multiple Access
  • 3G third-generation
  • HSPA HSPA+
  • 4G networks such as LTE and LTE Advanced.
  • Some other examples of data-centric networks include WiFi 802.11TM, MobitexTM and DataTACTM network communication systems.
  • the mobile device 100 may be provided with additional communication subsystems, such as the wireless LAN (WLAN) communication subsystem 105 also shown in FIG. 1.
  • the WLAN communication subsystem may operate in accordance with a known network protocol such as one or more of the 802.11TM family of standards developed by IEEE.
  • the communication subsystem 105 may be separate from, or integrated with, the communication subsystem 104 or with the short-range communications module 122.
  • the main processor 102 also interacts with additional subsystems such as a Random Access Memory (RAM) 106, a flash memory 108, a display 110, an auxiliary input/output (I/O) subsystem 112, a data port 1 14, a keyboard 116, a speaker 118, a microphone 120, the short-range communications 122 and other device subsystems 124.
  • the communication device may also be provided with an accelerometer 111, which may be used to detect gravity- or motion-induced forces and their direction. Detection of such forces applied to the device 100 may be processed to determine a response of the device 100, such as an orientation of a graphical user interface displayed on the display assembly 110 in response to a determination of the current orientation of which the device 100.
  • Some of the subsystems of the communication device 100 perform communication-related functions, whereas other subsystems can provide "resident" or on-device functions.
  • the display 110 and the keyboard 116 can be used for both communication-related functions, such as entering a text message for transmission over the network 200, and device- resident functions such as a calculator or task list.
  • a rendering circuit 125 is included in the device 100.
  • the rendering circuit 125 analyzes and processes the data file for visualization on the display 110.
  • Rendering data files originally optimized or prepared for visualization on large- screen displays on a portable electronic device display often requires additional processing prior to visualization on the small-screen portable electronic device displays. This additional processing may be accomplished by the rendering engine 125.
  • the rendering engine can be implemented in hardware, software, or a combination thereof, and can comprise a dedicated image processor and associated circuitry, or can be implemented within main processor 102.
  • the communication device 100 can send and receive communication signals over the wireless network 200 after required network registration or activation procedures have been completed.
  • Network access is associated with a subscriber or user of the communication device 100.
  • the communication device 100 To identify a subscriber, the communication device 100 requires a SIM/RUTM card 126 (i.e. Subscriber Identity Module or a Removable User Identity Module) or another suitable identity module to be inserted into a SIM/RUTM interface 128 in order to communicate with a network.
  • SIM/RUTM card 126 i.e. Subscriber Identity Module or a Removable User Identity Module
  • the SEVI/RUIM card 126 is one type of a conventional "smart card" that can be used to identify a subscriber of the communication device 100 and to personalize the communication device 100, among other things. Without the SIM/RUIM card 126, the communication device 100 is not fully operational for communication with the wireless network 200.
  • the SIM/RUIM card 126 By inserting the SIM/RUIM card 126 into the SIM/RUIM interface 128, a subscriber can access all subscribed services. Services can include: web browsing and messaging such as e-mail, voice mail, Short Message Service (SMS), and Multimedia Messaging Services (MMS). More advanced services can include: point of sale, field service and sales force automation.
  • the SIM/RUIM card 126 includes a processor and memory for storing information. Once the SIM/RUIM card 126 is inserted into the SIM/RUIM interface 128, it is coupled to the main processor 102. In order to identify the subscriber, the SEVI/RUIM card 126 can include some user parameters such as an International Mobile Subscriber Identity (EVISI).
  • EVISI International Mobile Subscriber Identity
  • the SIM/RUIM card 126 can store additional subscriber information for a mobile device as well, including datebook (or calendar) information and recent call information.
  • user identification information can also be programmed into the flash memory 108.
  • the communication device 100 may be a battery-powered device including a battery interface 132 for receiving one or more rechargeable batteries 130.
  • the battery 130 can be a smart battery with an embedded microprocessor.
  • the battery interface 132 is coupled to a regulator (not shown), which assists the battery 130 in providing power V+ to the communication device 100.
  • a regulator not shown
  • future technologies such as micro fuel cells can provide the power to the communication device 100.
  • the communication device 100 also includes an operating system 134 and software components 136 to 146 which are described in more detail below.
  • the operating system 134 and the software components 136 to 146 that are executed by the main processor 102 are typically stored in a persistent store such as the flash memory 108, which can alternatively be a read-only memory (ROM) or similar storage element (not shown).
  • a persistent store such as the flash memory 108
  • ROM read-only memory
  • portions of the operating system 134 and the software components 136 to 146 can be temporarily loaded into a volatile store such as the RAM 106.
  • Other software components can also be included, as is well known to those skilled in the art.
  • the subset of software applications 136 that control basic device operations, including data and voice communication applications, will normally be installed on the communication device 100 during its manufacture.
  • Other software applications include a message application 138 that can be any suitable software program that allows a user of the communication device 100 to send and receive electronic messages.
  • Messages that have been sent or received by the user are typically stored in the flash memory 108 of the communication device 100 or some other suitable storage element in the communication device 100.
  • some of the sent and received messages can be stored remotely from the device 100 such as in a data store of an associated host system that the communication device 100 communicates with.
  • the software applications can further include a device state module 140, a Personal Information Manager (PIM) 142, and other suitable modules (not shown).
  • the device state module 140 provides persistence, i.e. the device state module 140 ensures that important device data is stored in persistent memory, such as the flash memory 108, so that the data is not lost when the
  • the PIM 142 includes functionality for organizing and managing data items of interest to the user, such as, but not limited to, e-mail, contacts, calendar events, voice mails, appointments, and task items.
  • a PEVI application has the ability to send and receive data items via the wireless network 200.
  • PIM data items can be seamlessly integrated, synchronized, and updated via the wireless network 200 with the mobile device subscriber's corresponding data items stored and/or associated with a host computer system. This functionality creates a mirrored host computer on the communication device 100 with respect to such items. This can be particularly advantageous when the host computer system is the mobile device subscriber's office computer system.
  • Some or all of the data items stored at the communication device 100 may be indexed for searching on the device 100 either through a corresponding application, such as the PIM 142, or another suitable module.
  • the items may be searchable using a unified search process implemented in the device operating system 134.
  • application data items can be encapsulated in a searchable entity class and registered with a unified search engine on the device 100 that executes searches against all registered data repositories on the device based on received queries.
  • the search engine can also be configured to invoke a search process of external resources, such as Internet search engines or remote databases.
  • the communication device 100 also includes a connect module 144, and an information technology (IT) policy module 146.
  • the connect module 144 implements the communication protocols that are required for the
  • communication device 100 to communicate with the wireless infrastructure and any host system, such as an enterprise system, that the communication device 100 is authorized to interface with.
  • the connect module 144 includes a set of Application Programming Interfaces (APIs) that can be integrated with the communication device 100 to allow the communication device 100 to use any number of services associated with the enterprise system or with other systems accessible over the network 200.
  • APIs Application Programming Interfaces
  • the connect module 144 allows the communication device 100 to establish an end-to-end secure, authenticated communication pipe with the host system.
  • a subset of applications for which access is provided by the connect module 144 can be used to pass IT policy commands from the host system to the
  • the additional applications can be loaded onto the communication device 100 through at least one of the wireless network 200, the auxiliary I/O subsystem 112, the data port 1 14, the short-range communications subsystem 122, or any other suitable device subsystem 124.
  • This flexibility in application installation increases the functionality of the communication device 100 and can provide enhanced on-device functions, communication-related functions, or both.
  • secure communication applications can enable electronic commerce functions and other such financial transactions to be performed using the communication device 100.
  • the data port 114 enables a subscriber to set preferences through an external device or software application and extends the capabilities of the communication device 100 by providing for information or software downloads to the communication device 100 other than through a wireless communication network.
  • the alternate download path can, for example, be used to load an encryption key onto the communication device 100 through a direct and thus reliable and trusted connection to provide secure device communication.
  • the data port 114 can be any suitable port that enables data communication between the communication device 100 and another computing device.
  • the data port 114 can be a serial or a parallel port. In some instances, the data port 114 can be a USB port that includes data lines for data transfer and a supply line that can provide a charging current to charge the battery 130 of the communication device 100.
  • the short-range communications subsystem 122 provides for
  • the subsystem 122 can include an infrared device and associated circuits and components for short-range communication. Examples of short-range
  • communication standards include standards developed by the Infrared Data Association (IrDA), BluetoothTM, and the 802.11TM family of standards.
  • a received signal such as a text message, an e-mail message, or web page download will be processed by the communication subsystem 104 and input to the main processor 102.
  • the main processor 102 will then process the received signal for output to the display 110 or alternatively to the auxiliary I/O subsystem 112.
  • a subscriber can also compose data items, such as e-mail messages, for example, using the keyboard 116 in conjunction with the display 110 and possibly the auxiliary I/O subsystem 112.
  • the auxiliary subsystem 112 can include devices such as: a touchscreen, touchpad, mouse, track ball, infrared fingerprint detector, or a roller wheel with dynamic button pressing capability.
  • the keyboard 116 may be an alphanumeric keyboard and/or telephone-type keypad.
  • a composed item can be transmitted over the wireless network 200 through the communication subsystem 104. It will be appreciated that if the display 110 comprises a touchscreen, then the auxiliary subsystem 112 may still comprise one or more of the devices identified above.
  • the overall operation of the communication device 100 is substantially similar, except that the received signals are output to the speaker 118, and signals for transmission are generated by the microphone 120.
  • Alternative voice or audio I/O subsystems such as a voice message recording subsystem, can also be implemented on the communication device 100.
  • voice or audio signal output is accomplished primarily through the speaker 118, the display 110 can also be used to provide additional information such as the identity of a calling party, duration of a voice call, or other voice call related information.
  • FIG. 2 shows an exemplary block diagram of the communication subsystem component 104.
  • the communication subsystem 104 includes a receiver 150, a transmitter 152, as well as associated components such as one or more embedded or internal antenna elements 154 and 156, Local Oscillators (LOs) 158, and a processing module such as a Digital Signal Processor (DSP) 160.
  • LOs Local Oscillators
  • DSP Digital Signal Processor
  • Signals received by the antenna 154 through the wireless network 200 are input to the receiver 150, which can perform such common receiver functions as signal amplification, frequency down conversion, filtering, channel selection, and analog-to-digital (AID) conversion.
  • AID conversion of a received signal allows more complex communication functions such as demodulation and decoding to be performed in the DSP 160.
  • signals to be transmitted are processed, including modulation and encoding, by the DSP 160.
  • These DSP- processed signals are input to the transmitter 152 for digital-to-analog (D/A) conversion, frequency up conversion, filtering, amplification and transmission over the wireless network 200 via the antenna 156.
  • the DSP 160 not only processes communication signals, but also provides for receiver and transmitter control. For example, the gains applied to communication signals in the receiver 150 and the transmitter 152 can be adaptively controlled through automatic gain control algorithms implemented in the DSP 160.
  • the wireless link between the communication device 100 and the wireless network 200 can contain one or more different channels, typically different RF channels, and associated protocols used between the communication device 100 and the wireless network 200.
  • An RF channel is a limited resource that should be conserved, typically due to limits in overall bandwidth and limited battery power of the communication device 100.
  • the transmitter 152 is typically keyed or turned on only when it is transmitting to the wireless network 200 and is otherwise turned off to conserve resources.
  • the receiver 150 is periodically turned off to conserve power until it is needed to receive signals or information (if at all) during designated time periods.
  • Other communication subsystems such as the WLAN communication subsystem 105 shown in FIG. 1, may be provided with similar components as those described above configured for communication over the appropriate frequencies and using the appropriate protocols.
  • a computing device may be provided, such as communication device 100 for instance, that may authenticate a gesture input by a user of the computing device.
  • the gesture may comprise for instance a pattern traced by the user and captured by the computing device through a user input interface.
  • a gesture may be described as a trajectory in an n' -dimensional space as a function of time.
  • a gesture may comprise a motion of a point in two-dimensional space, for instance, a trajectory of one or more digits of a user through a tactile input interface such as a touchscreen or touchpad accessible to the computing device.
  • a gesture may comprise a motion of a point in three-dimensional space, for instance, a trajectory of an object or body part captured by an image sensor, magnetic sensor or ultrasound sensor accessible to the computing device.
  • the gesture may comprise a motion of the sensor in relation to a fixed environment.
  • a gesture may comprise a motion of a sensor in n-dimensional space, such as, a trajectory of an object or body part captured by an accelerometer, compass or gyro sensor accessible to the computing device.
  • a gesture may comprise multiple trajectories of objects or body parts captured by an n-dimensional sensor accessible to the computing device.
  • Digital systems such as computing devices, operate on discrete clock cycles, such that the gesture may be captured by a sensor as discrete samples in time typically at fixed time intervals.
  • a captured gesture may be defined as a sequence of sampling points g(t t ) , where the n-co-ordinates for each of the N sampling points s ; . represent the n-dimensional sensor readings sampled at time t i .
  • a two-dimensional captured gesture g( t, ) 300 may be plotted, for instance, with each captured sensor reading s i plotted as a discrete point.
  • the plotted coordinates for a point each corresponding to a one of the two dimensions of sensor data.
  • the original gesture trajectory may be approximated by interpolating between the points.
  • the captured gesture g( t, ) 300 is illustrated as a sequence of N sampled points s i 305 in two-dimensional space. While the captured gesture g( t, ) 300 is a two-dimensional trajectory, the original gesture may be created in n' -dimensions. The number of n-dimensions of the sensor readings may be the same as, or may vary from, the number of n' -dimensions of the original gesture.
  • n may equal n', such as a two- dimensional trajectory captured by a two-dimensional sensor, such as a user digit traced across a two-dimensional touch screen.
  • n may be less than n', such as a two- dimensional sensor capturing a three-dimensional gesture as a two-dimensional image, such as a camera capturing a motion of a user.
  • pattern recognition techniques may be employed to compare successive images in order to capture the motion of the gesture. Using pattern recognition techniques it is possible to identify at least one common anatomical point in each sampled image to identify a gesture motion or location.
  • n may be greater than n', such as a combination of accelerometer and position measurements combined to capture a two-dimensional or three-dimensional gesture.
  • n may be greater than n', such as a pair of two-dimensional sensors, such as a pair of cameras, may capture a three- dimensional gesture in three-dimensions by combining the pair of two- dimensional sensor readings.
  • each gesture drawn by a user includes a knowledge component and a biometric component.
  • the knowledge component is the pattern or general trajectory being traced by the user. The user must know the pattern and successfully re-trace the pattern within certain bounds for authentication to be successful.
  • the biometric component includes the specific motion dynamics and accuracy of each sample gesture to the general trajectory that the user is attempting to trace. Different users will create slightly different gestures with different motion dynamics. Furthermore, each user will exhibit their own personal range of variations between each repetition due to imperfect recall of the trajectory and imprecision of muscle activation. The personal range of variations provides a further biometric measure for identifying a user.
  • the captured gesture g( t, ) 300 includes both the overall captured trajectory, as interpolated between the sampled points, as well as specific motion dynamics which is captured by the locations of the sampled points along the captured trajectory and point location variation between gesture repetitions.
  • the specific motion dynamics arise as a speed of the object tracing the trajectory may vary at specific locations along the trajectory. Individuals may typically trace a trajectory with different motion dynamics when tracing the same trajectory. For a case where points are sampled at a fixed sampling period, for instance, a distribution of points along a trajectory will differ for different motion dynamics, even where the same trajectory is traced.
  • a gesture is considered predictable if each point on the trajectory can be predicted based upon one or more of the preceding points.
  • FIG. 4a an exemplary portion of a captured gesture 400 consisting of points 405, s ; . 410 and s j+l 415 is illustrated.
  • the points have been joined by interpolation vectors 406 411 to complete the sampled gesture 400.
  • Point s j+1 415 provides information content to the captured gesture 400 based upon both the direction and length of the interpolation vector 411 from the preceding point s ; . 410.
  • the information content of s j+1 415 may be assessed as the predictability of interpolation vector 411, having regard to interpolation vector 406 that joins points 405 and s t 410.
  • a most predictable next point s j+l 413 has been added to the exemplary portion of a sampled gesture 400, along with an interpolation vector 412 joining the most predictable next point s j+l 413 to points 5 405 and s t 410.
  • the information content provided by point s j+l 415 may be measured by the likelihood of the position of point s j+l 415 in relation to the anticipated most predictable point next point s j+l 413.
  • ⁇ i ⁇ N-l provides a measure of likelihood for a next point s j+l 415 at each of the points in the captured gesture 400.
  • an inner contour line 422 represents a higher probability than the outer contour lines 423, and is weighted the heaviest, with the weighting of the outer contour lines 423 decreasing with decreasing probability.
  • the probability density function 420 is illustrated as a series of discrete contour lines in FIG. 4b, that the probability density function 420 will typically provide a measure of likelihood for locations between the contour lines 421 which is commonly represented in diagrammatic form as a grey scale mapping with the grey scale value at each location representing a level of probability for that location.
  • FIG. 4b presents the probability density function 420 as a black and white ink drawing and accordingly only some contours of equal probability where represented for illustration purposes.
  • point s j+l 415 deviates from the direction of the most probable interpolation vector 412, and is located at a different distance from point s ; . 410 than the most predictable next point s j+l 413, point s j+l 415 is less likely than the most predictable next point s j+l 413.
  • a measure of the likelihood of the point s j+1 415 is found by entering the coordinates of point s j+l 415 into the probability density function 420. As indicated above, the less likely a location of point s j+1 415, the more information content that is provided to the sampled gesture 400.
  • sample point location areas 407, A i 414 and A j+l 417 each indicate an expected range of possible locations for a sampled point in future gesture repetitions.
  • the first ⁇ +l points of the gesture may not be provided with a sample location area.
  • a pre-defined location area may be specified for the first ⁇ +l points as a quality control measure to ensure the points do not vary more than the pre-defined location area between repetitions.
  • one or more of the first ⁇ +l points may be provided with a pre-defined range of possible locations for additional security.
  • a pre-determined allowable range may be imposed upon the one or more of the first ⁇ +l points as a constraint upon additional points of the captured gesture 400. Referring to FIG. 4c in the embodiment, for instance, if points
  • sample point location areas 407, A i 414 may be specified to define allowable variance for each of points 405 and s i 410 in relation to the rest of the captured gesture 400.
  • the sample point location area A j+l 417 for point s j+1 415 is shown with the probability density function 420.
  • the probability that a sampled point s j+l 415 of a given captured gesture 400 is located within the sample point location area A j+l 417 may be determined by integrating the robability density function 420 over the sample point location area A j+l 417: [0075]
  • a probability for the captured gesture 400 (the "gesture probability") may be calculated as the product of the probabilities that each of the points 405,
  • P gabTalk P 1 x P 2 x ... x P If .
  • the gesture information content is thus the sum of the information content of each sampled point:
  • the gesture information content provides a quantitative measure that may be compared with an information content of other authentication measures, such as a character password for instance.
  • FIG. 5 illustrates three sample gestures "8", "W” & ' ⁇ '. Multiple repetitions of each of the gestures is illustrated, with the gesture repetitions overlaid one atop another to illustrate possible variance between repetitions.
  • the gestures may have been captured, for instance, by a user drawing the gesture using a digit on a user input interface of a computing device, such as a touchscreen or touchpad.
  • FIG. 5 illustrates how gesture dynamics may differ between gestures.
  • the "8" is a continuous gesture with a single start and stop and no discontinuities through the gesture.
  • the gesture "W" includes two separate motions for each lobe, with a discontinuity in the middle where the gesture stops and reverses direction between lobes.
  • the gesture ' ⁇ ' includes a vertical straight line and then the "v" of the head of the arrow which requires a tracing digit to lift off the touch surface and be replaced.
  • many other gestures are possible including gestures with one or more components active at any one time, such as multi- touch gestures or images capturing multiple reference points.
  • an apparatus and method of authentication using captured gestures may be provided.
  • a first component of the embodiment is a gesture motion model.
  • the gesture motion model provides a probability density function f(s ⁇ s t , s t _ ... , s,. ) that indicates a measure of likelihood for a next point s j+1 415 in relation to a possible 2 or more preceding points up to ⁇ +1 preceding points.
  • the probability density function indicates a measure of likelihood for a next point s j+l 415 in relation to the two preceding points s t 410, 5 405.
  • the gesture motion model is created by capturing a plurality of different gestures from a plurality of users using n-dimensional sensor readings at each sample time on one or more model generating computing devices. A sufficient range of different gestures is required to ensure the model is effective for a broad range of gestures having varied gesture motion dynamics. A sufficient range of users is required to ensure the model is not a user-specific motion model.
  • a gesture motion model may correspond with a particular type of sensor, such as a user input interface of the computing device, such that a different motion model may be required for each type of sensor.
  • a gesture motion model may be created for other n-dimensional sensor inputs or n' -dimensional gesture trajectories as described above.
  • Standard statistical sampling criteria may be used to assist in ensuring sufficient data has been captured to fit the gesture motion model to the probability density function f(s
  • two points are used to predict a location of a next sampled point in the gesture.
  • the most predictable next point s j+l 413 may be determined using the
  • the interpolation vector 406 is equivalent to an average velocity of the gesture between the points 405 and s t 410.
  • next point s j+l 413 for a user drawing a gesture on a two-dimensional user input interface may be located as a linear extrapolation of the average velocity of the gesture between the preceding points 5 405 and s t 410. That is, it is most likely the gesture will continue to the next point in the same direction and with the same speed as the gesture travelled between the preceding points 405 and s t 410. Changes in direction and speed are less probable and accordingly provide more information content.
  • a mathematical description of the model may be simplified by a change of reference to locate point 405 as the origin and a basis direction x' oriented from point 405 towards point s ; . 410.
  • the embodiment may be generalised such that the probability function for an n-dimension sensor reading may be approximated as a product n one-dimensional probability density functions.
  • the integral of the probability density function to calculate the probability may be simplified by fitting an analytical function to the probability density data collected from gesture repetitions by multiple users.
  • well-known distributions include a Gaussian, exponential or Pareto distribution.
  • An appropriate standard distribution may be selected based upon evaluating each standard distribution and selecting a distribution that provides the least error for the information content of the gesture compared to the captured data.
  • FIG. 6a, 6b and 6c are plots illustrating: 6a) the probability density function f(Ax Ay') as a two-dimensional mapping; 6b) a surface plot of a slice of the probability density function f(Ax Ay') taken along the ' direction; and, 6b) a surface plot of a slice of the probability density function f(Ax Ay') taken along the ⁇ direction.
  • the probability density function f(Ax Ay') may be approximated as a product of n statistically independent probability distributions, which in the example of two-dimensions may be represented as: f x ( ⁇ ') f 2 (Ay') .
  • the functions fi(Ax') and f 2 (Ay') may each be represented, for example, by a pair of exponential distributions that approximate the left side and the right side of the distributions.
  • a user-specific uncertainty model may be generated.
  • the user-specific uncertainty model provides a measure of a user's personal gesture characteristics including variability of point locations, consistency of speed within the gesture and preferably identification of portions of a gesture that a user tends to reproduce with consistent accuracy and precision.
  • the user-specific uncertainty model is built by capturing from a user several repetitions of the same gesture. This process may be referred to as "training". The gesture repetitions may be captured from the user on a reference computing device. In an aspect, this may comprise the same computing device that will be used to subsequently authenticate the user.
  • the gesture repetitions may first be aligned by translating each repetition until the multiple gesture repetitions are positioned to maximise the overlap between gestures.
  • a centre of gravity calculation may be performed on the gesture repetitions, and the gesture repetitions aligned by translation to align the corresponding calculated centres of gravity.
  • the gesture repetitions may be re-sampled to have the same number of sampling points.
  • the speed of a user's gesture may vary during training, leading to slower gestures having more sampling points than faster gestures.
  • a baseline number of sampling points may be established, for instance by selecting the number of points from the gesture repetition having the fewest sampling points. After alignment each gesture repetitions that has more sampling points than the baseline number may be re-sampled by interpolating between the captured points to generate a re-sampled gesture having the baseline number of sampling points.
  • the re-sampling takes into account the motion dynamics recorded in the captured gesture to distribute the re-sampled points along the trajectory in a manner which reflects the motion dynamics of the gesture.
  • the re-sampling may include estimating a velocity at points along the gesture trajectory by calculating a difference between adjacent sampled points. The estimated velocity may be used when distributing the re-sampled points to ensure the re-sampled points include the recorded motion dynamics.
  • Each set of corresponding points between gesture repetitions may be compared to determine a spread or range of potential values.
  • the spread for the corresponding points comprises a basis for constructing an uncertainty range for each coordinate of each sample s t .
  • the uncertainty range comprises sample point location areas
  • an advantageous approximation for the uncertainty range may be made.
  • the approximation sets a maximum and a minimum value for each coordinate of the sample point based on a range of values sampled during the training repetitions. For example, in the case of a two-dimensional sensor reading, a rectangular uncertainty area may be defined.
  • the rectangular uncertainty area may be centred on a most probable point location and aligned to basis vectors ' and ⁇ which are recast for each point being a new origin and x' being the direction towards a next most probable point.
  • the integral of the probability density function P j+l in n- dimensions may be approximated as a product of the integral of each of the n linear probability density functions evaluated at boundaries of the uncertainty range determined by the user-specific motion model.
  • the n - linear probability density functions may be generated by fitting an analytical function to probability density data collected from multiple users when creating the gesture motion model.
  • ⁇ +l preceding points are used to determine a likelihood of the next point s j+l . Accordingly, the ⁇ +l points of the gesture do not have an associated probability and accordingly have an information content of zero. In some embodiments an allowable range of values for the sensor readings of the first ⁇ +l preceding points may be imposed as a constraint for additional security.
  • a sample captured gesture 700 is illustrated showing a point-wise average trajectory 705 from the training repetitions.
  • a most likely next point 712 is illustrated being preceded by points 710 711 that start at one end of the gesture 700. It is understood that there are additional points along the gesture 700, though only the first three are shown for demonstrative purposes.
  • a first interpolation vector 715 joins the first point 710 to the second point 711 and a second interpolation vector 716 joins the second point 711 to the most likely next (third) point 712.
  • the most likely next point 712 has a rectangular uncertainty area 720 centered on the average point location.
  • An axis of the first rectangular uncertainty area 720 is aligned on the direction of the second interpolation vector 716.
  • a security model may be implemented to authenticate a user based upon a gesture input.
  • the security model establishes criteria for authenticating a user.
  • one or more computing devices may be used to generate a gesture motion model relevant to a specific sensor and gesture type, to capture repeated gesture motion dynamics of a specific user to generate a user-specific uncertainty model based upon the gesture motion model and to build a security model for the user.
  • the security model may be used to authenticate a user when, for instance, a gesture is captured and compared to the security model.
  • the gesture motion model may be generated by a model generating computing device and the user-specific uncertainty model may be generated by a reference computing device. Generally, a user will be asked to enter repetitions of the gesture through a reference sensor of the reference computing device.
  • the reference computing device may comprise the computing device and the reference sensor may comprise the sensor.
  • a mobile computing device with a touch screen may be used to capture the gesture repetitions to build the user-specific uncertainty model, and the same mobile computing device and touch screen may be used to authenticate the user when using that device.
  • the reference computing device may comprise a different computing device.
  • the reference computing device comprises a laptop or a desktop computer and the computing device comprises a mobile computing device.
  • a gesture may be authenticated by capturing a gesture of the user with one or more sensors of the computing device.
  • the captured gesture includes a sequence of n- dimensional captured sensor readings s t sampled in time, each of the sequence of n-dimensional captured sensor readings comprising n-coordinate values.
  • the captured gesture may be compared to a reference gesture, created from the gesture motion model and the user-specific uncertainty model.
  • a reference gesture may be provided as a consolidation of the training gestures repeated during generation of the user-specific uncertainty model.
  • the training gestures may be consolidated by a point-wise average of the repetitions.
  • Each sample point of the captured gesture (“captured sensor reading”) may be compared with a corresponding sample point of the reference gesture (“reference sensor reading”) to determine whether each coordinate value of the captured sensor reading falls within an uncertainty range for the corresponding sample point of the reference sensor reading.
  • the uncertainty range having been determined during generation of the user-specific uncertainty model as described above.
  • the information content value generated for that reference sensor reading during generation of the user-specific uncertainty model may be added as a running sum of all captured sensor readings to calculate a gesture information content value for the captured gesture.
  • the security model may specify a threshold value such that if the gesture information content value for the captured gesture exceeds the threshold value, the user may be authenticated. In such a fashion, the security model may stress points with a high information content value, and place less weight on points with a low information content value.
  • a security model may authenticate a user where all points in a captured gesture fall within the respective uncertainty ranges of the user-specific uncertainty model.
  • the security model may further impose a limit to the uncertainty range boundaries, requiring either a higher or lower level of precision than that exhibited by the user during the training repetitions.
  • the security model may further define an uncertainty range for a first ⁇ +l points of the gesture though these points are not evaluated based upon preceding points during authentication.
  • the security model may allow for one or more sampled points to fall outside the corresponding uncertainty range boundary.
  • the allowance of missed sampled points makes the security model more tolerant of user gesture variation, at the trade off of some password security.
  • the security model may geometrically scale one or more of the uncertainty range boundaries by a factor to make the range either easier or harder for a user to authenticate a gesture.
  • a factor that increases a size of the uncertainty range boundaries makes authentication easier, while a factor that decreases a size of the uncertainty range boundaries makes authentication more difficult.
  • authentication security decreases with an increased uncertainty range and increases with a decreased uncertainty range.
  • a threshold for an information content of the gesture may be imposed.
  • the threshold may be enforced during training of the user-specific uncertainty model to ensure the gesture has sufficient information content to meet the threshold. Gestures that fall below the threshold may be rejected, requiring a user to enter a gesture of higher information content before the user- specific uncertainty model is accepted. Setting a higher threshold requires a gesture of greater complexity such as changes in gesture direction, changes in speed and length of the gesture.
  • the threshold may be set by a user of the computing device to ensure security, or the threshold may be imposed upon the computing device by a server operative to communication with the computing device over a network.
  • a corporate organisation may establish a security policy specifying a minimum security strength for passwords or gesture passwords.
  • a corporate server may communicate with the computing device to set the threshold in compliance with the security policy.
  • the threshold may be enforced by the security model during gesture capture to ensure the captured gesture has sufficient information content to meet the threshold before authentication the user.
  • the information content threshold for the user-specific certainty model may be different than the information content threshold for the security model.
  • a user-specific certainty model information content threshold may be set to ensure a gesture being trained has sufficient information content for security purposes. Since the training gesture is being entered in controlled circumstances with gesture repetition in a single session, the user- specific certainty model information content threshold may be set relatively high.
  • a security model information content threshold may be set somewhat lower to allow for variability during use where a user is entering the gesture once. The lower threshold provides a more tolerant authentication process, though a minimum level may be maintained to ensure sufficient security in the
  • the security model may allow for a lower level of information content at the individual point level, but maintain a minimum information content threshold at the gesture level.
  • the security model may require that a sum of the information content of the sampled points exceed the threshold, but impose no requirement on the information content of each sampled point.
  • the security model is flexible enough to allow different sampled points from each gesture repetition to fall within the uncertainty area boundaries, provided that a sufficient number of sampled points are accurately located such that the information content of the gesture meets or exceeds the threshold.
  • the security model may combine the geometric scaling and information content threshold.
  • the security model may employ an optimization algorithm to obtain a scaling factor and a threshold that maximises both the information content of the gesture and the acceptance rate of user gestures.
  • the systems' and methods' data may be stored in one or more data stores.
  • the data stores can be of many different types of storage devices and programming constructs, such as RAM, ROM, flash memory, programming data structures, programming variables, etc. It is noted that data structures describe formats for use in organizing and storing data in databases, programs, memory, or other computer-readable media for use by a computer program.
  • Code adapted to provide the systems and methods described above may be provided on many different types of computer-readable media including computer storage mechanisms (e.g., CD-ROM, diskette, RAM, flash memory, computer's hard drive, etc.) that contain instructions for use in execution by a processor to perform the methods' operations and implement the systems described herein.
  • computer storage mechanisms e.g., CD-ROM, diskette, RAM, flash memory, computer's hard drive, etc.
  • modules or processor includes but is not limited to a unit of code that performs a software operation, and can be implemented for example as a subroutine unit of code, or as a software function unit of code, or as an object (as in an object- oriented paradigm), or as an applet, or in a computer script language, or as another type of computer code.

Landscapes

  • Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

A computing device implemented method is provided for authenticating a user. In the method, a computing device may capture a gesture of the user with a sensor of the computing device as a sequence of captured sensor readings sampled in time. Each captured sensor reading may be compared with a corresponding reference sensor reading of a reference gesture to determine whether the captured sensor reading falls within a reference uncertainty range from the corresponding reference sensor reading. When the captured sensor reading falls within the reference uncertainty range, the device may include an information content value associated with that reference sensor reading in a threshold calculation. The device may authenticate the user when a sum of the included information content values exceeds a threshold value. A computing device may also be provided, the computing device operative to carry out the method.

Description

METHOD AND APPARATUS FOR GESTURE AUTHENTICATION
Field of the Invention
[0001] This application relates to computing device security. In particular this application relates to a method and apparatus for accepting and authenticating a gesture input by a user.
Background of the Invention
[0002] Computing device authentication commonly utilizes input of a sequence of characters by a user using a keyboard user input interface. This method of authentication is typically referred to as password security. The captured sequence of characters, or password, may be matched to a reference password by the computing device to authenticate the user. The requirement that a user use a keyboard user input interface limits the applicability of the method to devices that include keyboards, whether physical or virtual, and further limits the security of input passwords to the number of key sequences entered by the user as well as the possible key combinations presented by the keyboard user input interface.
[0003] One alternate method for authenticating a user replaces key entry from a keyboard input interface with user input locations captured by a user input interface such as a touchpad, touchscreen, mice, camera, etc... The sequence of captured user input locations may be combined to capture the motion or gesture created by the user.
[0004] The captured gesture may comprise a gesture password that may be compared with a reference gesture password to authenticate the user. Some techniques may use known pattern recognition methods to confirm a match between the captured gesture and the reference gesture password. Typical methods include pattern recognition classification methods, Hidden Markov Models (HMM) and Kalman filter techniques. [0005] Other techniques may divide the user input interface into input areas according to a pre-determined grid and compare the captured gesture with the input areas to generate a sequence of areas that have been traced by the gesture. The traced sequence of areas may be compared with a stored reference sequence of areas to validate the user. Multiple gestures may satisfy the comparison, provided they all traverse the same sequence of areas.
[0006] There is a need for a method and apparatus operative to authenticate a user's gesture that improves upon prior methods.
Brief Description of the Drawings
[0007] In drawings which illustrate by way of example only a preferred embodiment of the invention,
[0008] FIG. 1 is a block diagram of an embodiment of a mobile device.
[0009] FIG. 2 is a block diagram of an embodiment of a communication subsystem component of the mobile device of FIG. 1.
[0010] FIG. 3 is an example of a sampled gesture.
[0011] FIG. 4a is an example of a sampled gesture.
[0012] FIG. 4b is the example of FIG. 4a with a predicted most probable next point.
[0013] FIG. 4c illustrates the example of FIG. 4a with exemplary uncertainty areas.
[0014] FIG. 4d illustrates the example of FIG. 4b with exemplary uncertainty areas.
[0015] FIG. 5 illustrates example sampled gestures.
[0016] FIGS. 6a, 6b & 6c illustrate exemplary probability density functions for a gesture motion on a two-dimensional input interface. [0017] FIG. 7 illustrates an example gesture with exemplary uncertainty areas.
Detailed Description of the Invention
[0018] In an embodiment, a computing device implemented method for authenticating a user is provided, including the computing device capturing a gesture of the user with a sensor of the computing device, the captured gesture comprising a sequence of captured sensor readings sampled in time; comparing each captured sensor reading with a corresponding reference sensor reading of a reference gesture to determine whether the captured sensor reading falls within a reference uncertainty range from the corresponding reference sensor reading, and when the captured sensor reading falls within the reference uncertainty range, including an information content value associated with that reference sensor reading in a threshold calculation; and, authenticating the user when a sum of the included information content values exceeds a threshold value.
[0019] In an aspect of the method the reference uncertainty range for each of the reference sensor readings may be generated, prior to authenticating the user, by a reference computing device capturing a plurality of repetitions of the gesture from the user on a reference sensor, each captured repetition comprising a sequence of repeated reference sensor readings sampled in time; evaluating average coordinate values at each time sample for the plurality of repetitions to generate a reference gesture; and evaluating an uncertainty range for each average coordinate value at each time sample by comparing the coordinate values of each repetition at each time sample.
[0020] The information content values may be generated by the reference computing device calculating a probability for the uncertainty range for each time sample by integrating a probability density function defined for the reference sensor over the uncertainty range.
[0021] The probability density function may be provided to the reference computing device, the probability density function may be generated by capturing motion data comprising a plurality of different gestures from a plurality of users, each captured different gesture comprising a sequence of motion sensor readings sampled in time, each of the sequence of motion sensor readings comprising n- coordinate motion values; determining a likelihood for a next motion sensor reading by, for each of the motion sensor readings from the motion data, evaluating a change in value of the n-coordinate motion values of the next motion sensor reading from two or more preceding motion sensor readings, and evaluating a number of the next motion sensor readings for each of the change in value of the coordinate motion values; and, fitting the number of the next motion sensor readings for each of the change in value of the n-coordinate motion values to an analytical distribution.
[0022] Fitting the number of next motion sensor readings may comprise individually fitting the number of the next motion sensor readings for each of the change in value of a coordinate of the n-coordinate motion values to a statistically independent analytical distribution.
[0023] The integrating a probability density function defined for the sensor over the uncertainty range in may comprise: approximating the integrating by calculating a product of n one-dimensional probability functions, each of the n one-dimensional probability functions corresponding to one of the n-coordinates and evaluated over the uncertainty range for that coordinate.
[0024] In an aspect of the method, the gesture may be drawn by the user in n'- dimensions and wherein the captured gesture comprises a sequence of n- dimension captured sensor readings sampled in time, and, wherein n' does not equal and is less than or greater than n.
[0025] A computing device may be provided for carrying out the methods above.
[0026] The embodiments described herein may be implemented on a computing device, such as the communication device that illustrated in FIGS. 1 and 2 and described below. The computing device may exemplarily communicate with other devices over a wireless communication system or enterprise system. The computing device may be a mobile device with two-way communication and advanced data communication capabilities including the capability to
communicate with other mobile devices or computer systems through a network of transceiver stations. The communication device 100 can also have voice communication capabilities.
[0027] FIG. 1 is a block diagram of an exemplary embodiment of a
communication device 100. The communication device 100 includes a number of components such as a main processor 102 that controls the overall operation of the communication device 100. Communication functions, including data and voice communications, are performed through a communication subsystem 104. Data received by the communication device 100 can be decompressed and decrypted by decoder 103, operating according to any suitable decompression techniques, and encryption/decryption techniques according to various standards, such as Data Encryption Standard (DES), Triple DES, or Advanced Encryption Standard (AES)). Image data is typically compressed and decompressed in accordance with appropriate standards, such as JPEG, while video data is typically compressed and decompressed in accordance with appropriate standards, such as H.26x and MPEG-x series standards.
[0028] The communication subsystem 104 receives messages from and sends messages to a wireless network 200. In this exemplary embodiment of the communication device 100, the communication subsystem 104 is configured in accordance with one or more of Global System for Mobile Communication (GSM), General Packet Radio Services (GPRS) standards, Enhanced Data GSM Environment (EDGE) and Universal Mobile Telecommunications Service (UMTS). New standards are still being defined, but it is believed that they will have similarities to the network behavior described herein, and it will also be understood by persons skilled in the art that the embodiments described herein are intended to use any other suitable standards that are developed in the future. The wireless link connecting the communication subsystem 104 with the wireless network 200 represents one or more different Radio Frequency (RF) channels, operating according to defined protocols specified for GSM, GPRS, EDGE, or UMTS, and optionally other network communications. With newer network protocols, these channels are capable of supporting both circuit switched voice communications and packet switched data communications.
[0029] Other wireless networks can also be associated with the communication device 100 in variant implementations. The different types of wireless networks that can be employed include, for example, data-centric wireless networks, voice- centric wireless networks, and dual-mode networks that can support both voice and data communications over the same physical base stations. Combined dual- mode networks include, but are not limited to, Code Division Multiple Access (CDMA) or CDMA2000 networks, GSM/GPRS networks, third-generation (3G) networks like EDGE, HSPA, HSPA+, EVDO and UMTS, or fourth- generation (4G) networks such as LTE and LTE Advanced. Some other examples of data-centric networks include WiFi 802.11™, Mobitex™ and DataTAC™ network communication systems. Examples of other voice-centric data networks include Personal Communication Systems (PCS) networks like GSM and Time Division Multiple Access (TDMA) systems. The mobile device 100 may be provided with additional communication subsystems, such as the wireless LAN (WLAN) communication subsystem 105 also shown in FIG. 1. The WLAN communication subsystem may operate in accordance with a known network protocol such as one or more of the 802.11™ family of standards developed by IEEE. The communication subsystem 105 may be separate from, or integrated with, the communication subsystem 104 or with the short-range communications module 122. The main processor 102 also interacts with additional subsystems such as a Random Access Memory (RAM) 106, a flash memory 108, a display 110, an auxiliary input/output (I/O) subsystem 112, a data port 1 14, a keyboard 116, a speaker 118, a microphone 120, the short-range communications 122 and other device subsystems 124. The communication device may also be provided with an accelerometer 111, which may be used to detect gravity- or motion-induced forces and their direction. Detection of such forces applied to the device 100 may be processed to determine a response of the device 100, such as an orientation of a graphical user interface displayed on the display assembly 110 in response to a determination of the current orientation of which the device 100.
[0030] Some of the subsystems of the communication device 100 perform communication-related functions, whereas other subsystems can provide "resident" or on-device functions. By way of example, the display 110 and the keyboard 116 can be used for both communication-related functions, such as entering a text message for transmission over the network 200, and device- resident functions such as a calculator or task list.
[0031] A rendering circuit 125 is included in the device 100. When a user specifies that a data file is to be viewed on the display 110, the rendering circuit 125 analyzes and processes the data file for visualization on the display 110. Rendering data files originally optimized or prepared for visualization on large- screen displays on a portable electronic device display often requires additional processing prior to visualization on the small-screen portable electronic device displays. This additional processing may be accomplished by the rendering engine 125. As will be appreciated by those of skill in the art, the rendering engine can be implemented in hardware, software, or a combination thereof, and can comprise a dedicated image processor and associated circuitry, or can be implemented within main processor 102.
[0032] The communication device 100 can send and receive communication signals over the wireless network 200 after required network registration or activation procedures have been completed. Network access is associated with a subscriber or user of the communication device 100. To identify a subscriber, the communication device 100 requires a SIM/RUTM card 126 (i.e. Subscriber Identity Module or a Removable User Identity Module) or another suitable identity module to be inserted into a SIM/RUTM interface 128 in order to communicate with a network. The SEVI/RUIM card 126 is one type of a conventional "smart card" that can be used to identify a subscriber of the communication device 100 and to personalize the communication device 100, among other things. Without the SIM/RUIM card 126, the communication device 100 is not fully operational for communication with the wireless network 200. By inserting the SIM/RUIM card 126 into the SIM/RUIM interface 128, a subscriber can access all subscribed services. Services can include: web browsing and messaging such as e-mail, voice mail, Short Message Service (SMS), and Multimedia Messaging Services (MMS). More advanced services can include: point of sale, field service and sales force automation. The SIM/RUIM card 126 includes a processor and memory for storing information. Once the SIM/RUIM card 126 is inserted into the SIM/RUIM interface 128, it is coupled to the main processor 102. In order to identify the subscriber, the SEVI/RUIM card 126 can include some user parameters such as an International Mobile Subscriber Identity (EVISI). An advantage of using the SIM/RUIM card 126 is that a subscriber is not necessarily bound by any single physical mobile device. The SIM/RUIM card 126 can store additional subscriber information for a mobile device as well, including datebook (or calendar) information and recent call information.
Alternatively, user identification information can also be programmed into the flash memory 108.
[0033] The communication device 100 may be a battery-powered device including a battery interface 132 for receiving one or more rechargeable batteries 130. In at least some embodiments, the battery 130 can be a smart battery with an embedded microprocessor. The battery interface 132 is coupled to a regulator (not shown), which assists the battery 130 in providing power V+ to the communication device 100. Although current technology makes use of a battery, future technologies such as micro fuel cells can provide the power to the communication device 100.
[0034] The communication device 100 also includes an operating system 134 and software components 136 to 146 which are described in more detail below. The operating system 134 and the software components 136 to 146 that are executed by the main processor 102 are typically stored in a persistent store such as the flash memory 108, which can alternatively be a read-only memory (ROM) or similar storage element (not shown). Those skilled in the art will appreciate that portions of the operating system 134 and the software components 136 to 146, such as specific device applications, or parts thereof, can be temporarily loaded into a volatile store such as the RAM 106. Other software components can also be included, as is well known to those skilled in the art.
[0035] The subset of software applications 136 that control basic device operations, including data and voice communication applications, will normally be installed on the communication device 100 during its manufacture. Other software applications include a message application 138 that can be any suitable software program that allows a user of the communication device 100 to send and receive electronic messages. Various alternatives exist for the message application 138 as is well known to those skilled in the art. Messages that have been sent or received by the user are typically stored in the flash memory 108 of the communication device 100 or some other suitable storage element in the communication device 100. In at least some embodiments, some of the sent and received messages can be stored remotely from the device 100 such as in a data store of an associated host system that the communication device 100 communicates with.
[0036] The software applications can further include a device state module 140, a Personal Information Manager (PIM) 142, and other suitable modules (not shown). The device state module 140 provides persistence, i.e. the device state module 140 ensures that important device data is stored in persistent memory, such as the flash memory 108, so that the data is not lost when the
communication device 100 is turned off or loses power.
[0037] The PIM 142 includes functionality for organizing and managing data items of interest to the user, such as, but not limited to, e-mail, contacts, calendar events, voice mails, appointments, and task items. A PEVI application has the ability to send and receive data items via the wireless network 200. PIM data items can be seamlessly integrated, synchronized, and updated via the wireless network 200 with the mobile device subscriber's corresponding data items stored and/or associated with a host computer system. This functionality creates a mirrored host computer on the communication device 100 with respect to such items. This can be particularly advantageous when the host computer system is the mobile device subscriber's office computer system. Some or all of the data items stored at the communication device 100 may be indexed for searching on the device 100 either through a corresponding application, such as the PIM 142, or another suitable module. In addition, the items may be searchable using a unified search process implemented in the device operating system 134. For example, application data items can be encapsulated in a searchable entity class and registered with a unified search engine on the device 100 that executes searches against all registered data repositories on the device based on received queries. The search engine can also be configured to invoke a search process of external resources, such as Internet search engines or remote databases.
[0038] The communication device 100 also includes a connect module 144, and an information technology (IT) policy module 146. The connect module 144 implements the communication protocols that are required for the
communication device 100 to communicate with the wireless infrastructure and any host system, such as an enterprise system, that the communication device 100 is authorized to interface with.
[0039] The connect module 144 includes a set of Application Programming Interfaces (APIs) that can be integrated with the communication device 100 to allow the communication device 100 to use any number of services associated with the enterprise system or with other systems accessible over the network 200. The connect module 144 allows the communication device 100 to establish an end-to-end secure, authenticated communication pipe with the host system. A subset of applications for which access is provided by the connect module 144 can be used to pass IT policy commands from the host system to the
communication device 100. This can be done in a wireless or wired manner. These instructions can then be passed to the IT policy module 146 to modify the configuration of the device 100. Alternatively, in some cases, the IT policy update can also be done over a wired connection. [0040] Other types of software applications can also be installed on the communication device 100. These software applications can be third party applications, which are added after the manufacture of the communication device 100. Examples of third party applications include games, calculators, utilities, etc.
[0041] The additional applications can be loaded onto the communication device 100 through at least one of the wireless network 200, the auxiliary I/O subsystem 112, the data port 1 14, the short-range communications subsystem 122, or any other suitable device subsystem 124. This flexibility in application installation increases the functionality of the communication device 100 and can provide enhanced on-device functions, communication-related functions, or both. For example, secure communication applications can enable electronic commerce functions and other such financial transactions to be performed using the communication device 100.
[0042] The data port 114 enables a subscriber to set preferences through an external device or software application and extends the capabilities of the communication device 100 by providing for information or software downloads to the communication device 100 other than through a wireless communication network. The alternate download path can, for example, be used to load an encryption key onto the communication device 100 through a direct and thus reliable and trusted connection to provide secure device communication. The data port 114 can be any suitable port that enables data communication between the communication device 100 and another computing device. The data port 114 can be a serial or a parallel port. In some instances, the data port 114 can be a USB port that includes data lines for data transfer and a supply line that can provide a charging current to charge the battery 130 of the communication device 100.
[0043] The short-range communications subsystem 122 provides for
communication between the communication device 100 and different systems or devices, without the use of the wireless network 200. For example, the subsystem 122 can include an infrared device and associated circuits and components for short-range communication. Examples of short-range
communication standards include standards developed by the Infrared Data Association (IrDA), Bluetooth™, and the 802.11™ family of standards.
[0044] In use, a received signal such as a text message, an e-mail message, or web page download will be processed by the communication subsystem 104 and input to the main processor 102. The main processor 102 will then process the received signal for output to the display 110 or alternatively to the auxiliary I/O subsystem 112. A subscriber can also compose data items, such as e-mail messages, for example, using the keyboard 116 in conjunction with the display 110 and possibly the auxiliary I/O subsystem 112. The auxiliary subsystem 112 can include devices such as: a touchscreen, touchpad, mouse, track ball, infrared fingerprint detector, or a roller wheel with dynamic button pressing capability. The keyboard 116 may be an alphanumeric keyboard and/or telephone-type keypad. However, other types of keyboards can also be used. A composed item can be transmitted over the wireless network 200 through the communication subsystem 104. It will be appreciated that if the display 110 comprises a touchscreen, then the auxiliary subsystem 112 may still comprise one or more of the devices identified above.
[0045] For voice communications, the overall operation of the communication device 100 is substantially similar, except that the received signals are output to the speaker 118, and signals for transmission are generated by the microphone 120. Alternative voice or audio I/O subsystems, such as a voice message recording subsystem, can also be implemented on the communication device 100. Although voice or audio signal output is accomplished primarily through the speaker 118, the display 110 can also be used to provide additional information such as the identity of a calling party, duration of a voice call, or other voice call related information.
[0046] FIG. 2 shows an exemplary block diagram of the communication subsystem component 104. The communication subsystem 104 includes a receiver 150, a transmitter 152, as well as associated components such as one or more embedded or internal antenna elements 154 and 156, Local Oscillators (LOs) 158, and a processing module such as a Digital Signal Processor (DSP) 160. The particular design of the communication subsystem 104 is dependent upon the communication network 200 with which the communication device 100 is intended to operate. Thus, it should be understood that the design illustrated in FIG. 2 serves only as one example.
[0047] Signals received by the antenna 154 through the wireless network 200 are input to the receiver 150, which can perform such common receiver functions as signal amplification, frequency down conversion, filtering, channel selection, and analog-to-digital (AID) conversion. AID conversion of a received signal allows more complex communication functions such as demodulation and decoding to be performed in the DSP 160. In a similar manner, signals to be transmitted are processed, including modulation and encoding, by the DSP 160. These DSP- processed signals are input to the transmitter 152 for digital-to-analog (D/A) conversion, frequency up conversion, filtering, amplification and transmission over the wireless network 200 via the antenna 156. The DSP 160 not only processes communication signals, but also provides for receiver and transmitter control. For example, the gains applied to communication signals in the receiver 150 and the transmitter 152 can be adaptively controlled through automatic gain control algorithms implemented in the DSP 160.
[0048] The wireless link between the communication device 100 and the wireless network 200 can contain one or more different channels, typically different RF channels, and associated protocols used between the communication device 100 and the wireless network 200. An RF channel is a limited resource that should be conserved, typically due to limits in overall bandwidth and limited battery power of the communication device 100. When the communication device 100 is fully operational, the transmitter 152 is typically keyed or turned on only when it is transmitting to the wireless network 200 and is otherwise turned off to conserve resources. Similarly, the receiver 150 is periodically turned off to conserve power until it is needed to receive signals or information (if at all) during designated time periods. Other communication subsystems, such as the WLAN communication subsystem 105 shown in FIG. 1, may be provided with similar components as those described above configured for communication over the appropriate frequencies and using the appropriate protocols.
[0049] In an embodiment a computing device may be provided, such as communication device 100 for instance, that may authenticate a gesture input by a user of the computing device. In an embodiment, the gesture may comprise for instance a pattern traced by the user and captured by the computing device through a user input interface.
[0050] A gesture may be described as a trajectory in an n' -dimensional space as a function of time. By way of example, a gesture may comprise a motion of a point in two-dimensional space, for instance, a trajectory of one or more digits of a user through a tactile input interface such as a touchscreen or touchpad accessible to the computing device. Alternatively, a gesture may comprise a motion of a point in three-dimensional space, for instance, a trajectory of an object or body part captured by an image sensor, magnetic sensor or ultrasound sensor accessible to the computing device.
[0051] In an aspect, the gesture may comprise a motion of the sensor in relation to a fixed environment. For instance, a gesture may comprise a motion of a sensor in n-dimensional space, such as, a trajectory of an object or body part captured by an accelerometer, compass or gyro sensor accessible to the computing device. Alternatively, a gesture may comprise multiple trajectories of objects or body parts captured by an n-dimensional sensor accessible to the computing device.
[0052] Digital systems, such as computing devices, operate on discrete clock cycles, such that the gesture may be captured by a sensor as discrete samples in time typically at fixed time intervals. A captured gesture may be defined as a sequence of sampling points g(tt ) , where the n-co-ordinates for each of the N sampling points s;. represent the n-dimensional sensor readings sampled at time ti .
[0053] Referring to Figure 3, a two-dimensional captured gesture g( t, ) 300 may be plotted, for instance, with each captured sensor reading si plotted as a discrete point. The plotted coordinates for a point each corresponding to a one of the two dimensions of sensor data. The original gesture trajectory may be approximated by interpolating between the points.
[0054] In the example of Figure 3, the captured gesture g( t, ) 300 is illustrated as a sequence of N sampled points si 305 in two-dimensional space. While the captured gesture g( t, ) 300 is a two-dimensional trajectory, the original gesture may be created in n' -dimensions. The number of n-dimensions of the sensor readings may be the same as, or may vary from, the number of n' -dimensions of the original gesture.
[0055] In an embodiment, for instance, n may equal n', such as a two- dimensional trajectory captured by a two-dimensional sensor, such as a user digit traced across a two-dimensional touch screen.
[0056] In an embodiment, for instance, n may be less than n', such as a two- dimensional sensor capturing a three-dimensional gesture as a two-dimensional image, such as a camera capturing a motion of a user.
[0057] When using a camera to capture a motion of the user, pattern recognition techniques may be employed to compare successive images in order to capture the motion of the gesture. Using pattern recognition techniques it is possible to identify at least one common anatomical point in each sampled image to identify a gesture motion or location.
[0058] In an embodiment, for instance, n may be greater than n', such as a combination of accelerometer and position measurements combined to capture a two-dimensional or three-dimensional gesture. [0059] In an embodiment, for instance, n may be greater than n', such as a pair of two-dimensional sensors, such as a pair of cameras, may capture a three- dimensional gesture in three-dimensions by combining the pair of two- dimensional sensor readings.
[0060] In the context of authentication, each gesture drawn by a user includes a knowledge component and a biometric component. The knowledge component is the pattern or general trajectory being traced by the user. The user must know the pattern and successfully re-trace the pattern within certain bounds for authentication to be successful.
[0061] The biometric component includes the specific motion dynamics and accuracy of each sample gesture to the general trajectory that the user is attempting to trace. Different users will create slightly different gestures with different motion dynamics. Furthermore, each user will exhibit their own personal range of variations between each repetition due to imperfect recall of the trajectory and imprecision of muscle activation. The personal range of variations provides a further biometric measure for identifying a user.
[0062] The captured gesture g( t, ) 300 includes both the overall captured trajectory, as interpolated between the sampled points, as well as specific motion dynamics which is captured by the locations of the sampled points along the captured trajectory and point location variation between gesture repetitions. The specific motion dynamics arise as a speed of the object tracing the trajectory may vary at specific locations along the trajectory. Individuals may typically trace a trajectory with different motion dynamics when tracing the same trajectory. For a case where points are sampled at a fixed sampling period, for instance, a distribution of points along a trajectory will differ for different motion dynamics, even where the same trajectory is traced.
[0063] In relation to authentication, a gesture is considered predictable if each point on the trajectory can be predicted based upon one or more of the preceding points. A next point sj+l that is 100% probable (probability is 1) based upon one or more of the preceding points, is considered to have no (zero) information content for the purpose of authentication. Conversely, the less probable the next point sj+1 is based upon one or more of the preceding points, the more information content it provides.
[0064] Referring to FIG. 4a, an exemplary portion of a captured gesture 400 consisting of points 405, s;. 410 and sj+l 415 is illustrated. The points have been joined by interpolation vectors 406 411 to complete the sampled gesture 400. Point sj+1 415 provides information content to the captured gesture 400 based upon both the direction and length of the interpolation vector 411 from the preceding point s;. 410. In the example of Figure 4a, the information content of sj+1 415 may be assessed as the predictability of interpolation vector 411, having regard to interpolation vector 406 that joins points 405 and st 410.
[0065] Referring to FIG. 4b, a most predictable next point sj+l 413 has been added to the exemplary portion of a sampled gesture 400, along with an interpolation vector 412 joining the most predictable next point sj+l 413 to points 5 405 and st 410. The information content provided by point sj+l 415 may be measured by the likelihood of the position of point sj+l 415 in relation to the anticipated most predictable point next point sj+l 413.
[0066] A probability density function /(s | ), 1 < n < N-2 and 1+η
< i < N-l, provides a measure of likelihood for a next point sj+l 415 at each of the points in the captured gesture 400. Generally, a next sampled point location is predicted based upon a location of a preceding η+l points, where η is a fixed integer from 1 to N-2. So, for instance, where two points are used to predict a next sampled point location, η=1 and the probability density function for the next point sj+1 415 would thus depend upon the two preceding points, 405 and st 410. [0067] FIG. 4b illustrates the probability density function 420 as a series of concentric contours of equal probability density 421 about the most predictable next point sj+l 413 in relation to point s;. 410. A similar probability density function and next probable point would apply for the other points in the captured gesture 400. In the diagram, an inner contour line 422 represents a higher probability than the outer contour lines 423, and is weighted the heaviest, with the weighting of the outer contour lines 423 decreasing with decreasing probability.
[0068] It will be understood, that while the probability density function 420 is illustrated as a series of discrete contour lines in FIG. 4b, that the probability density function 420 will typically provide a measure of likelihood for locations between the contour lines 421 which is commonly represented in diagrammatic form as a grey scale mapping with the grey scale value at each location representing a level of probability for that location. FIG. 4b presents the probability density function 420 as a black and white ink drawing and accordingly only some contours of equal probability where represented for illustration purposes.
[0069] Since point sj+l 415 deviates from the direction of the most probable interpolation vector 412, and is located at a different distance from point s;. 410 than the most predictable next point sj+l 413, point sj+l 415 is less likely than the most predictable next point sj+l 413. A measure of the likelihood of the point sj+1 415 is found by entering the coordinates of point sj+l 415 into the probability density function 420. As indicated above, the less likely a location of point sj+1 415, the more information content that is provided to the sampled gesture 400.
[0070] Referring to FIG. 4c, the exemplary portion of a captured gesture 400 is illustrated with sample point location areas 407, Ai 414 and Aj+l 417. The sample point location areas A^ 407, Ai 414 and Aj+l 417 each indicate an expected range of possible locations for a sampled point in future gesture repetitions.
[0071] Since the first η+l points of the gesture cannot be predicted based upon a preceding point, the first η+l points of the gesture may not be provided with a sample location area. In an aspect, a pre-defined location area may be specified for the first η+l points as a quality control measure to ensure the points do not vary more than the pre-defined location area between repetitions.
[0072] Referring to FIG. 4c in the embodiment, for instance, if points st l 405 and 5;. 410 were the first two points in the sampled gesture 400, they would not have sample point location areas 407, Ai 414.
[0073] In an embodiment, one or more of the first η+l points may be provided with a pre-defined range of possible locations for additional security. In the embodiment, a pre-determined allowable range may be imposed upon the one or more of the first η+l points as a constraint upon additional points of the captured gesture 400. Referring to FIG. 4c in the embodiment, for instance, if points
405 and si 410 were the first two points in the sampled gesture 400, sample point location areas 407, Ai 414 may be specified to define allowable variance for each of points 405 and si 410 in relation to the rest of the captured gesture 400.
[0074] Referring to FIG. 4d, the sample point location area Aj+l 417 for point sj+1 415 is shown with the probability density function 420. The probability that a sampled point sj+l 415 of a given captured gesture 400 is located within the sample point location area Aj+l 417 may be determined by integrating the robability density function 420 over the sample point location area Aj+l 417:
Figure imgf000021_0001
[0075] A probability for the captured gesture 400 (the "gesture probability") may be calculated as the product of the probabilities that each of the points 405,
5;. 410 and sj+l 415 lie within the respective sample point location areas
407, 4 414 and Ai+l 417: Pgab„= P1 x P2 x ... x PIf .
[0076] The information content for the captured gesture 400, 1 gesture ("gesture information content"), may be calculated as the logarithm base 2 of the gesture probability: Igesture = -log2 Pgesture .
[0077] The gesture information content is thus the sum of the information content of each sampled point:
1 gesture = - ^g2 Pgesture = -lOg2 Pl - \Og2 P2 - . . . - \Og2 PN = I , + 12 + ... + I N .
[0078] The gesture information content provides a quantitative measure that may be compared with an information content of other authentication measures, such as a character password for instance.
[0079] By way of example, FIG. 5 illustrates three sample gestures "8", "W" & 'Ψ'. Multiple repetitions of each of the gestures is illustrated, with the gesture repetitions overlaid one atop another to illustrate possible variance between repetitions. The gestures may have been captured, for instance, by a user drawing the gesture using a digit on a user input interface of a computing device, such as a touchscreen or touchpad.
[0080] The example of FIG. 5 illustrates how gesture dynamics may differ between gestures. For instance, the "8" is a continuous gesture with a single start and stop and no discontinuities through the gesture. The gesture "W", includes two separate motions for each lobe, with a discontinuity in the middle where the gesture stops and reverses direction between lobes. The gesture 'Ψ' includes a vertical straight line and then the "v" of the head of the arrow which requires a tracing digit to lift off the touch surface and be replaced. [0081] As will be appreciated, many other gestures are possible including gestures with one or more components active at any one time, such as multi- touch gestures or images capturing multiple reference points.
[0082] In an embodiment, an apparatus and method of authentication using captured gestures may be provided. A first component of the embodiment is a gesture motion model. The gesture motion model provides a probability density function f(s \ st , st_ ... , s,. ) that indicates a measure of likelihood for a next point sj+1 415 in relation to a possible 2 or more preceding points up to η+1 preceding points. In an exemplary embodiment, for instance, the probability density function indicates a measure of likelihood for a next point sj+l 415 in relation to the two preceding points st 410, 5 405.
[0083] The gesture motion model is created by capturing a plurality of different gestures from a plurality of users using n-dimensional sensor readings at each sample time on one or more model generating computing devices. A sufficient range of different gestures is required to ensure the model is effective for a broad range of gestures having varied gesture motion dynamics. A sufficient range of users is required to ensure the model is not a user-specific motion model.
[0084] A gesture motion model may correspond with a particular type of sensor, such as a user input interface of the computing device, such that a different motion model may be required for each type of sensor. The description below assumes the same two-dimensional touch input interface example provided above, though a gesture motion model may be created for other n-dimensional sensor inputs or n' -dimensional gesture trajectories as described above.
[0085] Standard statistical sampling criteria may be used to assist in ensuring sufficient data has been captured to fit the gesture motion model to the probability density function f(s | £,., £,._!, ..., £,. ). In general, the greater the number of previously sampled points used to measure a likelihood for a next point sj+1 415, the more captured data is required to fit the model. In the examples provided in the present application, two points are used to predict a location of a next sampled point in the gesture.
[0086] Referring to the example of FIG. 4b, given points 405 and st 410, the most predictable next point sj+l 413 may be determined using the
interpolation vector 406. The interpolation vector 406 is equivalent to an average velocity of the gesture between the points 405 and st 410.
[0087] By sampling multiple users drawings multiple gestures, it has been determined experimentally that the most predictable next point sj+l 413 for a user drawing a gesture on a two-dimensional user input interface may be located as a linear extrapolation of the average velocity of the gesture between the preceding points 5 405 and st 410. That is, it is most likely the gesture will continue to the next point in the same direction and with the same speed as the gesture travelled between the preceding points 405 and st 410. Changes in direction and speed are less probable and accordingly provide more information content.
[0088] This experimental finding may not be applicable for other n' -dimensional trajectories or n-dimensional sensor readings.
[0089] A mathematical description of the model may be simplified by a change of reference to locate point 405 as the origin and a basis direction x' oriented from point 405 towards point s;. 410. In an embodiment, the probability density function for a most predictable next point sj+l 413 in two-dimensional space may be approximated by the product of two one-dimensional probability density functions: f(x',y') = ^'^ (y'), where ' and γ are basis vectors for the two-dimensional space. The embodiment may be generalised such that the probability function for an n-dimension sensor reading may be approximated as a product n one-dimensional probability density functions.
[0090] The integral of the probability density function to calculate the probability may be simplified by fitting an analytical function to the probability density data collected from gesture repetitions by multiple users. As known, well-known distributions include a Gaussian, exponential or Pareto distribution. An appropriate standard distribution may be selected based upon evaluating each standard distribution and selecting a distribution that provides the least error for the information content of the gesture compared to the captured data.
[0091] The probability density function may be further simplified by evaluating a difference between the next sampled point sj+l 415 and the most predictable next point si+l 413 : (Δχ',Δ/) = (χ' y'i+i-y'i+i ) > where (x'i+1 , j>'i+1) are the coordinates of the most predictable next point sj+l 413.
[0092] Referring to FIG. 6a, 6b and 6c, are plots illustrating: 6a) the probability density function f(Ax Ay') as a two-dimensional mapping; 6b) a surface plot of a slice of the probability density function f(Ax Ay') taken along the ' direction; and, 6b) a surface plot of a slice of the probability density function f(Ax Ay') taken along the γ direction.
[0093] In an embodiment, the probability density function f(Ax Ay') may be approximated as a product of n statistically independent probability distributions, which in the example of two-dimensions may be represented as: fx (Δχ') f2 (Ay') . Referring to FIG. 6b and FIG. 6c, the functions fi(Ax') and f2 (Ay') may each be represented, for example, by a pair of exponential distributions that approximate the left side and the right side of the distributions.
[0094] Once the gesture motion model has been created, a user-specific uncertainty model may be generated. The user-specific uncertainty model provides a measure of a user's personal gesture characteristics including variability of point locations, consistency of speed within the gesture and preferably identification of portions of a gesture that a user tends to reproduce with consistent accuracy and precision. [0095] The user-specific uncertainty model is built by capturing from a user several repetitions of the same gesture. This process may be referred to as "training". The gesture repetitions may be captured from the user on a reference computing device. In an aspect, this may comprise the same computing device that will be used to subsequently authenticate the user.
[0096] The gesture repetitions may first be aligned by translating each repetition until the multiple gesture repetitions are positioned to maximise the overlap between gestures. In an aspect, a centre of gravity calculation may be performed on the gesture repetitions, and the gesture repetitions aligned by translation to align the corresponding calculated centres of gravity.
[0097] Optionally, the gesture repetitions may be re-sampled to have the same number of sampling points. The speed of a user's gesture may vary during training, leading to slower gestures having more sampling points than faster gestures. To locate corresponding points on different gesture repetitions, a baseline number of sampling points may be established, for instance by selecting the number of points from the gesture repetition having the fewest sampling points. After alignment each gesture repetitions that has more sampling points than the baseline number may be re-sampled by interpolating between the captured points to generate a re-sampled gesture having the baseline number of sampling points.
[0098] Preferably the re-sampling takes into account the motion dynamics recorded in the captured gesture to distribute the re-sampled points along the trajectory in a manner which reflects the motion dynamics of the gesture. In an aspect, the re-sampling may include estimating a velocity at points along the gesture trajectory by calculating a difference between adjacent sampled points. The estimated velocity may be used when distributing the re-sampled points to ensure the re-sampled points include the recorded motion dynamics.
[0099] Each set of corresponding points between gesture repetitions may be compared to determine a spread or range of potential values. The spread for the corresponding points comprises a basis for constructing an uncertainty range for each coordinate of each sample st . In the embodiment of a two-dimensional sensor reading, the uncertainty range comprises sample point location areas
407, 4414 and Ai+l 417 illustrated in FIG. 4c.
[00100] In an embodiment, an advantageous approximation for the uncertainty range may be made. The approximation sets a maximum and a minimum value for each coordinate of the sample point based on a range of values sampled during the training repetitions. For example, in the case of a two-dimensional sensor reading, a rectangular uncertainty area may be defined. The rectangular uncertainty area may be centred on a most probable point location and aligned to basis vectors ' and γ which are recast for each point being a new origin and x' being the direction towards a next most probable point.
[00101] Use of a rectangular uncertainty area acceptably simplifies the calculation of probability since the anti derivative function of the ID probability functions may be evaluated at the limits of the integral. For typical probability density functions, such as an exponential distribution or Pareto distribution, an analytic antiderivative function is known:
Figure imgf000027_0001
[00102] Thus, the integral of the probability density function Pj+l in n- dimensions may be approximated as a product of the integral of each of the n linear probability density functions evaluated at boundaries of the uncertainty range determined by the user-specific motion model. As described above, the n - linear probability density functions may be generated by fitting an analytical function to probability density data collected from multiple users when creating the gesture motion model. [00103] The information content of each next point sj+l may then be calculated (using the preceding 2 to η+l points) = -log2 Pj+l , and summed to obtain the information content of the gesture, I gesture = -log2 Pgesture = -∑^+2log2 Pi+l .
[00104] As discussed above, η+l preceding points are used to determine a likelihood of the next point sj+l . Accordingly, the η+l points of the gesture do not have an associated probability and accordingly have an information content of zero. In some embodiments an allowable range of values for the sensor readings of the first η+l preceding points may be imposed as a constraint for additional security.
[00105] Referring to FIG. 7, a sample captured gesture 700 is illustrated showing a point-wise average trajectory 705 from the training repetitions. A most likely next point 712 is illustrated being preceded by points 710 711 that start at one end of the gesture 700. It is understood that there are additional points along the gesture 700, though only the first three are shown for demonstrative purposes.
[00106] A first interpolation vector 715 joins the first point 710 to the second point 711 and a second interpolation vector 716 joins the second point 711 to the most likely next (third) point 712. The most likely next point 712 has a rectangular uncertainty area 720 centered on the average point location. An axis of the first rectangular uncertainty area 720 is aligned on the direction of the second interpolation vector 716.
[00107] The larger the rectangular uncertainty area 720 at a point, as determined from the user-specific training, the larger the integral probability value Pj+l , and the smaller the information content for that point. Thus, points that are repeated with more precision during user-specific training, and a smaller uncertainty area, provide more information content and thus act as discriminators when authenticating a user. [00108] Each user will exhibit a different level of precision at different locations along the gesture. A user attempting to enter a gesture must replicate both the form of the gesture and the precision at the point level. Points that are demonstrated in training to have a higher degree of precise repeatability are associated with a smaller uncertainty range, requiring precision at that location when a gesture attempt is made.
[00109] Once the gesture motion model and the user-specific uncertainty model have been created, a security model may be implemented to authenticate a user based upon a gesture input. The security model establishes criteria for authenticating a user.
[00110] In an embodiment, one or more computing devices may be used to generate a gesture motion model relevant to a specific sensor and gesture type, to capture repeated gesture motion dynamics of a specific user to generate a user-specific uncertainty model based upon the gesture motion model and to build a security model for the user. The security model may be used to authenticate a user when, for instance, a gesture is captured and compared to the security model.
[00111] In an embodiment, the gesture motion model may be generated by a model generating computing device and the user-specific uncertainty model may be generated by a reference computing device. Generally, a user will be asked to enter repetitions of the gesture through a reference sensor of the reference computing device.
[00112] In an aspect, the reference computing device may comprise the computing device and the reference sensor may comprise the sensor. For instance, a mobile computing device with a touch screen may be used to capture the gesture repetitions to build the user-specific uncertainty model, and the same mobile computing device and touch screen may be used to authenticate the user when using that device. [00113] In an aspect, the reference computing device may comprise a different computing device. In an aspect the reference computing device comprises a laptop or a desktop computer and the computing device comprises a mobile computing device.
[00114] In an embodiment, once the security model has been implemented, a gesture may be authenticated by capturing a gesture of the user with one or more sensors of the computing device. The captured gesture includes a sequence of n- dimensional captured sensor readings st sampled in time, each of the sequence of n-dimensional captured sensor readings comprising n-coordinate values.
[00115] The captured gesture may be compared to a reference gesture, created from the gesture motion model and the user-specific uncertainty model. A reference gesture may be provided as a consolidation of the training gestures repeated during generation of the user-specific uncertainty model. In an aspect, the training gestures may be consolidated by a point-wise average of the repetitions.
[00116] Each sample point of the captured gesture ("captured sensor reading") may be compared with a corresponding sample point of the reference gesture ("reference sensor reading") to determine whether each coordinate value of the captured sensor reading falls within an uncertainty range for the corresponding sample point of the reference sensor reading. The uncertainty range having been determined during generation of the user-specific uncertainty model as described above.
[00117] When the captured sensor reading falls within the uncertainty range, the information content value generated for that reference sensor reading during generation of the user-specific uncertainty model may be added as a running sum of all captured sensor readings to calculate a gesture information content value for the captured gesture. The security model may specify a threshold value such that if the gesture information content value for the captured gesture exceeds the threshold value, the user may be authenticated. In such a fashion, the security model may stress points with a high information content value, and place less weight on points with a low information content value.
[00118] A security model may authenticate a user where all points in a captured gesture fall within the respective uncertainty ranges of the user-specific uncertainty model. The security model may further impose a limit to the uncertainty range boundaries, requiring either a higher or lower level of precision than that exhibited by the user during the training repetitions. As indicated above, the security model may further define an uncertainty range for a first η+l points of the gesture though these points are not evaluated based upon preceding points during authentication.
[00119] In an aspect, the security model may allow for one or more sampled points to fall outside the corresponding uncertainty range boundary. The allowance of missed sampled points makes the security model more tolerant of user gesture variation, at the trade off of some password security.
[00120] In an aspect, the security model may geometrically scale one or more of the uncertainty range boundaries by a factor to make the range either easier or harder for a user to authenticate a gesture. A factor that increases a size of the uncertainty range boundaries makes authentication easier, while a factor that decreases a size of the uncertainty range boundaries makes authentication more difficult. In general, authentication security decreases with an increased uncertainty range and increases with a decreased uncertainty range.
[00121] In an aspect, a threshold for an information content of the gesture may be imposed. The threshold may be enforced during training of the user-specific uncertainty model to ensure the gesture has sufficient information content to meet the threshold. Gestures that fall below the threshold may be rejected, requiring a user to enter a gesture of higher information content before the user- specific uncertainty model is accepted. Setting a higher threshold requires a gesture of greater complexity such as changes in gesture direction, changes in speed and length of the gesture. [00122] The threshold may be set by a user of the computing device to ensure security, or the threshold may be imposed upon the computing device by a server operative to communication with the computing device over a network. By way of example, a corporate organisation may establish a security policy specifying a minimum security strength for passwords or gesture passwords. A corporate server may communicate with the computing device to set the threshold in compliance with the security policy.
[00123] In an aspect, the threshold may be enforced by the security model during gesture capture to ensure the captured gesture has sufficient information content to meet the threshold before authentication the user.
[00124] In an aspect, the information content threshold for the user-specific certainty model may be different than the information content threshold for the security model. For instance, a user-specific certainty model information content threshold may be set to ensure a gesture being trained has sufficient information content for security purposes. Since the training gesture is being entered in controlled circumstances with gesture repetition in a single session, the user- specific certainty model information content threshold may be set relatively high. A security model information content threshold may be set somewhat lower to allow for variability during use where a user is entering the gesture once. The lower threshold provides a more tolerant authentication process, though a minimum level may be maintained to ensure sufficient security in the
authentication process.
[00125] In an aspect, the security model may allow for a lower level of information content at the individual point level, but maintain a minimum information content threshold at the gesture level. In the aspect, the security model may require that a sum of the information content of the sampled points exceed the threshold, but impose no requirement on the information content of each sampled point. In the aspect, the security model is flexible enough to allow different sampled points from each gesture repetition to fall within the uncertainty area boundaries, provided that a sufficient number of sampled points are accurately located such that the information content of the gesture meets or exceeds the threshold.
[00126] In an embodiment, the security model may combine the geometric scaling and information content threshold. In an aspect, the security model may employ an optimization algorithm to obtain a scaling factor and a threshold that maximises both the information content of the gesture and the acceptance rate of user gestures.
[00127] The systems and methods disclosed herein are presented only by way of example and are not meant to limit the scope of the subject matter described herein. Other variations of the systems and methods described above will be apparent to those in the art and as such are considered to be within the scope of the subject matter described herein. For example, it should be understood that steps and the order of the steps in the processing described herein may be altered, modified and/or augmented and still achieve the desired outcome.
[00128] The systems' and methods' data may be stored in one or more data stores. The data stores can be of many different types of storage devices and programming constructs, such as RAM, ROM, flash memory, programming data structures, programming variables, etc. It is noted that data structures describe formats for use in organizing and storing data in databases, programs, memory, or other computer-readable media for use by a computer program.
[00129] Code adapted to provide the systems and methods described above may be provided on many different types of computer-readable media including computer storage mechanisms (e.g., CD-ROM, diskette, RAM, flash memory, computer's hard drive, etc.) that contain instructions for use in execution by a processor to perform the methods' operations and implement the systems described herein.
[00130] The computer components, software modules, functions and data structures described herein may be connected directly or indirectly to each other in order to allow the flow of data needed for their operations. It is also noted that a module or processor includes but is not limited to a unit of code that performs a software operation, and can be implemented for example as a subroutine unit of code, or as a software function unit of code, or as an object (as in an object- oriented paradigm), or as an applet, or in a computer script language, or as another type of computer code.
[00131] A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by any one of the patent document or patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyrights whatsoever.

Claims

We Claim:
1. A computing device implemented method for authenticating a user comprising the computing device, the method comprising: capturing a gesture of the user with a sensor of the computing device, the captured gesture comprising a sequence of captured sensor readings sampled in time; comparing each captured sensor reading with a corresponding reference sensor reading of a reference gesture to determine whether the captured sensor reading falls within a reference uncertainty range from the corresponding reference sensor reading, and when the captured sensor reading falls within the reference uncertainty range, including an information content value associated with that reference sensor reading in a threshold calculation; and, authenticating the user when a sum of the included information content values exceeds a threshold value.
2. The method of claim 1 wherein the reference uncertainty range for each of the reference sensor readings are generated by a reference computing device arranged to perform the method step of: capturing a plurality of repetitions of the gesture from the user on a reference sensor, each captured repetition comprising a sequence of repeated reference sensor readings sampled in time; evaluating average coordinate values at each time sample for the plurality of repetitions to generate a reference gesture; and, evaluating an uncertainty range for each average coordinate value at each time sample by comparing coordinate values of each repetition of the reference sensor readings at corresponding time samples.
3. The method of claim 2 wherein the information content values are generated by the reference computing device arranged to perform the method steps of: calculating a probability for the uncertainty range for each time sample by integrating a probability density function defined for the reference sensor over the uncertainty range.
4. The method of claim 3 wherein the probability density function is provided to the reference computing device, the probability density function generated by a model generating computing device arranged to perform the method steps of: capturing motion data comprising a plurality of different gestures from a plurality of users, each captured different gesture comprising a sequence of motion sensor readings sampled in time, each of the sequence of motion sensor readings comprising n-coordinate motion values; determining a likelihood for a next motion sensor reading by, for each of the motion sensor readings from the motion data, evaluating a change in value of the n- coordinate motion values of the next motion sensor reading from two or more preceding motion sensor readings, and evaluating a number of the next motion sensor readings for each of the change in value of the n-coordinate motion values; and, fitting the number of the next motion sensor readings for each of the change in value of the n-coordinate motion values to an analytical distribution.
5. The method of claim 4 wherein fitting the number of next motion sensor readings comprises individually fitting the number of the next motion sensor readings for each of the change in value of a coordinate of the n-coordinate motion values to a statistically independent analytical distribution.
6. The method of claim 5 wherein integrating the probability density function defined for the sensor over the uncertainty range comprises: approximating the integrating by calculating a product of n one-dimensional probability functions, each of the n one-dimensional probability functions
corresponding to one of the n-coordinates and evaluated over the uncertainty range for that coordinate.
7. The method of any preceding claim, wherein the gesture is drawn by the user in n' -dimensions, and wherein the captured gesture comprises a sequence of n-dimension captured sensor readings sampled in time, and, wherein n' does not equal and is less than or greater than n.
8. The method of any preceding claim, wherein the gesture is drawn by the user moving the computing device, and wherein the computing device captures the gesture by detecting a change in location of the computing device.
9. The method of any preceding claim, wherein the sensor comprises a camera, and wherein the computing device captures the gesture by sampling image data captured by the camera.
10. The method of claim 9, further comprising the computing device identifying an anatomical point in each sampled image to identify a gesture location.
11. The method of any preceding claim, further comprising the computing device receiving a policy, the policy setting the threshold.
12. The method of any preceding claim as dependent on claim 2 wherein the reference computing device comprises the computing device and the reference sensor comprises the sensor.
13. A computing device operative to carry out the method of claim 1.
PCT/EP2012/051585 2011-01-31 2012-01-31 Method and apparatus for gesture authentication WO2012104312A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201161437958P 2011-01-31 2011-01-31
US61/437,958 2011-01-31

Publications (1)

Publication Number Publication Date
WO2012104312A1 true WO2012104312A1 (en) 2012-08-09

Family

ID=45607212

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2012/051585 WO2012104312A1 (en) 2011-01-31 2012-01-31 Method and apparatus for gesture authentication

Country Status (1)

Country Link
WO (1) WO2012104312A1 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014052942A (en) * 2012-09-10 2014-03-20 Seiko Epson Corp Head-mounted display device, control method of head-mounted display device, and authentication system
WO2014058662A2 (en) * 2012-10-09 2014-04-17 Lockheed Martin Corporation Secure gesture
CN105359153A (en) * 2013-06-14 2016-02-24 微软技术许可有限责任公司 Gesture-based authentication without retained credentialing gestures
WO2017171698A1 (en) * 2016-03-28 2017-10-05 Hewlett-Packard Development Company, L.P. Payment authentication
US10594683B2 (en) 2016-06-08 2020-03-17 International Business Machines Corporation Enforce data security based on a mobile device, positioning, augmented reality

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070041058A1 (en) * 2005-08-22 2007-02-22 Israel Disatnik Device and a method for identifying movement patterns
WO2009026337A1 (en) * 2007-08-20 2009-02-26 Gesturetek, Inc. Enhanced rejection of out-of-vocabulary words
US20100040293A1 (en) * 2008-08-12 2010-02-18 International Business Machines Corporation Kinematic Based Authentication
EP2166487A2 (en) * 2008-09-04 2010-03-24 Northrop Grumman Space & Mission Systems Corporation Security system utilizing gesture recognition

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070041058A1 (en) * 2005-08-22 2007-02-22 Israel Disatnik Device and a method for identifying movement patterns
WO2009026337A1 (en) * 2007-08-20 2009-02-26 Gesturetek, Inc. Enhanced rejection of out-of-vocabulary words
US20100040293A1 (en) * 2008-08-12 2010-02-18 International Business Machines Corporation Kinematic Based Authentication
EP2166487A2 (en) * 2008-09-04 2010-03-24 Northrop Grumman Space & Mission Systems Corporation Security system utilizing gesture recognition

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
FAUNDEZ-ZANUY ET AL: "On-line signature recognition based on VQ-DTW", PATTERN RECOGNITION, ELSEVIER, GB, vol. 40, no. 3, 6 November 2006 (2006-11-06), pages 981 - 992, XP005732819, ISSN: 0031-3203, DOI: 10.1016/J.PATCOG.2006.06.007 *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014052942A (en) * 2012-09-10 2014-03-20 Seiko Epson Corp Head-mounted display device, control method of head-mounted display device, and authentication system
WO2014058662A2 (en) * 2012-10-09 2014-04-17 Lockheed Martin Corporation Secure gesture
WO2014058662A3 (en) * 2012-10-09 2014-06-26 Lockheed Martin Corporation Secure gesture
GB2519710A (en) * 2012-10-09 2015-04-29 Lockheed Corp Secure gesture
US9195813B2 (en) 2012-10-09 2015-11-24 Lockheed Martin Corporation Secure gesture
CN105359153A (en) * 2013-06-14 2016-02-24 微软技术许可有限责任公司 Gesture-based authentication without retained credentialing gestures
CN105359153B (en) * 2013-06-14 2018-05-18 微软技术许可有限责任公司 The verification based on gesture in the case of no reservation certification gesture
WO2017171698A1 (en) * 2016-03-28 2017-10-05 Hewlett-Packard Development Company, L.P. Payment authentication
US10594683B2 (en) 2016-06-08 2020-03-17 International Business Machines Corporation Enforce data security based on a mobile device, positioning, augmented reality
US11146547B2 (en) 2016-06-08 2021-10-12 International Business Machines Corporation Enforce data security based on a mobile device, positioning, augmented reality

Similar Documents

Publication Publication Date Title
US11809696B2 (en) User interfaces to facilitate multiple modes of electronic communication
US11582176B2 (en) Context sensitive avatar captions
US9747491B2 (en) Dynamic handwriting verification and handwriting-based user authentication
US9526006B2 (en) System, method, and device of detecting identity of a user of an electronic device
US8625847B2 (en) Login method based on direction of gaze
EP3482331B1 (en) Obscuring data when gathering behavioral data
KR101963782B1 (en) Method for identifying user operation mode on handheld device and handheld device
CN111985265A (en) Image processing method and device
US20130181953A1 (en) Stylus computing environment
WO2018233438A1 (en) Human face feature point tracking method, device, storage medium and apparatus
US20140106711A1 (en) Method, user device and computer-readable storage for displaying message using fingerprint
CN108551519B (en) Information processing method, device, storage medium and system
EP2503479B1 (en) Login method based on direction of gaze
JP6667801B2 (en) Segment-based handwritten signature authentication system and method
KR20140079960A (en) Method, apparatus and computer-readable recording medium for running a program using recognizing fingerprint
EP2951746B1 (en) System and method of enhancing security of a wireless device through usage pattern detection
WO2012104312A1 (en) Method and apparatus for gesture authentication
CN107004073A (en) The method and electronic equipment of a kind of face verification
US20140232748A1 (en) Device, method and computer readable recording medium for operating the same
CN107370758B (en) Login method and mobile terminal
US11947758B2 (en) Diffusion-based handedness classification for touch-based input
WO2012046099A1 (en) Method, apparatus, and computer program product for implementing sketch-based authentication
CN111209050A (en) Method and device for switching working mode of electronic equipment
Kimon et al. Utilizing sequences of touch gestures for user verification on mobile devices
Ayyal Awwad An Adaptive Context-Aware Authentication System on Smartphones Using Machine Learning.

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12704006

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 22/11/2013)

122 Ep: pct application non-entry in european phase

Ref document number: 12704006

Country of ref document: EP

Kind code of ref document: A1