Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
  • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
  • G06F3/0416—Control or interface arrangements specially adapted for digitisers
  • G06F3/0418—Control or interface arrangements specially adapted for digitisers for error correction or compensation, e.g. based on parallax, calibration or alignment
  • G06F3/04186—Touch location disambiguation
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
  • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
  • G06F3/0412—Digitisers structurally integrated in a display
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
  • G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
  • G06F21/31—User authentication
  • G06F21/32—User authentication using biometric data, e.g. fingerprints, iris scans or voiceprints
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
  • G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
  • G06F21/31—User authentication
  • G06F21/36—User authentication by graphic or iconic representation
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
  • G06F21/60—Protecting data
  • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
  • G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
  • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
  • G06F3/0414—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means using force sensing means to determine a position
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
  • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
  • G06F3/0416—Control or interface arrangements specially adapted for digitisers
  • G06F3/04166—Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving
  • G06F3/041661—Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving using detection at multiple resolutions, e.g. coarse and fine scanning; using detection within a limited area, e.g. object tracking window
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
  • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
  • G06F3/042—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
  • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
  • G06F3/042—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means
  • G06F3/0421—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means by interrupting or reflecting a light beam, e.g. optical touch-screen
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
  • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
  • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
  • G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
  • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
  • G06V40/12—Fingerprints or palmprints
  • G06V40/13—Sensors therefor
  • G06V40/1306—Sensors therefor non-optical, e.g. ultrasonic or capacitive sensing
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
  • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
  • G06V40/12—Fingerprints or palmprints
  • G06V40/13—Sensors therefor
  • G06V40/1312—Sensors therefor direct reading, e.g. contactless acquisition
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
  • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
  • G06V40/12—Fingerprints or palmprints
  • G06V40/13—Sensors therefor
  • G06V40/1318—Sensors therefor using electro-optical elements or layers, e.g. electroluminescent sensing
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
  • G06V40/70—Multimodal biometrics, e.g. combining information from different biometric modalities
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
  • G06F2203/041—Indexing scheme relating to G06F3/041 - G06F3/045
  • G06F2203/04101—2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
  • G06F2203/041—Indexing scheme relating to G06F3/041 - G06F3/045
  • G06F2203/04106—Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
  • G06F2203/041—Indexing scheme relating to G06F3/041 - G06F3/045
  • G06F2203/04108—Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F2221/00—Indexing scheme relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
  • G06F2221/21—Indexing scheme relating to G06F21/00 and subgroups addressing additional information or applications relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
  • G06F2221/2139—Recurrent verification

Definitions

• the present invention relates to the field of user interface with a display.
• the present invention relates to a multi-level command sensing apparatus.
• a user In conventional user interfaces with mobile devices, a user typically controls a mobile device by using one or more fingers to touch or press a display or keys and buttons of the mobile device. Such touches or presses arc converted into user inputs or commands that may be configured to control the mobile device. It is beneficial to allow users to provide user commands as the user approaches, prior to touching, the screen of the mobile device. It is also beneficial to allow users to provide multi-levels of user commands with and/or without touching the screen of the mobile device.
• a multi-level command sensing apparatus includes a matrix of light sensors and light sources and that may also serve as die display.
• a method for performing multi-level command sensing by a multi-level command sensing apparatus comprises detecting a leakage current corresponding to changes of light conditions on a display by one or more light sensors of the multi-level command sensing apparatus, determining an action performed by a user based on the changes of light conditions on die display by a controller of the multi-level command sensing apparatus, determining a command based on the action performed by the controller of the multi-level command sensing apparatus, and executing the command by the controller of the multi-level command sensing apparatus.
• the controller when a finger, hand, palm, or object hovers or moves above a display (also referred to as a display panel, panel, or screen) or approaches the display before making direct contact, the controller can be configured to determine an approximate position and movement of the finger or object from the changes in the amount of light detected by the light sensors caused by the shadow, or shade, cast by the hovering object onto the panel.
• the light sources of the display may be either on or off.
• the light sensors can detect less light in the shaded area created by the hovering object.
• the light sources of the display may be turned on when the amount of ambient light detected is less than a certain predetermined amount.
• a method for determining movement of shade while operating under hovering mode includes a light detection sequence.
• the shade caused by the object also moves in certain directions corresponding to the movement of me object, including led to right, right to left, top to bottom, or bottom to top.
• the movement can also be diagonal or circular.
• the movement can also be perpendicular to the panel, for example, as an object moves closer to or away from the panel.
• the movement can also be combination of the above movements in 3 -dimensional space.
• the movements can be classified as 1 ) movement parallel to the panel 2) movement perpendicular to the panel, and 3) a combination of movement parallel and perpendicular to the panel.
• the system can have a movement Interpreter which can compare the movement to a list of predetermined patterns or reference values.
• a method for determining touch includes a light detection sequence. As an object approaches the panel, the shaded area becomes darker and smaller. When the shaded area becomes smaller than a predetermined size, smaller than a certain percentage or ratio from its initially detected size, darker than a certain percentage or ratio of its initially detected darkness, or darker than a predetermined amount, the panel can be configured to brighten the light sources at the shaded area before the object touches the panel.
• the l ight sensors can detect a brightening of the initially shaded area, due to the reflection and scattering of die light from the light sources off the object touching the screen, and thus can be able to determine when and in what location on the panel the touch input has occurred. If the brightening does not occur, the object has approached the screen to a certain proximity but has not made direct contact.
• a multi -level command sensing apparatus comprises a display, one or more Light sensors configured to detect a leakage current corresponding to changes of light conditions on the display, and a controller comprising one or more processors, where the controller is configured to: determine an action performed by a user based on the changes of light conditions on the display, determine a command based on the action performed, and execute the command determined.
• the changes of light conditions comprise a sequence of shadows detected on (he display, and the action performed by the user comprises a sequence of hovering motions without touching the display.
• the changes of light conditions comprise a sequence of brightened shadows detected on the display, and the action performed by the user comprises a sequence of touches on the display.
• the sequence of brightened shadows may be caused by reflected light and scattered light from an object touching the display.
• the sequence of touches includes a sequence of low pressure touches predefined by the designer or user.
• the sequence of touches includes a sequence of high pressure touches predefined by the designer or user.
• the sequence of touches can also include a sequence of multiple levels of pressure touches predefined by the designer or user.
• the controller may be further configured to compare the changes of light conditions on the display to a set of predefined changes of light conditions stored in a database of the multi-level command sensing apparatus; and identify the action performed by the user corresponding to the changes of light conditions on the display in response to a match being found in the set of predefined changes of light conditions.
• the controller may be farther configured to compare the action performed by the user to a set of predefined actions stored in a database of the multi-level command sensing apparatus, and identify the command corresponding to the action performed by the user in response to a match being found in the set of predefined actions.
• the controller may be further configured to authenticate die user based on the sequence of touches on the display, by continuously authenticating the user during an access to security sensitive information using the multi-level command sensing apparatus, and terminate the access to the security sensitive information in response to one or more mismatches found in the continuously authenticating process.
• FIGs. 1A-1D illustrate methods of detecting changes of light conditions on a display according to aspects of the present disclosure.
• FIGs. 2A-2B illustrate methods of detecting changes of light conditions on a display according to aspects of the present disclosure.
• FIG.3A-3B illustrate methods of detecting changes of light conditions as an object approaches a display according to aspects of the present disclosure.
• FIGs. 4A-4C illustrate methods of determining actions performed by a user based on the changes of light conditions on the display according to aspects of the present disclosure.
• FIGs.5A-5B illustrate methods of detecting changes of light conditions as an object approaches and touches a display according to aspects of the present disclosure.
• FIGs. 6A-6D illustrates methods of detecting changes of light conditions as finger approaches and touches a display according to aspects of the present disclosure.
• FIGs.7A-7C illustrate methods of sensing multiple levels of pressure as a display is being pressed according to aspects of the present disclosure.
• FIGs.8A-8C illustrate other methods of determining actions performed by a user based on the changes of light conditions on the display according to aspects of the present disclosure.
• FIGs.9A-9B illustrates examples of authenticating a user according to aspects of the present disclosure.
• FIGs. lOA-lOD illustrates other examples of authenticating a user according to aspects of the present disclosure.
• FIGs. 11 A- 11 D illustrates yet other examples of authenticating a user according to aspects of the present disclosure.
• FIG. 12 illustrates methods of defining different levels of pressure touches by a user according to aspects of the present disclosure.
• FIG. 13 illustrates an exemplary circuit for detecting a leakage current corresponding to changes of tight conditions on a display according to aspects of the present disclosure.
• FIGs. 14A-14C illustrate examples ofOLEDs with light sensors for detecting a leakage current corresponding to changes of light conditions according to aspects of the present disclosure.
• FIG. 15 A illustrates an exemplary subpixel circuit cell with forward bias according to aspects of the present disclosure
• FIG. 15B illustrates an exemplary subpixel circuit cell with reverse bias according to aspects of the present disclosure.
• FIG. 16 illustrates an exemplary pixel circuit cell with RGB subpixels according to aspects of the present disclosure.
• FIG. 17 illustrates an exemplary light sensing panel using a thin film transistor (TFT) panel structure according to aspects of the present disclosure.
• TFT thin film transistor
• FIG. 18 illustrates an exemplary display with light sensors for detecting changes of light conditions caused by an object hovering above the display according to aspects of (he present disclosure.
• FIG. 19 illustrates an exemplary display with light sensors for detecting changes of tight conditions caused by an object touching the display according to aspects of the present disclosure.
• FIG. 20 illustrates an exemplary controller of a multi-level command sensing apparatus according to aspects of the present disclosure.
• FIGs.21 A-21 D illustrate methods of performing multi-level command sensing according to aspects of the present disclosure. DESCRIPTION OF EMBODIMENTS
• a method for determining touch includes a light detection sequence.
• the shaded area created by the object on the panel becomes darker and smaller.
• a predetermined size for example smaller than a certain percentage or ratio from its initially detected size, darker man a certain percentage or ratio of its initially detected darkness, or darker than a predetermined value
• light sources embedded underneath the shaded area in die display will be turned on to detect a touch.
• a touch may be recognized when one or more light sensors, detect a brightened area in the initially shaded area, due to the reflection and scattering of the light from the light sources off the surface of the object touching the screen.
• Die touch sensing apparatus can determine when and at what location on the display the touch input has occurred. Alternatively, if the brightened area in the initially shaded area is not detected, it may be determined that the object has approached the display to certain proximity but has not yet made direct contact with the display.
• a method for performing multi-level command sensing in a multi-level command sensing apparatus includes a light detection sequence.
• the light sources are turned on randomly or sequentially.
• One or more cells or sub pixels turned on at once.
• the shaded area becomes darker and smaller.
• the light sensors can detect a brightening of the initially shaded area, due to the reflection and scattering of die light from the light sources off die object touching the screen, and thus can be able to determine when and in what location on the panel die touch input has occurred. If the brightening does not occur, the object has approached the screen to certain proximity but has not made direct contact.
• the controller may begin fingerprint capture, using methods described m following paragraphs. Based on the area of the image of die fingerprint that is captured, the controller can be configured to determine the relative pressure applied by the finger. The larger the area of the fingerprint captured, the higher die pressure that has been applied. The same procedure can be applied to determine pressure for a stylus with a soft dp, made of material such as rubber.
• an apparatus for determining shade movement, hovering and validity of a fingerprint includes a light refracting device (light refractor), a tight source, a light collecting device, and a controller.
• the light refracting device can, for example, be an active matrix organic light emitting diode (AMOLED) panel structure with reverse current measurement and amplification circuitry, and include an imaging surface and a viewing plane. Incident tight from the light source is projected directly or indirectly onto the imaging surface to create an image of the patterned object from the projected light onto die viewing plane.
• the apparatus is configured to have a thin form factor, which may be flexible or conformable, compared to conventional optical fingerprint acquisition apparatuses.
• the AMOLED panel includes the light source panel as well as light collecting devices.
• an apparatus for determining shade movement, hovering and validity of a fingerprint includes a light refracting device (light refractor), a tight source, a light collecting device, and a controller.
• the light refracting device can, for example, be a thin-film transistor (TFT) panel and include an imaging surface, a light receiving surface, a viewing plane, and light collecting devices.
• the light source can be an individually addressable panel of discrete light sources, for example, a liquid crystal display (LCD) panel or an AMOLED panel.
• Incident light from the light source is projected through the light receiving surface and projected directly or indirectly onto the imaging surface to create an image of the patterned object from the projected light onto the viewing plane.
• the apparatus is configured to have a Attn form factor, which may be flexible or conformable, compared to conventional optical fingerprint acquisition apparatuses.
• the TFT panel may be implemented as an add-on panel that is placed on top of the light source panel.
• a method for determining pressure or size of a fingerprint includes detecting the number of pixels or sub pixels that are brightened.
• the brightened pixels also referred to as brightened shadows, are detected by determining which pixels have a higher leakage current than a predetermined value or a predetermined percentage or ratio of its initial value. The larger the number of brightened pixels, the higher the pressure mat has been applied.
• certain numbers of brightened pixels correspond to certain pressures in some fixed relationship determined in advance.
• the number of brightened pixels is compared to the number of brightened pixels initially detected when the panel is touched. A larger number of brightened pixels man the number of brightened pixels initially detected corresponds to a higher pressure.
• the number of brightened pixels is compared to some reference touch input by the user, calibrated in advance. A larger number of brightened pixels than the number of brightened pixels determined in the reference touch correspond to a higher pressure.
• die number of brightened pixels is compared to touch input data collected by the device as the device is used over time. A certain characteristic of the data, such as its average, or some other reference determined by processing the data using various methods may be used. A larger number of brightened pixels than the number determined by processing the data corresponds to a higher pressure. A smaller number of brightened pixels than the number determined by processing the data corresponds to a smaller pressure.
• a method for determining size and validity of a fingerprint includes determining a set of light sources from a plurality of light sources for emitting light to a fingerprint, determining a set of sensor zones from a plurality of sensor zones for sensing scattered light from the fingerprint, determining a minimum distance between a sensor zone and a light source for sensing selective scattered light from the fingerprint, emitting light from the set of light sources to generate the scattered tight from the fingerprint, sensing the scattered tight in the set of sensor zones, and determining a validity of the fingerprint using the scattered light sensed m the plurality of sensor zones.
• the light refracting device supports basic touch screen functionality, as well as hovering object detection and gesture recognition, and pressure level detection using a combination of ambient light and the active matrix display as light sources.
• FIGs. 1 A-l B illustrate a method of detecting changes of light conditions on a display with the display being turned off according to aspects of the present disclosure.
• a finger 102 or object hovering above the panel 106 at a distance larger than 0 but close enough to cast a detectable shadow for example
• the presence of the finger 102 or object creates a shadow 104 on die panel due to the presence of ambient light 108 in the background.
• the shadow 104 on the panel 106 produces a difference in the leakage current between the shadowed area and the bright area measured by a matrix of light sensors embedded in the panel 106.
• the apparatus can then determine the position of the finger 102 or object based on the location of the shadow 104.
• An object hovering farther away from the panel may produce a smaller difference in leakage current between the shadowed area it creates on the panel and the brighter area outside the shadowed area, man the difference produced with the object hovering closer to the panel.
• the panel may be configured to delect a hovering object if the difference in leakage current between the shadowed area and the brighter area around the shadow is larger than a certain value that may be determined in advance or calculated based on present conditions.
• the panel may be configured to detect multiple shadows or only detect a certain number of the darkest shadows cast onto the panel.
• the hovering object 102 creates a shadow 105 on the panel 106 and a lower leakage current at the shadowed area relative to the area around the shadow.
• the shadow 105 becomes darker and creates an even lower leakage current than die example as in FIG. 1 A.
• FIGs. lC-1 D illustrate a method of detecting changes of light conditions on a display with the display being tamed on according to aspects of the present disclosure.
• the panel 106 is tamed on, which emits light 110 from the panel 106.
• die presence of die finger 102 or object creates a shadow 107 on the display panel due to the presence of ambient light 108 in the background and the emitted light 110.
• the shadow 107 on the panel 106 produces a difference in the leakage current between the shadowed area and the bright area measured by a matrix of ligh t sensors embedded in the panel 106, The apparatus can then determine the position of the finger 102 or object based on the location of die shadow 107.
• the hovering object 102 creates a shadow 109 on the panel 106 and a lower leakage current at the shadowed area relative to the area around the shadow.
• the shadow 109 becomes darker and creates an even lower leakage current than the example as in FIG. IC.
• the touch panel may include a light source panel, which may also be a display screen, along with a matrix of light sensors.
• the matrix of light sensors detect and locate the position of a finger or stylus placed on the panel to provide touch screen functionality on par with or surpassing other methods.
• a controller can be configured to detect the movement of hovering objects by detecting significant changes in amount of light detected either throughout the panel, or limited to a certain region. The changes of light conditions can be used to determine the position and/or movement of the hovering object.
• FIG. 2A illustrates a method of detecting changes of light conditions with the display being turned on
• FIG.2B illustrates a method of detecting changes of light conditions with the display being turned off according to aspects of the present disclosure.
• the examples of FIGs.2A-2B illustrate a finger or object 202 hovering near but not yet directly touches the display 206.
• the leakage current detected by light sensors in the exposed area 210 labeled as area B
• the leakage current detected in the shaded area 204 labeled as area A
• the display 206 may be lit by one or more illumination lights 207 from one or more light sources.
• the display 206 may be turned off, i.e. not lit by one or more illumination lights, and the method of detecting changes of light conditions with the display may still be implemented.
• FIG. 3 A illustrates a method of detecting changes of light conditions as an object approaches a display with the display being turned on
• FIG.3B illustrates a method of detecting changes of light conditions as an object approaches a display with the display being turned off according to aspects of the present disclosure.
• FIGs. 3A-3B illustrate the changes a controller may observe as the finger or object 302 approaches the panel. As the finger or object gets closer, for example moving from a distance of 25mm to 5mm, to die screen 306, the shadow of the finger or object on the screen becomes darker and darker (D->F), causing less leakage current at the shadowed area.
• the leakage current detected at region O 303 is larger than the leakage current detected at region E 304, which is larger than the leakage current detected at region F 305.
• the sensor can detect the leakage current variation and change to "touching" mode when the shadow becomes smaller than a predetermined size, or produces a leakage current smaller than a predetermined amount, in FIG. 3 A, with the display is turned on, with arrows 310 represent light being emitted from panel 306.
• a controller can be configured to detect hovering objects and thus execute commands even with the display off. For example, swiping a hand above the screen from one side to another (left to right or right to left) can cause the matrix of sensors to sense a change in ambient light being detected from one side of the screen to the other. This pattern can be utilized in a music player application to correspond to changing a song to cither the next or the previous song, depending on the direction of the swipe. Swiping a hand above the screen from the top of the screen to the bottom of the screen, or from the bottom of the screen to the top of the screen can be programmed to correspond to adjusting the volume up or down.
• FIGs.4A-4C illustrate methods of determining actions performed by a user based on the changes of light conditions on the display according to aspects of the present disclosure.
• FIGs.4A-4C illustrate examples of different gestures that may be produced by a hovering finger or object 402.
• a controller can be configured to recognize the gestures by the movement of the shadow (for example 404, 406, 408, or 410) on the panel, which produces differences in leakage current measured by the light sensors.
• Gesture recognition can be performed while hovering or touching the screen.
• Various gestures or a combination of gestures can be programmed to represent various commands. Detection of a stationary hovering object may also be configured to correspond to various commands depending on the situation. For example, detection of a hovering object over a link or button on the screen for a certain amount of time may be configured to open a secondary menu related to the link or button.
• specific gestures or actions in a certain sequence such as swiping above the screen from left to right, right to left, top to bottom, bottom to top, left to right to left, right to left to right, top to bottom to top, bottom to top to bottom, a direct touch motion gesture followed by pressure detection, circling motion, etc., or some and any combination of the above can be used to correspond to various commands during general use or to certain actions or directions in a game.
• FIG. SA illustrates a method of detecting changes of light conditions as an object approaches and touches a display with the display being turned on.
• FIG. 5B illustrates a method of detecting changes of tight conditions as an object approaches and touches a display with the display being turned off according to aspects of the present disclosure.
• FIG.5A illustrates the differences in leakage current that may be observed by a controller when a finger or object 502 makes duvet contact with the panel 504 when the display is on.
• FIG. SB illustrates the differences in the leakage current that may be observed by a controller when a finger or object 502 makes direct contact with the panel 504 when the display is off.
• the device when the display is off, the device may be in "standby" mode and ready to accept input to wake up the display, after which the device may be unlocked according to various user verification methods.
• the matrix of sensors With the presence of ambient light, the matrix of sensors may be able to detect die presence of a finger on the screen for a pre-designated period of time, after which the display can be woken up to read and verify a fingerprint.
• Other methods may also be used to initially detect the presence of a finger on the screen for the designated period of time - for example, a pressure sensor can be used Alternatively, other methods can be used to initially signal the device to wake up in order to turn on the display and read a fingerprint - for example, pressing the home or power button, of double tapping the screen.
• FIG. 6A illustrates a method of detecting changes of light conditions as a finger or other object 602 approaches a display 604 before making direct contact, with the display turned off;
• FIG.6B illustrates a method of detecting changes of light conditions as a finger or other object 602 approaches and touches a display 604 with the display turned off according to aspects of the present disclosure.
• FIG. 6C illustrates a method of detecting changes of light conditions as finger or other object 602 approaches a display 604 before making direct contact;
• FIG. 6D illustrates a method of detecting changes of light conditions as a finger or other object 602 approaches and touches a display 604 with the display being turned on according to aspects of the present disclosure.
• the area at which it makes contact may be brighter than before due to the light generated by the screen reflecting off the finger or object back onto the panel.
• the light sensors thus measure a higher leakage current at the area of contact
• FIGs. 7A-7C illustrate a method of sensing multiple levels of pressure as a finger or other object touches a display according to aspects of the present disclosure.
• the size of the scanned image of the finger or object 702 on the panel 704 changes hi correspondence to the changes of pressure of the finger or object on the screen.
• the pressure of die finger can be determined by the area of touch captured by a controller.
• FIG.7A illustrates a simple touch, in which the controller scans a small area of the fingerprint 706.
• FIG. 7B illustrates a slight higher pressure relative to the simple touch applied to the screen, in which the controller scans a larger fingerprint area 708.
• FIG. 7C illustrates an even higher relative pressure applied to the screen, in which the controller scans an even larger fingerprint area 710.
• a controller may be configured to provide pressure sensing capabilities based on the area of touch captured.
• the area of touch may be compared to a predetermined reference value, which may be an absolute default value, the initial value detected upon initial contact, a value determined through calibration through input by the user, or some reference value determined through other methods.
• the area of touch in relation to the reference value may be used to determine the level of pressure exerted by the finger or other object on the screen.
• a smaller area of touch relative to the reference value can be configured to correspond to a lighter pressure, and a larger area of touch relative to the reference value can be configured to correspond to a higher pressure. Any number of different discrete levels of pressure may be designated in comparison to the reference value, or varying levels of pressure may be determined on a continuous scale.
• the multiple levels of pressure being applied to a display may be customized on an individual user basis.
• a user may train the controller of the multi-level command sensing device to learn the levels of light pressure touch, medium pressure touch, and high pressure touch, such as the different levels of pressure being applied and their corresponding different sizes of areas as shown in FIG.7A, FIG.7B, and FIG. ?C, respectively.
• the pressure detection functionality is also compatible with stylus pens with a soft tip, such as rubber. Pressure can be detected in a rapid tap, a pressing hold, or continuously while the finger or stylus moves across the screen.
• FIGs.8A-8C illustrate other methods of determining actions performed by a user based on the changes of light conditions on the display according to aspects of the present disclosure.
• paths of a finger or an object for example 802, 804, 806, 808, 810, 812, 814, or 816) can be traced by a finger on the panel.
• the changes in light conditions in the areas on the panel may be used to determine the location of the finger or the object.
• swiping a hand across the screen can cause the matrix of sensors to sense a change in light being detected from one side of the screen to the other.
• This gesture can be programmed as commands that correspond to examining the next or previous item while browsing through some list of items.
• the gesture can also be programmed as commands along with a direct touch swipe to represent another level of browsing through items or data. For example, the hovering swipe gesture may flip to die next page of items, while the direct touch swipe switches to the next item on the same page.
• FIGs.9A-9B illustrates examples of authenticating a user according to aspects of the present disclosure.
• a user may designate a specific area 902 on a display 904 for unlocking the device.
• a default area may also be designated.
• the device may only be unlocked if the finger is placed on or within the correct pre-designated area, and the validity of the fingerprint is subsequently verified.
• a controller may be configured to provide fingerprint recognition capabilities. Fingerprints may be read anywhere on the screen and in any orientation, or the functionality may be enabled at a certain region or regions of the screen. Multiple fingerprints may also be read simultaneously.
• the controller may also support simultaneous and/or continuous fingerprint recognition during the use of the device, in which it captures and identifies fingerprints while the finger is moving on the screen or while the user is typing.
• a combination of methods for unlocking the device may be used and intuitively integrated - for example, after waking up the display, a pattern may be drawn, and the finger may remain on the screen without lifting for a short period of time after drawing the pattern, so that the device may verify the validity of the fingerprint
• This can provide an additional layer of security for the device, requiring both a correct pattern and a valid fingerprint.
• This method of doable verification can be seamlessly executed with a panel mat serves both as a display and a fingerprint sensor, as it does not require two separate actions - one of drawing the pattern on the screen, and one of scanning the fingerprint on a separate fingerprint sensor - on the part of the user.
• a fingerprint may be read for verification to unlock the device anywhere on the display screen, in any position or orientation.
• a certain area may be specifically designated for fingerprint capture for unlocking the device by the user.
• multiple fingerprints or a palm print can be read and identified
• the apparatus may be configured to continuously authenticate the user during an access to security sensitive information.
• the apparatus may be configured to authenticate the user when the user touches one or more links on a display 904, such as Link A 906 ⁇ to Link L 906L, or when the user presses one or more buttons, such as button 908) to button 901 ⁇ 2, or when the user types using a displayed keyboard 910.
• the apparatus may tenninate the access to the security sensitive information.
• the controller can provide an additional layer of security for sensitive mobile applications after the phone itself has been unlocked by requiring that the fingerprint be read and verified in the moment that the icon for a particular mobile application is selected.
• the fingerprint may be directly scanned on the screen in the moment of or after making a selection without having to be lifted and placed on another area on the device. Access may be granted if the fingerprint is consistent with that of the owner of the device or otherwise determined to be valid.
• the device may recognize fingerprints on the screen, directly during active operation of device for sensitive data or material, such as a bank account The device may be configured to lock if a foreign fingerprint or fingerprints are detected during the operation. An exact number or threshold may be specified.
• FIGs. 10A- 10D illustrates other examples of authenticating a user according to aspects of the present disclosure.
• the verification of a fingerprint may be performed after first drawing a pattern to unlock the device. After waking the display, a pattern is drawn (for example, 1004, 1006, 1008, 1010 or 1012), and the finger remains on the screen without lifting for a short period of time, allowing the device to verify the validity of the fingerprint (for example, 1014a, 1014b, 1014c, 1014d, 1014c, or 10141).
• the device may be configured to unlock only after both the correct pattern is drawn and the validity of the fingerprint is verified.
• FIGs. 11 A- 11 D illustrate other examples of authenticating a user according to aspects of the present disclosure.
• fingerprint recognition may be performed during active use of the touchscreen.
• the controller can be configured to capture fingerprints , such as i 102a, i 102b, i I02c, 1102ct or 1102c, on the screen during active operation of the device for sensitive information, such as a bank account
• the device may be configured to lock if an unauthenttcated fingerprint or fingerprints are detected. An exact number or threshold may be specified.
• FIG. 12 illustrates examples of various pressure-based touch commands according to aspects of the present disclosure.
• Different commands such as 1202 to 1232. may be designated based on different levels of pressures of touches in addition to different types of gestures/motions on the screen.
• the different commands corresponding to different levels of pressure being applied to a display may be customized by a user of the multi-level command sensing device.
• the user may train the controller of the multi-level command sensing device to learn the different individualized pressures of touches, such as the different areas of up, down, left, right, diagonal, circular, zigzag motions, etc. as shown by numerals 1202 to 1232 in FIG. 12.
• FIG. 12 illustrates examples of various pressure-based touch commands according to aspects of the present disclosure.
• Different commands such as 1202 to 1232. may be designated based on different levels of pressures of touches in addition to different types of gestures/motions on the screen.
• the different commands corresponding to different levels of pressure being applied to a display may be customized by a user
• the exemplary control circuit 13 illustrates an exemplary circuit for detecting a leakage current corresponding to changes in light conditions on a display according to aspects of the present disclosure.
• FIG. 13 an exemplary control circuit for determining whether an object or finger is hovering or touching the panel and its corresponding position is shown.
• the exemplary control circuit 13(H) comprises a leakage current detection phase, a comparison phase, and a command generation phase.
• the exemplary control circuit 1300 includes one or more pixel sampling blocks 1302, one or more pixel leakage current detectors 1304, one or more current converters 1306, a plurality of comparators 1308, a plurality of registers 1310, one or more code matching blocks 1312, a plurality of AND blocks 1314, and one or more command generators 1316.
• the exemplary control circuit 1300 further includes a register divider generator 1320, an average generator 1322, a reference leakage current generator 1324, a counter 1326, a generator comparator 1328, and a command selection block 1330.
• the above components are communicatively coupled as shown in FIG. 13 to form the exemplary control circuit 1300.
• control circuit can be configured to compare the leakage currents of each position in the matrix of sensors to a reference. Based on reference leakage currents that come from experimental data, the controller can determine the difference between hovering and touching modes on the screen. Reference leakage voltage for hovering can be smaller than the minimum leakage voltage of touching mode.
• Each comparator detects the pixels' leakage currents, which is converted to voltage and compared to a resistor divider voltage generator, !t generates a command which represents the action on the screen.
• a set of user defined commands may he generated and stored in a database or memory of die multi-level command sensing apparatus.
• the processing can be synchronized using a synchronization clock.
• FIG. 14A- 14C illustrate examples of OLEDs with light sensors for detecting a leakage current corresponding to changes of light conditions according to aspects of the present disclosure.
• FIG. 14A illustrates a unidirectional OLED 1402 with light sensors;
• FIG. 14B illustrates a conformable OLED 1404 with light sensors;
• FIG. 14C illustrates a bidirectional OLED 1406 with light sensors according to aspects of the present disclosure.
• both top emission and bottom emission type OLED structures can be used as the main component of a fingerprint acquisition apparatus.
• OLED devices such as small molecule OLED, polymer OLED, or solution based OLED, may be utilized as main OLED device structures.
• Both transparent and non-transparent OLED panels can be used as the main component of a fingerprint acquisition apparatus.
• Both thin panel and flexible or conformable types of OLED panels can be used as the main component of a fingerprint acquisition apparatus.
• An active matrix OLED (AMOLED) panel can be used as the main component of a fingerprint acquisition apparatus.
• An AMOLED panel may include subpixcl areas (red, green, and blue subpixels) and a driving circuit area (thin film transistor and capacitor).
• the brightness of each subpixel can be adjusted by the driving and switching transistors and capacitors and by controlling the amount of current injected to the OLED subpixels.
• the dimension of subpixels can be formed using OLED material deposition techniques. For instance, the size and position of subpixels can be set by using shadow masks during the OLED material evaporation process.
• An OLED may have a layered structure with the following sequence: anode / hole injection layer / hole transport layer / emissive layer / electron transport layer / electron injection layer / cathode.
• ITO and other transparent conducting materials having high work function can be used for anode materials, and metals such as aluminum and magnesium can be used for cathode materials.
• the imaging surface would be at the bottom of the substrate, and the light emission plane would be the cathode layer.
• the optical structure may include the transparent layers between the substrate and the cathode.
• FIG. ISA illustrates an exemplary subpixel circuit cell with forward bias according to aspects of the present disclosure
• FIG. 15B illustrates an exemplary subpixel circuit cell with reverse bias according to aspects of the present disclosure.
• FIG. 1 SA shows an exemplary AMOLED subpixel unit cell circuit (2D-driving TFT circuit with subpixels).
• the driving area may include a driving transistor, switching transistor, holding capacitor, and reverse current sensor.
• FIG. 1SB shows the reverse current read and amplified in the OLED circuit structure.
• FIG. 16 illustrates an exemplary pixel circuit cell with RGB subpixels according to aspects of die present disclosure.
• an AMOLED panel has a three-subpixel structure.
• a blue subpixel can be used as a light source while the neighboring green or red subpixels may be used as a sensor because the band gap of blue subpixels is larger than mat of the green or red subpixels.
• FIG. 16 shows an exemplary R/G/B pixel structure where the blue subpixel is the light source, and the green or red subpixel is die sensor. The reverse voltage can be biased in the sensor subpixel when the lighting subpixel is turned on.
• the I-V curves correspond with subpixel structures m FIG. 16.
• the amount of reverse current in the sensor subpixel under reverse bias is increased when light is reflected, refracted, or scattered from a fingerprint to die sensor subpixel.
• the amount of reverse current can be measured using current sensing circuits in the driving circuit area.
• the reverse current signal can be amplified using an amplification circuit, and/or a signal processor.
• the amplified current signal can then be processed to generate a fingerprint image by a signal processing algorithm.
• the OLED panel resolution can be controlled by varying the size and density of each subpixel and by setting the subpixel structure of the OLED panel.
• an OLED panel may have a larger lighting component (e.g. blue subpixels) and a smaller sensor component (e.g. green and/or red subpixels).
• subpixel structures can have different sizes.
• the subpixel density can be enhanced by changing pixel shape from stripe type to circular or diamond shape.
• an OLED subpixel structure can have different shapes, such as square, rectangle, circle, diamond, etc.
• the patterning of the subpixel structure can be fabricated by using fine metal mask processes, ink-jet printing, or laser transfer technologies.
• FIG. 17 illustrates an exemplary light sensing panel using a thin film transistor (TFT) panel structure according to aspects of the present disclosure.
• TFT thin film transistor
• Each cell of the TFT panel structure can be an addressable light sensing component, referred to as a sensing pixeL
• capture sensor 1700 includes a passivation layer 1718, which can be formed of SiNx.
• passivation layer 1718 On top of passivation layer 1718, a storage capacitor layer is formed including first electrode 1715.
• This storage capacitor layer is preferably formed from indium tin oxide (1TO), which is conductive and transparent.
• an insulating layer 1717 is formed, preferably of SiNx.
• a second electrode 1714 is formed, preferably of tin oxide. First electrode 1715, insulating layer 1717 and second electrode 1714 together form the storage capacitor. Over second electrode 1714, another insulating layer 1716 is formed, which can be formed from SiNx. A layer of glass layer 1711 is placed over insulating layer 1716, A fingerprint to be imaged is placed on glass layer 1711, which may be referred to herein as the imaging surface.
• a light sensing unit 1712 (also referred to as a light sensor), which is preferably a thin-film transistor, and a switching unit 1713, which is also preferably a thin- film transistor, arc horizontally arranged on a passivation layer 1718.
• a back light 1720 irradiates light upward to be passed through the fingerprint capture sensor 1700.
• back light 1720 can be separated from a lower, exposed surface of passivation layer 1718. It is also considered, however, mat backlight 1720 be placed against lower surface of passivation layer 1718.
• Backlight 1720 can he an LED or any other type of light source.
• a source electrode 1712-S of the light sensing unit 1712 and a drain electrode 1713-D of the switching unit 1713 arc electrically connected through second electrode 1714.
• a gate electrode 1712-G of the light sensing unit 1712 is connected to first electrode 1715.
• a first light shielding layer 1713-sh is placed between insulating layer 1717 and passivation layer 1718 at switching unit 1713. As detailed below, first light shielding layer 1713-sh blocks light from backlight 1720 from reaching switching unit 1713.
• second light shielding layer 1722 is positioned between glass layer 1711 and insulating layer 1716 at switching unit 1713 to shield switching unit 1713 from light passing through or reflected from glass layer 1711.
• a photosensitive layer 1712-P such as amorphous silicon (a-Si:H) is formed between the drain electrode 1712-0 and source electrode 1712-S of the light sensing unit 1712.
• photosensitive layer 1712-P allows current to flow in response to a predetermined amount of light striking a surface of photosensitive layer 1712-P. In this way, when more than a predetermined quantity of light is received at a surface of photosensitive layer 1712-P, current flows through the drain electrode 1712-D and the source electrode 1712-S.
• a second light shielding layer 1722 is first placed on glass layer 1711 via evaporation, sputtering or any other method.
• Glass layer 1711 is preferably between about 5 and 10 urn, though may be either thicker or thinner.
• Light shielding layer 1722 is preferably formed from a metal such as aluminum, but may be formed from any suitable light blocking material.
• insulating layer 1716 is formed on top of glass layer 1711 and second light shielding layer 1722. As noted above, insulating layer 1716 is preferably formed from SiNx.
• Photosensitive layer 1712-P is then formed over insulating layer 1716. As discussed above, photosensitive layer 1712-P is preferably formed from a-Si:H.
• drain electrode 1713-D are each preferably formed of ITO, but may be formed of any suitable conductor.
• insulating layer 1717 is formed and over insulating layer 1717 first electrode 1715 is formed. Insulating layer 1717 is preferably formed from SiNx and first electrode 1715 is preferably formed of ⁇ but may be formed of any suitable conductor.
• gate electrode 1712-G of light sensing unit 1712 and light shield 1713-sh are formed. Preferably, gate electrode 1712-G and light shielding layer 1713-sh are each formed of ⁇ , but may be formed of any suitable material and light shielding layer 1713-sh does not need to be formed from the same material as gate electrode 1712-G.
• passivation layer 1718 which is preferably formed from SiNx, is formed over first electrode 1715, gate electrode 1712-G and light shielding layer 1713-sh.
• backlight 1720 can cither be attached to the lower, exposed surface of passivation layer 1718 or separately supported.
• an image capture sensor can have substantially the same structure as capture sensor shown in FIG. 17 except that conductive ITO layer is placed beneath glass layer and an insulating layer, which can be formed of SiNx, is placed below ITO layer. Because ITO layer is conductive, electrostatic charge built up on glass layer can be discharged by connecting ⁇ layer to a ground. This can prevent damage to capture sensor.
• Image capture sensor can be fabricated in substantially the same manner as image capture sensor except that ITO layer is formed over glass layer and insulating layer is formed over ITO layer prior to forming light shielding layer over insulating layer.
• an image capture sensor can have substantially the same structure as capture sensor shown in FIG. 17.
• the capture sensor includes a light sensing unit, which is substantially the same and light sensing unit, and switching unit, which is substantially the same as switching unit, formed between an insulating layer and a passivation layer.
• above insulating layer capture sensor includes a substrate layer having a plurality of fiber-optic strands running in a direction perpendicular to a surface of substrate layer.
• the diameter of the fiber-optic strands 330a forming substrate layer is from about 4 urn to about 8 umin diameter and more preferably about 6 urn in diameters, though larger or smaller diameters can also be used.
• Substrate layer can be formed from glass fiber optic strands 330a or fiber optic strands of other substantially transparent materials including polymers. Fiber optic sheets can be used to form die substrate layer,
• FIG. 18 illustrates an exemplary display with light sensors for detecting changes of light conditions caused by an object 1802 hovering above the display according to aspects of the present disclosure.
• FIG. 19 illustrates an exemplary display with light sensors for detecting changes of light conditions caused by an object 1902 touching the display according to aspects of the present disclosure.
• a light sensor panel can be implemented as an add-on panel that is placed on top of a light source panel
• the light source panel can be, for example, an LCD panel or an AMOLED panel.
• FIG. 18 illustrates a TFT type light sensor panel 1804 is placed on top of an LCD display panel structure 1806 as an add-on panel.
• FIG. 19 illustrates a TFT type light sensor panel 1904 is placed on top of an LCD display panel structure 1906 as an add-on panel.
• a TFT type light sensor panel is placed on top of an LCD panel structure as an add-on panel.
• the sensing pixels of the TFT type light sensing panel can be individually addressable and can be activated according to a designated sensor zone pattern.
• TFT light Sensor panel may be aligned with an LCD panel suoicture, wherein non-transparent components of the TFT light sensor panel are aligned with the black matrix areas of the LCD display panel structure.
• the TFT light sensor panel is aligned with the LCD panel structure.
• the non-transparent components on the TFT light sensor panel are aligned with the black matrix area on the LCD display panel structure.
• the black matrix areas of the LCD display panel arc non-transparent and therefore would block the transmission of the display backlight
• the light sensor panel can be designed so that its non-transparent areas can be aligned with the black matrix areas of the LCD panel.
• the LCD display emits light through the transparent areas of the LCD display, this light can be used as the light source for the light sensor panel.
• the LCD display can individually control cells (individually addressable) to emit light as discrete light sources that are projected into the light refractor according to a designated illumination pattern.
• the light refracting device can, for example, also be a thin-film transistor (TFT) add-on panel placed on top of an LCD or AMOLED display panel structure that acts as a panel of light sources. Incident light from the light source panel is projected through the light receiving surface and projected directly or indirectly onto the imaging surface to create an image of the patterned object from the projected light onto the viewing plane.
• TFT thin-film transistor
• This multi-level command sensing apparatus can be also used as a touch sensor when implemented in a mobile device.
• FIG. 20 illustrates an exemplary controller of a multi-level command sensing apparatus according to aspects of the present disclosure.
• a controller 2000 of the muiti-icvei command sensing apparatus may include one or more processors 2002, a network interface 2004, a database 2006, a multi-level command sensing engine, 2008, a memory 2010, and a user interface 2012.
• the multi-level command sensing apparatus may be a part of a mobile device.
• a mobile device is usually equipped with a touch sensor. If a mobile device was equipped with the multi-level command sensing apparatus of the present disclosure, then the touch sensor would not be required, as the multi-level command sensing apparatus may also be used as a touch sensor.
• a mobile device can be configured to include a multi-level command sensing apparatus for fingerprint recognition.
• the mobile device may comprise a wireless transceiver which is capable of transmitting and receiving wireless signals via wireless antenna over a wireless communication network.
• Wireless transceiver may be connected to a bus by a wireless transceiver bus interface.
• the wireless transceiver bus interface may, in some embodiments be at least partially integrated with wireless transceiver.
• Some embodiments may include multiple wireless transceivers and wireless antennas to enable transmitting and/or receiving signals according to a corresponding multiple wireless communication standards such as, for example, versions of IEEE Std. 802.11, CDMA, WCDMA, LTE, UMTS, GSM, AMPS. Zigbce and Bluetooth*, etc.
• a corresponding multiple wireless communication standards such as, for example, versions of IEEE Std. 802.11, CDMA, WCDMA, LTE, UMTS, GSM, AMPS. Zigbce and Bluetooth*, etc.
• the mobile device may also comprise a SPS receiver capable of receiving and acquiring SPS signals via a SPS antenna.
• the SPS receiver may also process, in whole or in part, acquired SPS signals for estimating a location of the mobile device.
• processors may also be utilized to process acquired SPS signals, in whole or in part, and/or calculate an estimated location of the mobile device, in conjunction with the SPS receiver. Storage of SPS or other signals for use hi performing positioning operations may be performed in memory or registers (not shown).
• the mobile device may comprise digital signal processors) (DSP(s)) connected to the bus by a bus in terface, processors) connected to die bus by a bus interface and memory.
• DSP(s) digital signal processors
• the bus interface may be integrated with die DSP(s), processors) and memory.
• functions may be performed in response execution of one or more machine-readable instructions stored in memory such as on a computer-readable storage medium, such as RAM, ROM, FLASH, or disc drive, just to name a few example.
• the one or more instructions may be executable by processors), specialized processors, or DSP(s).
• the memory may comprise a non-transitory processor-readable memory and/or a computer-readable memory that stores software code (programming code, instructions, etc.) dial are executable by processors) and/or DSP(s) to perform functions described herein.
• the wireless transceiver may communicate with processors) and/or DSP(s) through the bus to enable the mobile device to be configured as a wireless station as discussed above.
• Processors) and/or DSP(s) may execute instructions to execute one or more aspects of processes/methods discussed above in connection with FIGs.2IA- 21D.
• a user interface may comprise any one of several devices such as, for example, a speaker, microphone, display device, vibration device, keyboard, touch screen, etc.
• the user interface may enable a user to interact with one or more applications hosted on die mobile device.
• devices of user interface may store analog or digital signals on the memory to be further processed by DSP(s) or processor in response to action from a user.
• applications hosted on the mobile device may store analog or digital signals on the memory to present an output signal to a user.
• die mobile device may optionally include a dedicated audio input/output (I/O) device comprising, for example, a dedicated speaker, microphone, digital to analog circuitry, analog to digital circuitry, amplifiers and/or gain control.
• the mobile device may comprise touch sensors responsive to touching or pressure on a keyboard or touch screen device.
• the mobile device may also comprise a dedicated camera device for capturing still or moving imagery.
• the dedicated camera device may comprise, for example an imaging sensor (e.g., charge coupled device or CMOS imager), lens, analog to digital circuitry, frame buffers, etc.
• additional processing, conditioning, encoding or compression of signals representing captured images may be performed at the processors) or DSP(s).
• a dedicated video processor may perform
• the dedicated video processor may decodc ⁇ decompress stored image data for presentation on a display device on the mobile device.
• the mobile device may also comprise sensors coupled to the bus which may include, for example, inertial sensors and environment sensors.
• Inertial sensors may comprise, for example accelerometers (e.g., collectively responding to acceleration of the mobile device in three dimensions), one or more gyroscopes or one or more magnetometers (e.g., to support one or more compass applications).
• Environment sensors of the mobile device may comprise, for example, temperature sensors, barometric pressure sensors, ambient light sensors, and camera imagers, microphones- just to name tew examples.
• the sensors may generate analog or digital signals that may be stored in memory and processed by DPS(s) or processors) in support of one or more applications such as, for example, applications directed to positioning or navigation operations.
• the mobile device may comprise a dedicated modem processor capable of performing baseband processing of signals received and down- converted at a wireless transceiver or SPS receiver.
• the dedicated modem processor may perform baseband processing of signals to be up-converted for transmission by the wireless transceiver.
• baseband processing may be performed by a processor or DSP (e.g., proccssor($) or DSP(s)).
• FIG. 21 A illustrates method of performing multi-level command sensing according to aspects of the present disclosure.
• the method detects a leakage current corresponding to changes of light conditions on a display by one or more light sensors of a muni-level command sensing apparatus
• the method determines an action performed by a user based on the changes of light conditions on the display by a controller of the multi-level command sensing apparatus.
• the method determines a command based on the action performed by the controller of the multi-level command sensing apparatus.
• the method executes the command by the controller of the multi-level command sensing apparatus.
• the changes of light conditions may comprise a sequence of shadows detected on the display, and the action performed by the user may comprise a sequence of hovering motions without touching the display.
• the changes of light conditions may also comprise a sequence of brightened shadows detected on die display, and the action performed by the user may comprise a sequence of touches on the display.
• the sequence of brightened shadows can be caused by reflected light and scattered light from an object touching the display.
• the sequence of touches may include a sequence of low pressure touches predefined by the user.
• the sequence of touches may include a sequence of high pressure touches predefined by me user,
• FIG. 21B illustrates a method of determining an action performed by a user based on changes of tight conditions on a display according to aspects of the present disclosure.
• the method compares the changes of light conditions on the display to a set of predefined changes of light conditions stored in a database of the multi-level command sensing apparatus.
• the method identifies the action performed by the user corresponding to the changes of light conditions on the display in response to a match being found in the set of predefined changes of light conditions.
• FIG. 2 IC illustrates determining a command based on the action performed by the user according to aspects of the present disclosure.
• the method compares the action performed by the user to a set of predefined actions stored in a database of the multi-level command sensing apparatus.
• the method identifies the command corresponding to the action performed by the user in response to a match being found in the set of predefined actions.
• FIG.21D illustrates a method of authenticating a user based on a sequence of touches on the display by the user according to aspects of the present disclosure.
• the method authenticates the user based on the sequence of touches on the display.
• the method may continuously authenticate the user during an access to security sensitive information using the multi-level command sensing apparatus, and terminate the access to the security sensitive information in response to one or more mismatches found in the continuously authenticating process.
• the invention can be implemented in any suitable form, including hardware, software, firmware, or any combination of these.
• the invention may optionally be implemented partly as computer software running on one or more data processors and/or digital signal processors.
• the elements and components of an embodiment of the invention may be physically, functionally, and logically implemented in any suitable way. Indeed, the functionality may be implemented in a single unit, in a plurality of units, or as part of other functional units. As such, the invention may be implemented in a single unit or may be physically and functionally distributed between different units and processors.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• General Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Human Computer Interaction (AREA)
• Computer Security & Cryptography (AREA)
• Software Systems (AREA)
• Computer Hardware Design (AREA)
• Multimedia (AREA)
• Databases & Information Systems (AREA)
• Health & Medical Sciences (AREA)
• Bioethics (AREA)
• General Health & Medical Sciences (AREA)
• User Interface Of Digital Computer (AREA)
• Position Input By Displaying (AREA)

Priority Applications (3)

Applications Claiming Priority (4)

Publications (1)

Family

ID=56622232

Family Applications (1)

Country Status (6)

Cited By (1)

Families Citing this family (27)

Citations (6)

Family Cites Families (42)

• 2016
  • 2016-04-29 US US15/143,303 patent/US9454259B2/en active Active
  • 2016-08-15 CN CN201680083017.5A patent/CN108700954A/zh active Pending
  • 2016-08-15 EP EP16884131.0A patent/EP3400510B1/en active Active
  • 2016-08-15 KR KR1020187022537A patent/KR102144205B1/ko active IP Right Grant
  • 2016-08-15 WO PCT/US2016/046983 patent/WO2017119929A1/en active Application Filing
  • 2016-09-06 TW TW105128756A patent/TWI606387B/zh active
  • 2016-09-06 US US15/257,733 patent/US9606672B2/en active Active
  • 2016-09-21 US US15/272,269 patent/US9830009B2/en active Active

Patent Citations (6)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events