CN109791599B

CN109791599B - Under-screen optical sensor module for on-screen fingerprint sensing

Info

Publication number: CN109791599B
Application number: CN201780047331.2A
Authority: CN
Inventors: 皮波; 何毅
Original assignee: Shenzhen Goodix Technology Co Ltd
Current assignee: Shenzhen Goodix Technology Co Ltd
Priority date: 2016-09-17
Filing date: 2017-08-03
Publication date: 2023-11-14
Anticipated expiration: 2037-08-03
Also published as: CN109791599A; EP3479295A4; EP3479295A1; WO2018049944A1

Abstract

An apparatus (200) and an optical sensor module (702) are provided for providing on-screen fingerprint optical sensing by using an off-screen optical sensor module (702), the off-screen optical sensor module (702) capturing and detecting return light emitted by an LCD display screen for displaying an image and reflected by a top surface of a display assembly (423).

Description

Under-screen optical sensor module for on-screen fingerprint sensing

Priority claims and related patent applications

This patent document claims the benefit and priority of the following prior patent applications: (1) U.S. provisional patent application No. 62/396,153 entitled "LCD underscreen optical sensor module for on-screen fingerprint sensing" filed at 9/17 of 2016; (2) U.S. provisional patent application serial No. 62/412,777 entitled "LCD underscreen optical sensor module for on-screen fingerprint sensing" filed 10/25/2016; and (3) U.S. provisional patent application filed on 7, 3, 2017, with application number 62/468,337 entitled "LCD underscreen optical sensor module for on-screen fingerprint sensing through a peripheral taskbar display area in a device display".

This patent document also claims the benefit and priority of and is part of the continuous application filed on 7, 6, 7, 2017, entitled "optical collimator for on-screen fingerprint sensing under-screen optical sensor module". The above-mentioned U.S. patent application 15/616,856 also claims the benefits and priorities of the following prior patent applications: (1) U.S. provisional patent application Ser. No. 62/347,073 filed on 6/7 of 2016; (2) U.S. provisional patent application Ser. No. 62/363,832 filed at 7/18/2016; (3) U.S. provisional patent application Ser. No. 62/363,823 filed at 7/18/2016; (4) U.S. patent application Ser. No. 15/421,249 filed on 1/31/2017 (as part of its continuous application), which also claims the benefits and priority of the following prior patent applications: (5) U.S. provisional patent application Ser. No. 62/289,328 filed on day 31 of 1/2016; (6) U.S. provisional patent application Ser. No. 62/330,833 filed 5/2/2016; (7) U.S. provisional patent application Ser. No. 62/347,073 filed on 6/7 of 2016; and (8) International patent application number PCT/US2016/038445 filed on 6/20/2016 (as part of the continuous application thereof), and published on 12/22/2016 with publication number WO2016/205832, which also claims the benefit and priority of U.S. provisional patent application number 62/181,718 filed on 18/2015.

In addition, the present patent document claims the benefit and priority of U.S. patent application Ser. No. 15/421,249 entitled "on-screen optical sensor Module for on-screen fingerprint sensing" filed on 1/31/2017. U.S. patent application 15/421,249 also claims the benefits and priority of the following prior patent applications: (1) U.S. provisional patent application Ser. No. 62/289,328 filed on day 31 of 1/2016; (2) U.S. provisional patent application Ser. No. 62/330,833 filed 5/2/2016; (3) U.S. provisional patent application Ser. No. 62/347,073 filed 6/20/2016; and (4) International patent application number PCT/US2016/038445 filed on 6/20/2016 (as part of the continuous application thereof), which also claims the benefit and priority of (5) U.S. provisional patent application number 62/181,718 filed on 18/2015.

The entire disclosure of the above-mentioned patent application is incorporated by reference as part of this patent document.

Technical Field

This patent document relates to an optical sensor module capable of performing one or more sensing operations, such as fingerprint or other parameter measurements, based on optical sensing in an electronic device (e.g., a mobile device, a wearable device, or a larger system).

Background

The various sensors may be implemented in an electronic device or system to provide certain desired functions.

A sensor that enables user authentication is one example of a sensor for a device that includes a portable or mobile computing device (e.g., laptop, tablet, smart phone), gaming system, various databases, information systems, or larger computer control system, and may use a user authentication mechanism to protect personal data and prevent unauthorized access. User authentication on an electronic device may be performed by one or more forms of biometric identifiers, which may be used alone or on the basis of conventional cryptographic authentication methods. One common form of biometric identifier is a fingerprint pattern of a person. The fingerprint sensor may be built into the electronic device to read the fingerprint pattern of the user so that the device can only be unlocked by an authorized user of the device by authenticating the fingerprint pattern of the authorized user. Another example of a sensor for an electronic device or system is a biomedical sensor, e.g. a heartbeat sensor in a wearable device like a wristband device or a watch, etc. In summary, different sensors may be provided in the electronic device to achieve different sensing operations and functions.

The fingerprint may be used for user authentication to access an electronic device, computer control system, electronic database or information system, either as a separate authentication method or in combination with one or more authentication methods such as a password authentication method or the like. For example, the devices include portable or mobile computing devices (e.g., laptops, tablets, smartphones) and gaming systems, user authentication mechanisms may be used to secure personal data and prevent unauthorized access. As another example, a computer or computer controlled device or system for an organization or business should be protected to allow access only by authorized personnel to protect information or use of the organization or business's device or system. The information stored in the portable device and computer controlled databases, devices or systems may be personal information in nature, such as personal contacts or phone books, personal photos, personal health information or other personal information, or confidential information used exclusively by organizations or businesses, such as business financial information, employee data, business secrets, and other proprietary information. If the security of accessing the electronic device or system is compromised, such data may be accessed by others, resulting in loss of personal privacy or loss of valuable confidential information. In addition to the security of information, protecting access to computers and computer-controlled devices or systems also allows for the use of devices or systems controlled by a computer or computer processor, such as computer-controlled automobiles and other systems such as ATMs.

Secure access to devices such as mobile devices or systems such as electronic databases, computer controlled systems, etc. may be achieved in different ways, for example using user passwords. However, passwords can be easily propagated or obtained, and such properties of passwords can reduce security levels. Moreover, users need to remember a password to use an electronic device or system, and if the user forgets the password, the user needs to take some password recovery procedure to gain authentication or otherwise regain access to the device, which can be cumbersome for the user and have various practical limitations and inconveniences. Personal fingerprinting may be used to enable user authentication to mitigate some of the undesirable effects associated with passwords while enhancing data security.

An electronic device or system including a portable or mobile computing device may use a user authentication mechanism to protect personal or other confidential data and to prevent unauthorized access. User authentication on an electronic device or system may be performed in one or more forms of biometric identifiers, which may be used alone or on the basis of conventional cryptographic authentication methods. One form of biometric identifier is a fingerprint pattern of a person. The fingerprint sensor may be built into the electronic device or information system to read the fingerprint pattern of the user so that the device can only be unlocked by an authorized user of the device by authenticating the fingerprint pattern of the authorized user.

Disclosure of Invention

The optical sensor module may be placed under a liquid crystal display (liquid crystal display, LCD) screen to provide optical sensing functions including optical fingerprint sensing. In some implementations, optical sensing is provided for determining whether a contacted object is from a living person.

In one aspect, the disclosed technology may be used to construct an electronic device capable of detecting a fingerprint by optical sensing, comprising: a liquid crystal display (liquid crystal display, LCD) screen providing touch sensing operation and including an LCD display panel structure to display an image; a top transparent layer formed over the device screen as a user touch interface for the touch sensing operation and as an interface for transmitting light from the display panel structure to display an image to a user; and an optical sensor module under the display panel structure to receive the detection light passing through the LCD screen to detect a fingerprint, wherein the optical sensor module includes an optical collimator array of optical collimators receiving the detection light, and an optical sensor array of optical sensors receiving the detection light from the optical collimator array.

In another aspect, the disclosed technology may be used to construct an electronic device capable of detecting a fingerprint by optical sensing, the electronic device comprising: (1) A liquid crystal display (liquid crystal display, LCD) screen providing touch sensing operation and including an LCD display panel structure to display an image; an LCD backlight module coupled to the LCD screen to generate backlight to the LCD screen for displaying images; (2) A top transparent layer formed over the device screen as a user touch interface for the touch sensing operation and as an interface for transmitting light from the display panel structure to display an image to a user; (3) An optical sensor module located under the LCD display panel structure to receive the detection light reflected from the top transparent layer and passing through the LCD screen to detect a fingerprint; (4) One or more detection light sources separated from the LCD backlight module and located below the LCD display panel structure to generate detection light passing through the LCD display panel structure to illuminate a designated fingerprint sensing area on the top transparent layer, the fingerprint sensing area being visually different from a surrounding area of the top transparent layer for a user to place a finger for optical fingerprint sensing; and (5) a device control module coupled to the optical sensor module and processing the output of the optical sensor module to determine whether a fingerprint detected by the optical sensor module matches a fingerprint of an authorized user and to detect a biometric parameter different from the fingerprint by optical sensing in addition to detecting the fingerprint to indicate whether a touch associated with the detected fingerprint at the top transparent layer is from a living person.

In yet another aspect, the disclosed technology may be used to construct an electronic device capable of detecting a fingerprint by optical sensing, the electronic device comprising: a liquid crystal display (liquid crystal display, LCD) screen providing touch sensing operation and including an LCD display panel structure to display an image; an LCD backlight module coupled to the LCD screen to generate backlight to the LCD screen to display an image; a top transparent layer formed over the LCD screen as a user touch interface for the touch sensing operation and as an interface for transmitting light from the display panel structure to display an image to a user; an optical sensor module is positioned below the LCD panel structure to receive light returned from the top transparent layer to detect a fingerprint. The optical sensor module includes a transparent block in contact with the display panel substrate to receive light from the display panel structure, an optical sensor array to receive the light, and an optical imaging module to image the light received in the transparent block onto the optical sensor array. One or more detection light sources, separate from the LCD backlight module, are positioned below the LCD display panel structure to generate the detection light passing through the LCD display panel structure, and further illuminate a designated fingerprint sensing area on the top transparent layer, the fingerprint sensing area being visually different from a surrounding area of the top transparent layer for a user to place a finger for optical fingerprint sensing.

The foregoing and other aspects, implementations, and features of the disclosed technology are described in more detail in the following figures, description, and claims.

Drawings

Fig. 1 is a block diagram of an example of a system with a fingerprint sensing module that may be implemented to include the optical fingerprint sensor disclosed in this document.

Fig. 2A and 2B illustrate one exemplary implementation of an electronic device 200, the electronic device 200 having a touch-sensitive display screen assembly and an optical sensor module positioned below the touch-sensitive display screen assembly.

Fig. 3A and 3B show examples of devices implementing the optical sensor modules in fig. 2A and 2B.

Fig. 4A and 4B illustrate an example of one implementation of an optical sensor module located below a display screen assembly for implementing the designs of fig. 2A and 2B.

Fig. 5A, 5B and 5C illustrate signal generation for light returned from a sensing area on a top sensing surface under two different optical conditions to facilitate understanding of the operation of an off-screen optical sensor module.

FIGS. 6A-6C, 7, 8A-8B, 9, and 10A-10B illustrate example designs of an off-screen optical sensor module.

Fig. 11 shows the imaging of a fingerprint sensing area on a top transparent layer by an imaging module under different lay down conditions, wherein the imaging device images the fingerprint sensing area onto an optical sensor array and the imaging device may be optically transmissive or optically reflective.

Fig. 12 illustrates an example of operations of a fingerprint sensor to reduce or eliminate unwanted contributions from background light in fingerprint sensing.

Fig. 13 illustrates a process of operating an off-screen optical sensor module for capturing a fingerprint pattern.

Fig. 14, 15, and 16 show examples of an operation procedure of determining whether an object in contact with an LCD display screen is a part of a living finger by illuminating the finger with light of two different colors.

Figures 17A-17B, 18 and 19A-19C illustrate optical collimator designs for optical fingerprint sensing suitable for implementing the disclosed underscreen optical sensor module technology.

Fig. 20, 21, 22A and 22B show examples of various designs for fingerprint sensing using an off-screen optical sensor module that uses an array of optical collimators or pinholes to direct signal light carrying fingerprint information to the array of optical sensors.

Fig. 23 and 24 show examples of an off-screen optical sensor module with an optical collimator.

Fig. 25 shows an example of an optical collimator array that reduces background light reaching a photodiode array in an off-screen optical sensor module using optical filtering.

Fig. 26A, 26B, 27 and 28 show examples of optical collimator designs for optical sensing under LCD display screens.

Fig. 29, 30 and 31 show improved optical imaging resolution based on pinhole camera effects when designing an optical sensor module.

Fig. 32 shows an example of an LCD under-screen optical sensor module that optically senses using an array of optical pinholes.

Fig. 33 includes fig. 33A and 33B, and shows an example of an optical fingerprint sensor under an LCD display panel with an optical deflection or diffraction device or layer.

34A, 34B and 34C illustrate examples of LCD diffuser layer designs for improved underscreen optical sensing.

Fig. 35A and 35B illustrate examples of LCD reflective layer designs for improved underscreen optical sensing.

Fig. 36 shows an example of an LCD light source design for improved underscreen optical sensing of an LCD.

Fig. 37 shows an example of enhancement characteristics for improved underscreen optical sensing of an LCD.

FIG. 38 shows an example of an LCD waveguide design for improved under-screen optical sensing of an LCD.

Fig. 39 shows an example of an LCD backlight source and illumination source for improved underscreen optical sensing of an LCD.

Fig. 40 shows two different fingerprint patterns of the same finger at different pressing forces: a lightly pressed fingerprint and a heavily pressed fingerprint.

Detailed Description

An electronic device or system may be equipped with a fingerprint authentication mechanism to improve security of an access device. Such electronic devices or systems may include portable or mobile computing devices, such as smartphones, tablets, wrist-worn devices, and other wearable or portable devices, as well as larger electronic devices or systems, such as personal computers in portable or desktop form, ATMs, various terminals for commercial or government use to various electronic systems, databases or information systems, and including automobiles, boats, trains, planes, and other motor transportation systems.

Fingerprint sensing is useful in mobile applications and other applications where secure access is used or required. For example, fingerprint sensing may be used to provide secure access to mobile devices and secure financial transactions, including online shopping. It is desirable to include robust and reliable fingerprint sensing suitable for mobile devices and other applications. In mobile, portable, or wearable devices, fingerprint sensors are expected to minimize or eliminate occupancy of fingerprint sensing due to limited space on these devices, especially considering the need for maximum display area on a given device.

The light generated by the display screen for displaying an image must pass through the top surface of the display screen in order to be seen by the user. A finger may touch the top surface to interact with light at the top surface such that reflected or scattered light at the touched surface area carries the spatial image information of the finger and returns to the display panel below the top surface. In touch-sensitive display devices, the top surface is the touch-sensitive interface that interfaces with the user, and this interaction between the light used to display the image and the user's finger or hand occurs continuously, but this information-carrying light that returns to the display panel is wasted in a large amount and is not used in most touch-sensitive devices. Among various mobile or portable devices having touch-sensitive display and fingerprint-sensitive functions, fingerprint sensors are often devices separate from the display screen, or are disposed on the same surface as the display screen outside the display screen area, such as in some models of apple iPhone and samsung smartphones, or on the back of smartphones, such as in some models of smart phones like chinese, association, millet or google, to avoid taking up valuable space on the front for placing a large display screen. These fingerprint sensors are devices that are separate from the display screen, requiring compactness to save space for display and other functions, while also providing reliable and fast fingerprint sensing with spatial image resolution above a certain acceptable level. However, in many fingerprint sensors, the need for compactness and the need to provide a high spatial image resolution for capturing fingerprint patterns are in direct conflict with each other, as the high spatial image resolution when capturing fingerprint patterns based on various suitable fingerprint sensing technologies (e.g. capacitive touch sensing or optical imaging) requires a large sensor area with a large number of sensing pixels.

Examples of sensor technology and implementations of sensor technology described in this patent document provide an optical sensor module that uses, at least in part, light from a display screen as illumination probe light to illuminate a fingerprint sensing area on a touch sensing surface of the display screen to perform one or more sensing operations based on optical sensing of such light. One suitable display screen for implementing the disclosed optical sensor technology may be based on a variety of display technologies or configurations, including display screens having light emitting display pixels without the use of a backlight, where each individual pixel generates light for forming a display image on the screen, such as a liquid crystal display (liquid crystal display, LCD) screen, an organic light emitting diode (organic light emitting diode, OLED) display screen, or an electroluminescent display screen.

In disclosed examples of integrating optical sensing into an LCD based on the disclosed optical sensor technology, an off-screen LCD optical sensor may be used to detect a portion of light used to display an image in an LCD screen, where the portion of light used to display the screen may be scattered light, reflected light, or some stray light. For example, in some implementations, image light of a backlight-based LCD screen may be reflected or scattered back into the LCD display screen as returned light when it encounters a user's finger or palm, or an object such as a user pointer device of a stylus. Such returned light may be captured to perform one or more optical sensing operations using the disclosed optical sensor techniques. Since light from the LCD display screen is used for optical sensing, an optical sensor module based on the disclosed optical sensor technology can be specifically designed to be integrated into the LCD display screen in the following manner: optical sensing operations and functions are provided to enhance the overall functionality, device integration, and user experience of an electronic device or system (e.g., a smart phone, tablet computer, or mobile/wearable device) while maintaining the display operations and functions of an LCD display screen without interference.

In addition, in various implementations of the disclosed optical sensing technology, one or more designated detection light sources may be provided to generate additional illumination detection light for optical sensing operations by the under-LCD screen optical sensing module. In such applications, the backlight from the LCD screen and the probe light from one or more designated probe light sources together form illumination light for optically sensing operation.

With respect to additional optical sensing functions other than fingerprint detection, optical sensing may be used to measure other parameters. For example, the disclosed optical sensor technology is capable of measuring the pattern of a person's palm of a large touch area available across the entire LCD display screen (in contrast, some designated fingerprint sensors, such as those in the home button of an apple iPhone/iPad device, have quite small and designated off-screen fingerprint sensing areas that are highly limited in the size of the sensing area, which may not be suitable for sensing large patterns). As another example, the disclosed optical sensor technology may be used not only to capture and detect patterns of a finger or palm associated with a person using optical sensing, but may also use optical sensing or other sensing mechanisms, such as different light absorption behaviors of blood at different wavelengths of light, to detect whether the captured or detected pattern of a fingerprint or palm is from a live person's hand via a "live finger" detection mechanism, which may be based on the fact that: the fingers of a person are typically moving or stretching due to the person's natural movement or motion (intentional or unintentional), or the fingers of a living person are typically pulsed as blood flows through the body in connection with the heartbeat. In one implementation, the optical sensor module may detect a change in returned light from the finger or palm due to a heartbeat/blood flow change, thereby detecting whether a living heartbeat is present in an object that appears as a finger or palm. User authentication may enhance access control based on a combination of optical sensing of fingerprint/palm patterns and positive determination of the presence of living persons. For another example, the optical sensor module may include a sensing function for measuring glucose level or oxygen saturation based on optical sensing of returned light from a finger or palm. As another example, when a person touches an LCD display screen, the change in touch force can be reflected in one or more ways, including deformation of the fingerprint pattern, a change in contact area between the finger and the screen surface, widening of the fingerprint ridge, or dynamic changes in blood flow. These and other variations can be measured by optical sensing based on the disclosed optical sensor technology and can be used to calculate touch force. Such touch force sensing can be used to add more functionality to the optical sensor module than fingerprint sensing.

For useful operational or control features related to touch sensing aspects of an LCD display screen, the disclosed optical sensor technology may provide a trigger function or additional function based on one or more sensing results from the optical sensor module to perform certain operations related to touch sensing control of the LCD display screen. For example, the optical properties (e.g., refractive index) of the finger skin are typically different from other artifacts. The optical sensor module may be accordingly designed to selectively receive and detect returned light caused by a finger in contact with the surface of the LCD display screen, while returned light caused by other objects is not detected by the optical sensor module. Such object selective optical detection may be used to provide useful user control through touch sensing, such as waking up a smartphone or device only via a touch of a human finger or palm, while touches of other objects do not cause waking up of the device for energy-saving operation and prolonged use of the battery. This operation may be accomplished by control based on the output of the optical sensor module to control the wake-up circuit operation of the LCD display screen by turning off most of the LCD pixels (and also turning off the LCD backlight) to be in "sleep" mode without lighting, while turning on one or more illumination sources (e.g., LEDs) for the optical sensor module under the LCD panel in flash mode to intermittently flash to the screen surface to sense any touch of a person's finger or palm. With this design, the optical sensor module operates one or more illumination sources to produce a "sleep" mode wake-up induced blinking light, such that the optical sensor module is able to detect the return light of such wake-up induced light caused by a finger touching on the LCD display screen, and in response to the front detection, turn on or "wake-up" the LCD backlight and the LCD display screen. In some implementations, the wake-up sensing light may be in a spectral range where infrared light is not visible, so the user does not experience any visual flickering of light. The LCD display screen operation may be controlled to provide improved fingerprint sensing by eliminating the optically sensed backlight for the fingerprint. For example, in one implementation, one frame fingerprint signal is generated per display scan frame. If two frames of fingerprint signals are generated that are related to the screen display, wherein one frame of fingerprint signal is generated when the LCD display screen is on and the other frame of fingerprint signal is generated when the LCD display screen is off, the difference of the two frames of fingerprint signals can be used to reduce the effect of ambient background light. In some implementations, by operating the fingerprint sensing frame rate to be half the display frame rate, background light noise in fingerprint sensing can be reduced.

An optical sensor module based on the disclosed optical sensor technology can be coupled to the back of an LCD display screen without creating a designated area on the surface side of the LCD display screen that would take up valuable device surface space in some smart phones, tablet computers, or wearable devices and the like. This aspect of the disclosed technology may be used to provide certain advantages or benefits in device design and product integration or manufacturing.

In some implementations, an optical sensor module based on the disclosed optical sensor technology may be configured as a non-invasive module that can be easily integrated into a display screen without requiring changes to the design of the LCD display screen to provide the desired optical sensing functionality, such as fingerprint sensing. In this regard, an optical sensor module based on the disclosed optical sensor technology may be independent of the design of a particular LCD display screen design due to the following properties of the optical sensor module: the optical sensing of such an optical sensor module is performed by detecting light emitted by one or more illumination sources of the optical sensor module and returned from the top surface of the display area, and the disclosed optical sensor module is coupled as an under-screen optical sensor module to the back surface of the LCD display screen for receiving the returned light from the top surface of the display area, thereby eliminating the need for a specific sensing port or sensing area separate from the display screen area. Thus, such an off-screen optical sensor module may be used in combination with an LCD display screen to provide optical fingerprint sensing and other sensor functions on the LCD display screen without using a specially designed LCD display screen with hardware specifically designed to provide such optical sensing. This aspect of the disclosed optical sensor technology enables various LCD display screens in smartphones, tablets, or other electronic devices to have enhanced functionality from optical sensing of the disclosed optical sensor technology.

For example, for existing phone component designs that do not provide a separate fingerprint sensor, like some apple iPhone or samsung Galaxy smartphones, such existing phone component designs may integrate an off-screen optical sensor module as described herein without changing the touch-sensitive display screen component to provide increased on-screen fingerprint sensing functionality. Because the disclosed optical sensing does not require a separate designated sensing area or port, integration of on-screen fingerprint sensing as disclosed herein does not require substantial changes to existing phone component designs or touch-sensitive display modules having a touch-sensitive layer and a display layer, like some apple iPhone/samsung Galaxy phones have front fingerprint sensors outside the display screen area, or some smartphones like hua, millet, google, or some models of association have designated rear fingerprint sensors on the back. Based on the optical sensing technology disclosed in this document, there is no need for external sensing ports and external hardware buttons on the outside of the device, which requires the addition of the disclosed optical sensor module for fingerprint sensing. The added optical sensor module and associated circuitry is under the display screen within the phone housing and can conveniently fingerprint sense on the same touch sensing surface of the touch screen.

As another example, due to the above-described nature of optical sensor modules for fingerprint sensing, smartphones incorporating such optical sensor modules can be updated with improved designs, functionality, and integration mechanisms without impacting or burdening the design or manufacture of LCD display screens to provide desired flexibility for device manufacturing and improvement/upgrades in product cycles while maintaining availability of updated versions of optical fingerprint sensors in smartphones, tablet computers, or other electronic devices that use LCD display screens. In particular, the touch sensitive layer or LCD display layer can be updated at the next product release without any significant hardware changes to the fingerprint sensing features implemented with the disclosed underscreen optical sensor module. In addition, by using a new version of the off-screen optical sensor module, on-screen optical sensing for fingerprint sensing or other optical sensing functions that is improved by such an optical sensor module can be added to the new product version by using the new version of the off-screen optical sensor module without requiring significant changes to the handset assembly design, including adding additional optical sensing functions.

The above or other features of the disclosed optical sensor technology may be implemented to provide improved fingerprint sensing and other sensing functions to new generation electronic devices, particularly for smartphones, tablet computers, and other electronic devices having an LCD display screen that provides various touch sensing operations and functions and enhances the user experience of such devices. The features of the optical sensor module disclosed in this patent document are applicable to various display panels based on different technologies including both LCD and OLED displays. The following specific examples are directed to an LCD display panel and an optical sensor module placed under the LCD display panel.

In implementations of the disclosed features, additional sensing functions or sensing modules such as biomedical sensors may be provided, for example, a heartbeat sensor in a wearable device like a wristband device or a wristwatch. In general, different sensors may be provided in an electronic device or system to achieve different sensing operations and functions.

The disclosed techniques may be implemented as devices, systems, and techniques that provide for performing optical sensing and authentication of human fingerprints to verify access attempts to locked computer-controlled devices (such as mobile devices or computer-controlled systems) that are equipped with a fingerprint detection module. The disclosed technology may be used to secure access to a variety of electronic devices and systems, including portable or mobile computing devices (such as laptop computers, tablet computers, smartphones, and gaming devices), as well as other electronic devices or systems (such as electronic databases, automobiles, bank ATMs), and the like.

Fig. 1 is a block diagram of an example of a system 180 having a fingerprint sensing module including a fingerprint sensor 181, which may be implemented to include an optical fingerprint sensor based on optical fingerprint sensing as disclosed in this document. The system 180 includes a fingerprint sensor control circuit 184 and a processor 186, which processor 186 may include one or more processors for processing the fingerprint pattern and determining whether the input fingerprint pattern is that of an authorized user. The system 180 uses the fingerprint sensor 181 to obtain a fingerprint and compares the obtained fingerprint to a stored fingerprint to enable or disable functions in the device 188 that are protected by the system 180. In operation, the processor 186 controls access to the device 188 based on whether the captured user fingerprint is from an authorized user. As shown, the fingerprint sensor 181 may include a plurality of fingerprint sensing pixels, such as pixels 182A-182E that collectively represent at least a portion of a fingerprint. For example, the system 180 may be implemented at an ATM as the device 188 to determine a fingerprint of a customer requesting access to funds or other transactions. Based on a comparison of the user's fingerprint obtained from fingerprint sensor 181 with one or more stored fingerprints, system 180 may cause device 188 to grant the requested access to the user account in response to positive identification, or may deny access in response to negative identification. As another example, the device 188 may be a smart phone or a portable device, and the system 180 is a module integrated into the device 188. As another example, the device 188 may be a door or security portal of a facility or home that uses the fingerprint sensor 181 to grant or deny access. As another example, the device 188 may be an automobile or other vehicle that uses the fingerprint sensor 181 to link to the start of the engine and identify whether a person is authorized to operate the automobile or vehicle.

As a specific example, fig. 2A and 2B illustrate one exemplary implementation of an electronic device 200, the electronic device 200 having a touch-sensitive display screen assembly and an optical sensor module positioned below the touch-sensitive display screen assembly. In this particular example, the display technology may be implemented by using a backlight to optically illuminate an LCD display screen of LCD pixels, or by another display screen (e.g., an OLED display screen) having light emitting display pixels without using a backlight. The electronic device 200 may be a portable device such as a smart phone or tablet computer, and the electronic device 200 may be the device 188 shown in fig. 1.

Fig. 2A illustrates a front side of an electronic device 200 that is similar to some features in some existing smartphones or tablets. The device screen occupies all, most, or a significant portion of the front side space on the front side of the electronic device 200 and provides fingerprint sensing functionality on the device screen, such as one or more sensing areas for receiving a finger on the device screen. As an example, fig. 2A shows a fingerprint sensing area for finger touch in a device screen that may be illuminated as a clearly identifiable region or area for a user to place a finger for fingerprint sensing. Such a fingerprint sensing area may be used to display an image like the rest of the device screen. As shown, in various implementations, the device housing of the electronic device 200 may have sides that support side control buttons commonly found in various smartphones currently on the market. Also, as shown in one example of the upper left corner of the device housing in fig. 2A, one or more optional sensors may be provided on the front side of the electronic device 200 outside the device screen.

Fig. 2B shows an example of the structural configuration of a module related to optical fingerprint sensing disclosed in this document in the electronic apparatus 200. The device screen assembly shown in fig. 2B includes: for example, a touch-sensitive screen module having a touch-sensitive layer on top, and a display screen module having a display layer under the touch-sensitive module. An optical sensor module is coupled to and below the display screen assembly module to receive and capture the returned light from the top surface of the touch-sensitive screen module and direct and image the returned light onto an optical sensor array of optical-sensitive pixels or photodetectors that converts optical images in the returned light into pixel signals for further processing. Below the optical sensor module is a device electronics structure that contains some of the electronic circuitry for the optical sensor module and other components in the electronic device 200. The device electronics may be disposed inside the device housing and may include a portion of the underside of the optical sensor module as shown in fig. 2B.

In some implementations, the top surface of the device screen assembly may be a surface of an optically transparent layer that serves as a user touch-sensitive surface to provide a variety of functions, such as (1) a display output surface through which light carrying a display image passes to the viewer's eye, (2) a touch-sensitive interface that receives user touches for touch-sensitive operation of the touch-sensitive module, and (3) an optical interface for on-screen fingerprint sensing (and possibly one or more other optical sensing functions). Such an optically transparent layer may be a rigid layer such as a glass or crystalline layer or a flexible layer.

One example of a display screen is an LCD display having an LCD layer and a thin film transistor (thin film transistor, TFT) structure or substrate. The LCD display panel is a multi-layer liquid crystal display (liquid crystal display, LCD) module that includes an LCD display backlight source (e.g., an LED lamp) that emits LCD illumination light for LCD pixels, an optical waveguide layer for guiding the backlight, and an LCD structural layer that may include, for example, a Liquid Crystal (LC) cell layer, an LCD electrode, a transparent conductive ITO layer, an optical polarizer layer, a color filter layer, and a touch sensitive layer. The LCD module also includes a backlight diffuser positioned below the LCD structural layer and above the optical waveguide layer to spatially diffuse the backlight to illuminate the LCD display pixels, and an optically reflective film layer positioned below the optical waveguide layer to recycle the backlight light source to the LCD structural layer to improve light use efficiency and display brightness.

Referring to fig. 2B, the optical sensor module in this example is located under the LCD display panel to capture the returned light from the top touch-sensitive surface and to acquire a high resolution image of the fingerprint pattern when the user's finger is in contact with the sensing area on the top surface. In other implementations, the disclosed underscreen optical sensor module for fingerprint sensing may be implemented on a device without touch sensing features. In addition, suitable display panels may have various screen designs other than OLED displays.

Fig. 3A and 3B show examples of devices implementing the optical sensor modules in fig. 2A and 2B. Fig. 3A shows a cross-sectional view of a portion of a device containing an off-screen optical sensor module. Fig. 3B shows on the left side a view of the front side of a device with a touch-sensitive display, representing a fingerprint sensing area on the lower part of the display screen, and on the right side a perspective view of a portion of the device containing an optical sensor module located underneath the display screen assembly of the device. Fig. 3B also shows an example of a layout of a flexible tape with circuit elements.

In the design examples of fig. 2A, 2B, 3A and 3B, the optical fingerprint sensor design is different from some other fingerprint sensor designs that use a fingerprint sensor structure that is independent of the display screen and that has a physical demarcation between the display screen and the fingerprint sensor on the surface of the mobile device (e.g., a button-like structure in the opening of the top glass cover plate in some mobile phone designs). In the design shown herein, the optical fingerprint sensor for detecting fingerprint sensing and other optical signals is located below the top cover glass or layer (e.g., fig. 3A) such that the top surface of the cover glass serves as the top surface of the mobile device as a continuous and uniform glass surface across the vertically stacked and vertically overlapping display screen layers and optical detector sensors. Such a design for integrating optical fingerprint sensing and touch sensitive display screens under a common and uniform surface provides benefits including improved device integration, enhanced device packaging, enhanced resistance of the device to external elements, failures, wear and tear, and enhanced user experience during ownership of the device.

Referring back to fig. 2A and 2B, the illustrated off-screen optical sensor module for on-screen fingerprint sensing may be implemented in a variety of configurations.

In one implementation, a device based on the above design may be configured to include a device screen providing touch-sensitive operation and including a display panel structure having an LCD for forming a display image, a top transparent layer formed over the device screen as a user touch interface for the touch-sensitive operation and as an interface for transmitting light from the display structure to display the image to a user, and an optical sensor module under the display panel structure to receive light returned from the top transparent layer to detect a fingerprint.

The device disclosed in this document and other devices may be further configured to include various features.

For example, a device electronic control module may be included in the device to authorize user access to the device when the detected fingerprint matches the fingerprint of the authorized user. Furthermore, the optical sensor module is configured to detect, in addition to the fingerprint, a biometric parameter different from the fingerprint by optical sensing to indicate whether a touch associated with the detected fingerprint at the top transparent layer is from a living person, and if (1) the detected fingerprint matches the fingerprint of the authorized user, and (2) the detected biometric parameter indicates that the detected fingerprint is from a living person, the device electronic control module is configured to authorize access to the device by the user. The biometric parameter may include, for example, whether the finger contains a human blood flow or heartbeat.

For example, the device may include a device electronic control module coupled to the display panel structure to provide power to the light emitting display pixels and control the image display of the display panel structure, and in a fingerprint sensing operation, the device electronic control module operates to turn off the light emitting display pixels in one frame and turn on the light emitting display pixels in the next frame to allow the optical sensor array to capture two fingerprint images with and without light emitting display pixel illumination to reduce the effects of background light in fingerprint sensing.

For another example, the device electronic control module may be coupled to the display panel structure to provide power to the LCD display panel and turn off backlight power to the LCD display panel in the sleep mode, and the device electronic control module may be configured to wake up the display panel structure from the sleep mode when the optical sensor module detects the presence of human skin at a designated fingerprint sensing region of the top transparent layer. More specifically, in some implementations, the device electronic control module may be configured to operate one or more illumination sources in the optical sensor module to intermittently emit light while turning off power to the LCD display panel (in sleep mode), directing the intermittently emitted light to a designated fingerprint sensing area of the top transparent layer to monitor whether there is human skin in contact with the designated fingerprint sensing area for waking up the device from sleep mode.

For another example, the device may include a device electronic control module coupled to the optical sensor module to receive information of a plurality of detected fingerprints obtained by sensing a touch of a finger, and the device electronic control module is operative to measure a change in the plurality of detected fingerprints and determine a touch force that causes the measured change. For example, the change may include a change in the fingerprint image due to a touch force, a change in the touch area due to a touch force, or a change in the pitch of the fingerprint ridges.

For another example, the top transparent layer may include a designated fingerprint sensing area for a user to touch with a finger for fingerprint sensing, and the optical sensor module under the display panel structure may include a transparent block in contact with the display panel substrate to receive light emitted from the display panel structure and returned from the top transparent layer, and the optical sensor module may further include an optical sensor array to receive the light and an optical imaging module to image the light received in the transparent block onto the optical sensor array. The optical sensor module may be disposed relative to a designated fingerprint sensing region and configured to: the returned light is selectively received by total internal reflection at the top surface of the top transparent layer when in contact with human skin, and is not received from the designated fingerprint sensing area when in contact with no human skin.

As another example, an optical sensor module may be configured to include an optical wedge positioned below a display panel structure to modify a total reflection condition on a bottom surface of the display panel structure engaged with the optical wedge to allow light to be extracted from the display panel structure through the bottom surface, the optical sensor module may further include an optical sensor array to receive light extracted from the display panel structure from the optical wedge, and an optical imaging module positioned between the optical wedge and the optical sensor array to image light from the optical wedge onto the optical sensor array.

Specific examples of an off-screen optical sensor module for on-screen fingerprint sensing are provided below.

Fig. 4A and 4B illustrate an example of one implementation of an optical sensor module located below a display screen assembly for implementing the designs in fig. 2A and 2B. The device in fig. 4A-4B includes a display assembly 423 having a top transparent layer 431, the top transparent layer 431 formed over the display assembly 423 as a user touch interface for touch sensing operations and as an interface for transmitting light from a display structure to display an image to a user. In some implementations, the top transparent layer 431 may be a cover glass or a crystalline material. The display assembly 423 may include a display module 433 below the top transparent layer 431. The LCD display layer allows for partial optical transmission such that light from the top surface may partially pass through the LCD display layer to the LCD under-screen optical sensor module. For example, the LCD layer includes electrodes and wiring structures that optically function as an array of holes and light scattering objects. A device electronics module 435 may be disposed below the LCD display panel to control the operation of the device and perform functions for a user to operate the device.

The optical sensor module 702 in this particular implementation example is located below the display module 433. One or more illumination sources 703 are provided for the optical sensor module 702 and can be controlled to emit light at least partially through the display module 433 to illuminate an active sensing area 615 on the top transparent layer 431 within the device screen area for a user to place a finger therein for fingerprint identification. Illumination from one or more illumination sources 703 may be directed to an active sensing area 615 on the top surface as if such illumination came from the viewing zone 613. As shown, the finger 445 is placed in an illuminated effective sensing area 615, which effective sensing area 615 is the effective sensing area for fingerprint sensing. A portion of the reflected or scattered light in the active sensing area 615 illuminated by the LCD pixels in view area 613 is directed into the optical sensor module below display module 433 and a photodetector sensing array within the optical sensor module receives such light and captures fingerprint pattern information carried by the received light.

In such a design, one or more illumination sources 703 are used to provide illumination for optical fingerprint sensing, and in some implementations, each illumination source 703 may be controlled to be intermittently turned on at a lower period to reduce the optical power used for optical sensing operations. In some implementations, the fingerprint sensing operation may be implemented in a two-step process: first, one or more illumination light sources 703 are turned on in a flash mode without turning on the LCD display panel to sense whether a finger touches the effective sensing area 615 using a flash, and once a touch in the effective sensing area 615 is detected, the optical sensing module is operated to perform fingerprint sensing based on optical sensing, and the LCD display panel may be turned on.

In the example of fig. 4B, the off-screen optical sensor module includes a low index optically transparent block 701 coupled to the display panel, the low index optically transparent block 701 receiving return light from the top surface of the device assembly, the off-screen optical sensor module further including an optical sensor module 702 that performs optical imaging and imaging capture. Light from the illumination source 703, upon reaching the top surface of the cover, e.g., the top surface of the cover at the active sensing area 615 that is touched by a user's finger, is reflected or scattered back from the top surface of the cover. When the top surface of the cover plate in the effective sensing area 615 is in close contact with the fingerprint ridge, the light reflection under the fingerprint ridge differs from the light reflection under the fingerprint valley at another location where there is no skin or tissue of the finger, due to the presence of skin or tissue of the finger in contact at that location. This difference in light reflection conditions at the locations of the ridges and valleys in the area of the top surface of the cover sheet where the finger touches forms an image representing the image or spatial distribution of the ridges and valleys of the touched portion of the finger. The reflected light is directed back to the display module 433 and, after passing through the aperture of the display module 433, reaches the interface of the low index optically transparent block 701 of the optical sensor module. The refractive index of the low-index optically transparent block 701 is configured to be smaller than that of the LCD display panel so that the returned light can be extracted from the LCD display panel into the low-index optically transparent block 701. Once the returned light is received within the low index optically transparent block 701, such received light enters an optical imaging unit that is part of the optical sensor module 702 and is sensed by a photodetector sensor array or optical sensing array within the optical sensor module 702. The difference in light reflection between the fingerprint ridges and valleys causes a contrast in the fingerprint image. Shown in fig. 4B is a microcontroller or MCU 601 coupled to an optical sensor module 702 and other circuitry such as a smartphone host processor 705 on the host circuit board.

In this particular example, the optical path design is: light enters the top surface of the cover plate within the total reflection angle on the top surface between the substrate and the air interface and is most effectively collected by the imaging optics and imaging sensor array in the optical sensor module 702. In this design, the images of the fingerprint ridge/valley areas exhibit the greatest contrast. Such imaging systems may have undesirable optical distortions that can adversely affect fingerprint sensing. Thus, based on the optical distortion at the optical sensor array along the optical path of the return light, the acquired image may also be corrected by distortion correction during imaging reconstruction at the output signal of the optical sensor array in the optical sensor module 702. By scanning the test image pattern of one row of pixels at a time over the sensing area of the X-direction lines and the Y-direction lines, a distortion correction coefficient can be generated from the pattern captured at each photodetector pixel. Such a correction process may also use an image resulting from tuning individual pixels one at a time and scanning the entire image area of the photodetector array. Such correction factors need only be generated once after the sensor is assembled.

Background light from the environment (e.g., sunlight or room light) may enter the image sensor through the aperture in the display assembly 423 through the top surface of the LCD panel. Such background light may create a background baseline in the valuable image from the finger, and such background baseline is undesirable. Different methods may be used to reduce this baseline intensity. One example is to tune the illumination source 703 on and off at a frequency f and the image sensor acquires the received image at the same frequency by phase synchronizing the source drive pulses with the image sensor frame. In this operation, only one of the images of different phases contains light emitted from the light source. By subtracting the odd and even frames it is possible to obtain an image which is mostly composed of the light emitted by the modulated illumination source. Based on the design, each display scan frame generates a frame of the fingerprint signal. If two consecutive signal frames are subtracted by tuning the illumination on in one frame and tuning the illumination off in the other frame, the ambient background light effects can be minimized or largely eliminated. In some implementations, the fingerprint sensing frame rate may be half the display frame rate.

A portion of the light from the illumination source 703 may also pass through the top surface of the cover plate and into the finger tissue. This portion of the optical power is scattered around and a portion of this scattered light may eventually be collected by the imaging sensor array in the optical sensor module. The light intensity of the scattered light depends on the skin tone of the finger and the blood concentration in the finger tissue, and this information carried by the scattered light on the finger is useful for fingerprint sensing and can be detected as part of a fingerprint sensing operation. For example, by integrating the intensity of the region of the user's finger image, it may be observed that the increase/decrease in blood concentration depends on the phase of the user's heartbeat. Such features may be used to determine the heart rate of the user, whether the user's finger is a live finger, or a spoof device with a fake fingerprint pattern.

One or more illumination sources 703 in fig. 4B may be designed to emit light of different colors or wavelengths, and an optical sensor module may capture return light of different colors or wavelengths from a person's finger. By recording the respective measured intensities of the return light in different colors or wavelengths, information about the skin tone of the user can be determined. For example, when a user registers a finger for a fingerprint authentication operation, the optical fingerprint sensor also measures the intensities of scattered light from the colors a and B of the finger as intensities Ia and Ib. The ratio Ia/Ib may be recorded for comparison with later measurements of the fingerprint when the user's finger is placed on the sensing area. The method may help reject rogue devices that may not match the user's skin tone.

The one or more illumination sources 703 may be controlled by the same microcontroller or MCU 601 that is used to control the image sensor array in the optical sensor module 702. The one or more illumination sources 703 may be pulsed for short periods of time with a low duty cycle to intermittently emit light and provide pulsed light for image sensing. The image sensor array may be operated to monitor the light pattern with the same pulse duty cycle. If there is a human finger touching the active sensing area 615 on the screen, the image captured at the imaging sensing array in the optical sensor module 702 may be used to detect touch events. A microcontroller or MCU 601 connected to the image sensor array in the optical sensor module 702 may be operated to determine if the touch is a human finger touch. If a human finger touch event is determined, the microcontroller or MCU 601 can be operated to wake up the smartphone system, turn on the illumination light source 703 for performing optical fingerprint sensing, and acquire a complete fingerprint image using normal mode. The image sensor array in the optical sensor module 702 would send the acquired fingerprint image to the smartphone host processor 705, which smartphone host processor 705 may be operative to match the captured fingerprint image with a registered fingerprint database. If there is a match, the smart phone will unlock the phone and initiate normal operation. If the acquired image is not matched, the smart phone may feed back the authentication failure to the user. The user may try again, or enter a password.

In the example of fig. 4A and 4B, the under-screen optical sensor module optically images a fingerprint pattern of a touching finger in contact with the top surface of the display screen onto the photodetector sensing array using a low index optically transparent block 701 and an optical sensor module 702 having a photodetector sensing array. An optical imaging or detection axis 625 from the active sensing region 615 to the photodetector array in the optical sensor module 702 is shown in fig. 4B. The front end of the optical sensor module 702, in front of the low index optically transparent block 701 and the photodetector sensing array, forms a volumetric imaging module to achieve suitable imaging for optical fingerprint sensing. Due to the optical distortion in the imaging process, as explained above, distortion correction may be used to achieve the desired imaging operation.

In the optical sensing disclosed herein by the off-screen optical sensor module and other designs in fig. 4A-4B, the optical signal from the active sensing area 615 on the top transparent layer 431 to the off-screen optical sensor module includes different light components. Fig. 5A, 5B and 5C illustrate signal generation for light returned from the active sensing area 615 under different optical conditions to facilitate understanding of the operation of the off-screen optical sensor module.

Fig. 5A shows how illumination light from illumination source 703 propagates through display module 433 after passing through top transparent layer 431 and generates a different return light signal to the off-screen optical sensor module, including a light signal carrying fingerprint pattern information. For simplicity, two illumination rays 80 and 82 at two different locations are directed toward the top transparent layer 431 without undergoing total reflection at the interface of the top transparent layer 431. In particular, the illumination light rays 80, 82 are perpendicular or nearly perpendicular to the top transparent layer 431. The finger tissue 60 is in contact with the active sensing area 615 on the top transparent layer 431. As shown, illumination light 80, after passing through top transparent layer 431, reaches the finger ridge in contact with top transparent layer 431 to generate a light beam 183 in finger tissue and another light 185, 603 back toward display module 433. After passing through the top transparent layer 431, the illumination light 82 reaches the finger valleys located above the top transparent layer 431 to generate light 185 that returns from the interface of the top transparent layer 431 to the display module 433, a light beam 189 that enters the finger tissue, and a light beam 187 that is reflected by the finger valleys.

In the example of fig. 5A, it is assumed that the equivalent refractive index of the finger skin at 550nm is about 1.44, and the cover glass refractive index of the top transparent layer 431 is about 1.51. The finger ridge-cover glass interface reflects a portion of the light of illumination ray 80 as reflected light 185 to the underlying layer 524 under the display module 433. The reflectivity may be low in some LCD panels, for example, about 0.1%. Most of the illumination light 80 becomes a beam 183 that is transmitted into the finger tissue 60, and the finger tissue 60 causes scattering of the beam 183, producing scattered light 191 that returns to the display module 433 and the underlayer 524. The scattering of the transmitted beam 189 from the illumination ray 82 in the finger tissue also contributes to the returned scattered light 191.

The illumination light 82 at the finger skin valley locations 63 is reflected by the cover glass surface, for example, about 3.5% of the incident light energy (as light 185) to the underlayer 524, and the finger valley surface reflects about 3.3% (beam 187) of the incident light energy to the underlayer 524. The total reflectance was about 6.8%. Most of the light beam 189 is transmitted into the finger tissue 60. A portion of the optical power in the transmitted beam 189 in the finger tissue is scattered by the finger tissue, contributing to the scattered light 191 toward and into the underlayer 524.

Thus, the light reflections from the various interfaces or surfaces at the finger valleys and finger ridges of the touching finger are different, the reflectance differences carry fingerprint map information, and the reflectance differences can be measured to extract the fingerprint pattern of the portion that is in contact with the top transparent layer 431 and is illuminated by the OLED light.

Fig. 5B and 5C show the optical paths of two additional types of illumination light under different conditions and at the top surface at different locations relative to the valleys or ridges of the finger, including the total reflection condition at the interface with the top transparent layer 431. The illustrated illumination light produces different return light signals, including light signals that carry fingerprint pattern information to the off-screen optical sensor module. Assuming that the top transparent layer 431 and the display module 433 are bonded together without any air gap therebetween, illumination light having a large angle of incidence with the top transparent layer 431 will be totally reflected at the cover glass-air interface. Fig. 5A, 5B, and 5C illustrate examples of three sets of diverging beams: (1) illumination light ray 82, having a small angle of incidence to top transparent layer 431 and no total reflection (fig. 5A), (2) high contrast light beams 201, 202, 211, and 212, are totally reflected at top transparent layer 431 when the cover glass surface is not touched, and can be coupled into finger tissue when a finger touches top transparent layer 431 (fig. 5B and 5C), and (3) escape light beams having a large angle of incidence are totally reflected at top transparent layer 431 even at the location of finger tissue touches.

For illumination light 82, the cover glass surface reflects approximately 0.1% to 3.5% of the light against light 185, which is transmitted into bottom layer 524, and the finger skin reflects approximately 0.1% to 3.3% of the light against light beam 187, which is also transmitted into bottom layer 524. The difference in reflection depends on whether the illumination light 82 meets the finger skin ridge 61 or the skin valley 63. The remaining light beam 189 is coupled into the finger tissue 60.

For high contrast beams 201 and 202, if the cover glass surface is not touched, the cover glass surface reflects nearly 100% of the light to beams 205 and 206, respectively. When the finger skin ridge touches the cover glass surface at the location of high contrast beams 201 and 202, a large portion of the optical power is coupled into finger tissue 60 through beams 203 and 204.

For high contrast illumination beams 211 and 212, if the cover glass surface is not touched, the cover glass surface reflects nearly 100% of the light to beams 213 and 214, respectively. When a finger touches the cover glass surface and the finger skin valleys are just in the location of illumination beams 211 and 212, no optical power is coupled into finger tissue 60.

As shown in fig. 5A, the light beam coupled into the finger tissue 60 will be randomly scattered via the finger tissue to form scattered light 191, and a portion of such scattered light 191 will pass through the display module 433 to the optical sensor module.

Thus, in the area irradiated by the high contrast beam, the finger skin ridges and valleys cause different optical reflections, and the reflection difference pattern carries fingerprint pattern information. A high contrast fingerprint signal may be achieved by comparing such differences.

Based on the designs in fig. 2A and 2B, the disclosed underscreen optical sensing techniques may optically capture fingerprints in various configurations.

For example, the particular implementation in FIG. 4B may be implemented in various configurations based on optical imaging through the use of a bulk imaging module in an optical sensing module. Fig. 6A-6C, 7, 8A-8B, 9, 10A-10B, 11, and 12 illustrate examples of various implementations, additional features, and operations of an off-screen optical sensor module design for optical fingerprint sensing.

Fig. 6A, 6B and 6C show examples of an on-screen optical sensor module based on optical imaging through a lens for capturing a fingerprint from a finger 445 pressing against a display top transparent layer 431. Fig. 6C is an enlarged view of the portion of the optical sensor module shown in fig. 6B. An off-screen optical sensor module as shown in fig. 6B, which includes an optically transparent spacer 604 engaged with the bottom surface of the display module 433 to receive light returned from the active sensing area 615 on the top surface of the top transparent layer 431, and a microlens 651 between the optically transparent spacer 604 and the photodiode array 623, the microlens 651 imaging the received returned light from the active sensing area 615 onto the photodiode array 623. Similar to the imaging system in the example of fig. 4B, the imaging system for the optical sensor module in fig. 6B may experience image distortion, and appropriate optical correction calibration may be used to reduce such distortion, for example, the distortion correction method described for the system in fig. 4B.

Similar to the assumptions in fig. 5A, 5B, and 5C, it is assumed that the equivalent refractive index of the finger skin at 550nm is about 1.44, and that the refractive index of the bare cover glass is about 1.51 for the top transparent layer 431. When the display module 433 is bonded to the top transparent layer 431 without any air gaps, total internal reflection occurs at a large angle equal to or greater than the critical angle of incidence of the interface. If the cover glass top surface is not contacted, the angle of incidence of total reflection is about 41.8 °, and if the finger skin touches the cover glass top surface, the angle of total reflection is about 73.7 °. The corresponding total reflection angle difference is about 31.9 °.

In this design, the micro-lenses 651 and photodiode array 623 define a viewing angle θ that captures an image of the contact finger in the effective sensing region 615. To detect a desired portion of the cover glass surface in the effective sensing region 615, the viewing angle can be properly aligned by controlling a physical parameter or configuration. For example, the viewing angle may be aligned to detect total internal reflection of the LCD display assembly. Specifically, the viewing angle θ is aligned to sense the effective sensing area 615 on the cover glass surface. The active sensing area 615 may be considered a mirror such that the photodetector array effectively detects the field of view 613 in the LCD display and the image projected onto the photodetector array by the active sensing area 615. The photodiode/photodiode array 623 can receive an image of the field 613 reflected by the active sensing region 615. When a finger touches the active sensing area 615, a portion of the light may couple into the ridges of the fingerprint, which may cause the photodiode array to receive light from the ridge locations to appear as a darker image of the fingerprint. Since the geometry of the optical detection path is known, fingerprint image distortions caused in the optical path in the optical sensor module can be corrected.

As a specific example, consider that the distance H from the detection module center axis to the cover glass top surface in fig. 6B is 2mm. This design can directly cover an effective sensing area 615 of 5mm, the effective sensing area 615 having a width Wc on the cover glass. Adjusting the thickness of optically transparent spacer 604 can adjust detector position parameter H and can optimize effective sensing area width Wc. Since H includes the thickness of the top transparent layer 431 and the display module 433, the application design should consider these layers. Optically transparent spacer 604, microlenses 651, and photodiode array 623 may be integrated beneath color coating 619 on the bottom surface of top transparent layer 431.

Fig. 7 illustrates an example of another design consideration for the optical imaging design of the optical sensor module shown in fig. 6A-6C by using special shims 618 instead of optically transparent shims 604 in fig. 6B-6C to increase the size of the effective sensing region 615. The spacer 618 is designed to have a width Ws, a thickness Hs, a low Refractive Index (RI) ns, and the spacer 618 is located under the display module 433, for example, attached (e.g., adhered) to the bottom surface of the display module 433. The end surfaces of the pads 618 are angled or sloped surfaces that engage with the microlenses 651. This relative position of the spacer and lens is different from the lens being located below the optically transparent spacer 604 in fig. 6B-6C. The microlens 651 and photodiode array 623 are assembled into an optical detection module having a detection angle size θ. The detection axis 625 is curved due to optical refraction at the interface between the gasket 618 and the display module 433 and optical refraction at the interface between the top transparent layer 431 and air. The local angles of incidence φ 1 and φ 2 are determined by the refractive indices (refractive indices, RIs), ns, nc and na of the component materials.

If nc is greater than ns, then φ 1 is greater than φ 2. Thereby, the refraction amplifies the sensing width Wc. For example, assuming that the equivalent refractive index RI of the finger skin is about 1.44 at 550nm and the refractive index RI of the cover glass is about 1.51, the total reflection incident angle is estimated to be about 41.8 ° if the cover glass top surface is not touched and the total reflection angle is about 73.7 ° if the finger skin touches the cover glass top surface. The corresponding total reflection angle difference is about 31.9 °. If the spacer 618 is made of the same material as the cover glass, the distance from the center of the detection module to the top surface of the cover glass is 2mm, and if the detection angle is θ=31.9°, the effective sensing area width Wc is about 5mm. The corresponding local angle of incidence of the central axis is Φ1=Φ2=57.75°. If the material of the spacer 618 has a refractive index ns of about 1.4 and Hs is 1.2mm, the detection module is tilted at Φ1=70°. The effective sensing area width is increased to greater than 6.5mm. Under these parameters, the detection angle width in the cover glass is reduced to 19 °. Thus, the imaging system of the optical sensor module may be designed to desirably enlarge the size of the effective sensing area 615 on the top transparent layer 431.

When the refractive index of the spacer 618 is designed to be sufficiently low (e.g., such that By MgF ₂ 、CaF ₂ Or even air to form a spacer), the width Wc of the active sensing region 615 is no longer limited by the thickness of the top transparent layer 431 and the display module 433. This property provides the desired design flexibility. In principle, if the detection module has sufficient resolution, it is even possible to increase the effective sensing area to cover the whole display screen.

Because the disclosed optical sensor technology can be used to provide a large sensing area to capture patterns, the disclosed under-screen optical sensor module can be used not only to capture and detect patterns of fingers, but also to capture and detect patterns of larger size, such as the palm of a person associated with a person, for user authentication.

Fig. 8A-8B show examples of another design for the optical imaging design of the optical sensor module shown in fig. 7, in which the opposite detection angle θ' of the photodiode array in the display screen surface and the distance L between the microlens 651 and the spacer 618 are provided. Fig. 8A shows a cross-sectional view along a direction perpendicular to the surface of the display screen, and fig. 8B shows a view of the device as seen from the bottom or top of the display screen. Fill material 618c may be used to fill the space between microlenses 651 and photodiode array 623. For example, the filler material 618c may be the same material as the gasket 618 or a different material. In some designs, the filler material 618c may be an air gap.

Fig. 9 shows another example of an off-screen optical sensor module based on the design of fig. 7, wherein one or more illumination sources 614 are arranged to illuminate an active sensing area 615 of the top surface for optical fingerprint sensing. The illumination source 614 may be an extended type or a collimated type source such that all points within the effective sensing area 615 are illuminated. The illumination source 614 may be a single element light source or an array of light sources.

Fig. 10A-10B illustrate examples of an off-screen optical sensor module using optical couplers 628 in the shape of thin wedges to improve optical detection at the photodiode array 623. Fig. 10A shows a cross section of a device structure with an off-screen optical sensor module for fingerprint sensing, and fig. 10B shows a top view of the device screen. An optical coupler 628 (having a refractive index ns) is positioned below the display panel structure to modify the total reflection condition on the bottom surface of the display panel structure to which the optical coupler 628 is engaged to allow light to be extracted from the display panel structure through the bottom surface. The photodiode array 623 receives light extracted from the display panel structure from the optical coupler 628, and an optical imaging module 621 is located between the optical coupler 628 and the photodiode array 623 to image the light from the optical coupler 628 onto the photodiode array 623. In the example shown, optical coupler 628 includes a sloped optical wedge surface facing the optical imaging module and photodiode array 623. Also, as shown, there is a free space between the optical coupler 628 and the optical imaging module 621.

The reflectivity is 100% with the highest efficiency if the light is totally reflected at the sensing surface of the top transparent layer 431. However, if the light is parallel to the cover glass surface, the light is also totally reflected at the bottom of the OLED display module. The optical coupler 628 is used to modify the local surface angle so that light can be coupled out for detection at the photodiode array 623. The micro-holes in the display module 433 provide the desired light propagation path for light to pass through the display module 433 for off-screen optical sensing. The actual light transmission efficiency may gradually decrease if the light transmission angle becomes too large or when the TFT layer becomes too thick. When the angle is close to the total reflection angle, i.e. about 41.8 deg., and the cover glass refractive index is 1.5, the fingerprint image appears to be good. Accordingly, the wedge angle of the optical coupler 628 may be adjusted to several degrees so that the detection efficiency may be improved or optimized. If a higher refractive index of the cover glass is selected, the total reflection angle becomes smaller. For example, if the cover glass is made of sapphire having a refractive index of about 1.76, the total reflection angle is about 34.62 °. The detected light transmission efficiency in the display is also improved. Therefore, this design uses thin wedges to set the detection angle higher than the total reflection angle, and/or uses a cover glass material with a high refractive index to improve detection efficiency.

In the underscreen optical sensor module in fig. 6A-10B, the effective sensing area 615 on the top transparent surface is not vertical or perpendicular to the detection axis 625 of the optical sensor module, such that the image plane of the sensing area is also not vertical or perpendicular to the detection axis 625. Thus, the plane of the photodiode array 623 may be tilted with respect to the detection axis 625 to achieve high quality imaging at the photodiode array 623.

Fig. 11 shows three example configurations of such tilting. Fig. 11 (1) shows that the effective sensing region 615a is tilted and not perpendicular to the detection axis 625. In the particular case shown in (2), the active sensing area 615b is aligned on the detection axis 625, and its image plane will also lie on the detection axis 625. In practice, the microlenses 651 can be partially cut away to simplify packaging. In various implementations, microlenses 651 can also be transmissive or reflective lenses. For example, a specific route is shown in (3). Effective sensing region 615c is imaged by micro imaging mirror 651 a. The photodiode array 623b is aligned to detect a signal.

In the above designs using microlenses 651, the effective aperture of the microlenses 651 can be designed to be larger than the aperture of the holes in the LCD display layer, which allows light to be transmitted through the LCD display module for optical fingerprint sensing. Such a design may reduce the undesirable effects of wiring structures and other scattering objects in the LCD display module.

Fig. 12 illustrates an example of an operation of a fingerprint sensor for reducing or eliminating effects from background light in fingerprint sensing. The optical sensor array may be used to capture various frames and the captured frames may be used to perform differential and average operations between frames to reduce the effects of background light. For example, in frame a, the illumination source for optical fingerprint sensing is turned on to illuminate the area touched by the finger, and in frame B, the illumination is changed or turned off. Subtracting the signal of frame B from the signal of frame a may be used in image processing to reduce the undesirable effects of background light.

Undesired background light in fingerprint sensing may also be reduced by providing suitable optical filtering in the optical path. One or more optical filters may be used to filter out ambient light wavelengths, such as near infrared light and some red light, etc. In some implementations, such optical filter coatings may be fabricated on surfaces of optical components, including display floors, prismatic surfaces, or sensor surfaces, among others. For example, if a human finger absorbs a substantial portion of the energy of light having a wavelength below 580nm, if one or more optical filters or optical filter coatings can be designed to filter light having wavelengths from 580nm to infrared, the effect of ambient light on optical detection in fingerprint sensing can be greatly reduced.

Fig. 13 shows an example of an operation procedure for correcting image distortion in the optical sensor module. At step 1301, one or more illumination sources are controlled and operated to emit light in a particular region, and the light emission of such pixels is modulated by frequency F. At step 1302, the frame rate of the imaging sensor under the display panel is operated to capture images at the same frequency F and a certain frame rate. In an optical fingerprint sensing operation, a finger is placed on top of a display panel cover substrate and the presence of the finger modulates the intensity of light reflection from the top surface of the display panel cover substrate. An imaging sensor under the display captures the fingerprint modulated reflected light pattern. At step 1303, demodulation of the signal from the image sensor is synchronized at frequency F, and background filtering is performed. The resulting image reduces the effects of background light and includes images produced by light emitted from the pixels. At step 1304, the captured image is processed and calibrated to correct image system distortion. At step 1305, the corrected image is used as a human fingerprint image for user authentication.

The same optical sensor used to capture the user's fingerprint may also be used to capture scattered light from the illuminated finger, as shown by scattered light 191 scattered into the substrate in fig. 5A. The detector signals from the region of interest of the scattered light 191 in fig. 5A scattered into the bottom layer may be integrated to produce an intensity signal. The intensity variation of the intensity signal is evaluated to determine the heart rate of the user.

The fingerprint sensor may be hacked by a malicious individual who is able to obtain the fingerprint of an authorized user and copy the stolen fingerprint pattern on a carrier similar to a human finger. Such an unauthorized fingerprint pattern may be used on a fingerprint sensor to unlock a target device. Thus, the fingerprint pattern, although a unique biometric identifier, may not itself be a completely reliable or secure identification. The off-screen optical sensor module may also be used as an anti-spoof sensor for sensing whether an input object having a fingerprint pattern is a finger from a living person and for determining whether the fingerprint input is a fingerprint spoof attack. No separate optical sensor is required to provide such anti-spoofing sensing function. The anti-spoofing can provide a high-speed response without compromising the overall response speed of the fingerprint sensing operation.

Fig. 14 shows an exemplary extinction coefficient of a material monitored in blood, the optical absorption in blood being different between the visible spectrum range of red light, e.g. 660nm, and the infrared range of infrared light, e.g. 940 nm. By using probe light to illuminate the finger at a second, different wavelength, such as a first visible wavelength (color a) and an infrared wavelength (color B), a difference in optical absorption of the input object can be captured to determine if the touching object is a finger from a living person. The one or more illumination sources for providing illumination for optical sensing may be used to emit different colors of light to emit detection or illumination light of at least two different optical wavelengths for in vivo finger detection using different optical absorption behaviors of blood. When a human heart beats, pulse pressure pumps blood to flow in an artery, so the extinction ratio of a material monitored in the blood varies with the pulse. The received signal carries a pulse signal. These properties of blood can be used to detect whether the material being monitored is a live or fake fingerprint.

Fig. 15 shows a comparison between optical signal behavior in reflected light from inanimate material (e.g., a fake finger) and a live finger. The optical fingerprint sensor may also be used as a heartbeat sensor to monitor living organisms. When two or more wavelengths of probe light are detected, the difference in extinction ratio can be used to quickly determine whether the material being monitored is a living organism, such as a living fingerprint. In the example shown in fig. 15, probe light of different wavelengths is used, one being visible wavelength and the other being infrared wavelength as shown in fig. 14.

When the inanimate material touches the top cover glass over the fingerprint sensor module, the received signal reveals an intensity level associated with the surface pattern of the inanimate material and the received signal does not contain a signal component associated with a live human finger. However, when a live finger touches the top cover glass, the received signal reveals signal characteristics associated with the live, which include significantly different intensity levels due to the different extinction ratios of the different wavelengths. This method does not take a long time to determine whether the touch material is part of a living person. In fig. 15, the pulse-like signal reflects the case of multiple touches, not the blood pulse. Similar multiple touches of inanimate material do not show differences caused by a living finger.

Such optical sensing of different optical absorption behavior of blood at different optical wavelengths can be performed in a short period for living finger detection and can be faster than optical detection of the heartbeat of a person using the same optical sensor.

In an LCD display, the LCD backlighting light is white light, thus comprising light in the visible and IR spectral ranges for performing the above mentioned living finger detection at the optical sensor module. An LCD color filter in the LCD display module may be used to allow the optical sensor module to obtain the measurements in fig. 14 and 15. In addition, the detection light source 436 for generating illumination light for optical sensing may be operated to emit detection light at selected visible and IR wavelengths at different times, and the photodiode array 623 captures reflected detection light of two different wavelengths to determine whether the touched object is a living finger based on the above-described operations shown in fig. 14 and 15. Notably, although the detection light reflected at the selected visible and IR wavelengths at different times may reflect different light absorption characteristics of the blood, the fingerprint image is always captured by the detection light of the selected visible and IR wavelengths at different times. Thus, fingerprint sensing can be performed at both visible and infrared wavelengths.

Fig. 16 shows an example of an operation procedure of determining whether an object in contact with an LCD display screen is a part of a living finger by operating one or more illumination light sources for optical sensing to illuminate the finger with two different colors of light.

As another example, the disclosed optical sensor technology may be used to detect whether a captured or detected pattern of a fingerprint or palm is from a live human hand by a "live finger" detection mechanism by other mechanisms besides the different optical absorption of blood at different optical wavelengths described above. For example, a living human finger is typically moving or stretching due to the natural movement or motion (intentional or unintentional) of the person, or is typically pulsed as blood flows through the body in connection with a heartbeat. In one implementation, the optical sensor module may detect a change in returned light from the finger or palm due to a change in heartbeat/blood flow, thereby detecting whether a living heartbeat is present in an object that appears as a finger or palm. User authentication may enhance access control based on a combination of optical sensing of fingerprint/palm patterns and positive determination of the presence of living persons. As another example, when a person touches an LCD display screen, the change in touch force may be reflected in one or more ways, including deformation of the fingerprint pattern, a change in the contact area between the finger and the screen surface, widening of the fingerprint ridge, or dynamic changes in blood flow. These or other changes may be measured by optical sensing based on the disclosed optical sensor technology and may be used to calculate touch force. Such touch force sensing may be used to add further functionality to the optical sensor module than fingerprint sensing.

In the above examples, as shown in fig. 4B and 6B, fingerprint patterns are captured on the optical sensor array via the imaging module, and optical distortion typically reduces image sensing fidelity. Such image distortion may be corrected in various ways. For example, a known pattern may be used to generate an optical image at the optical sensor array, and image coordinates in the known pattern may be correlated with the optical image produced by distortion at the optical sensor array to calibrate the imaging sensor signals output by the optical sensor array for fingerprint sensing. The fingerprint sensing module calibrates the output coordinates with reference to the image of the standard pattern.

Various implementations of the disclosed optical sensor modules may be made in accordance with the disclosure in this patent document.

For example, the display panel may be configured to: each of which may be individually controlled; the display panel includes an at least partially transparent substrate and a substantially transparent cover substrate. The optical sensor module is positioned under the display panel to sense an image formed on top of the surface of the display panel. The optical sensor module may be used to sense an image formed by light emitted from the display panel pixels. The optical sensor module may include a transparent block having a refractive index lower than that of the display panel substrate, an imaging sensor block having an imaging sensor array, and an optical imaging lens. In some implementations, the low refractive index block has a refractive index in the range of 1.35 to 1.46 or 1 to 1.35.

As another example, a method for fingerprint sensing may be provided in which light emitted from a display panel is reflected by a cover substrate, and a finger located on top of the cover substrate interacts with the light to modulate a light reflection pattern by a fingerprint. The imaging sensing module under the display panel is used for sensing the reflected light pattern image and reconstructing a fingerprint image. In one implementation, the emitted light from the display panel is modulated in the time domain, and the imaging sensor is synchronized with the modulation of the emitted pixels, the demodulation process based on such a setting rejects most of the background light (not the light from the target pixel).

Various designs for the disclosed underscreen optical sensor module for optical fingerprint sensing consider the priority of U.S. provisional patent application serial No. 62/181,718, filed on US patent application serial No. 62/289,328, and U.S. provisional application serial No. 62/330,833, the invention of annex 3 entitled "multifunctional fingerprint sensor and package" (page 41 text and page 26 drawing), and the international patent application entitled "multifunctional fingerprint sensor with optical sensing capability" filed on USPTO, month 6, month 20, and filed on USPTO, month 2016 (priority of U.S. provisional patent application serial No. 62/181,718, filed on month 2015, month 18, and filed on month 12, month 22, and published on 2016/205832), and the international patent application entitled "multifunctional fingerprint sensor with optical sensing protection against fingerprint spoofing", filed on SIPO, month 11, date (priority of U.S. patent application serial No. 62/249,2017, and published on month 11, and published on year 2016/6292). The entire disclosure of the above-mentioned patent application is incorporated by reference as part of the disclosure of this patent document.

In various implementations of the herein disclosed underscreen optical sensor module technology for fingerprint sensing, optical imaging of illuminated touch portions of a finger into an optical sensor array in an underscreen optical sensor module may be accomplished without using an imaging module such as a lens that images light returned from the touch portions of the finger under optical illumination. One technical challenge of optical fingerprint sensing without an imaging module is how to control the propagation of returned light, which may spatially disturb the returned light at the optical sensor array from different locations on the touch portion of the finger, such that when such returned light reaches the optical sensor array, spatial information at the different locations may be lost. This challenge can be addressed by detecting the fingerprint by optical sensing using an optical collimator or pinhole array instead of an optical imaging module in an off-screen optical sensor module. An apparatus for implementing such optical fingerprint transmission may include: a device screen providing touch-sensitive operation and comprising a display panel structure having light-emitting display pixels, each pixel operable to emit light to form a portion of a display image; a top transparent layer formed over the device screen as a user touch interface for touch sensing operations and an interface for transmitting light from the display structure to display an image to a user; and an optical sensor module located below the display panel structure to receive light emitted by at least a portion of the light emitting display pixels of the display structure and returned from the top transparent layer to detect a fingerprint, the optical sensor module comprising an optical sensor array that receives the returned light, the optical sensor module further comprising an optical collimator array or pinhole array located in a path of the returned light to the optical sensor array. The optical collimator array is used to collect the returned light from the display panel structure and separate the light from different locations in the top transparent layer while directing the collected returned light to the optical sensor array.

Imaging using collimators relies on the use of different collimators at different locations to spatially separate light from different regions of the fingerprint to different optical detectors in an array of optical detectors. Each collimator may be designed along its thickness or length to control a narrow field of the optical view of each collimator, e.g. only light from a small area on an illuminated finger is captured by each collimator and projected onto several adjacent optical detectors in an array of optical detectors. For example, each collimator may be designed to be large, such as a few hundred microns, along the thickness or length of the collimator, such that the field of the optical view of each collimator may allow the collimator to transmit imaging light to a small area on the optical detector array, such as one optical detector or a few adjacent optical detectors in the optical detector array (e.g., in some cases, an area of tens of microns on each side of the optical detector array).

The following section explains by way of example how an optical collimator array or pinhole array is used for off-screen optical fingerprint sensing, using optical collimators in hybrid sensing pixels each having a capacitive sensor for capturing fingerprint information and an optical sensor for capturing fingerprint information when optical fingerprint sensing.

Fig. 17A and 17B show two examples of a hybrid sensor pixel design that combines capacitive sensing and optical sensing in the same sensor pixel.

Fig. 17A shows an example of a fingerprint sensor device 2100 that incorporates a capacitive sensor on an optical sensor basis for each sensor pixel of an array of sensor pixels when capturing fingerprint information. By combining a capacitive sensor and an optical sensor, a fingerprint image obtained using the optical sensor may be used to better resolve a 3D fingerprint structure obtained using the capacitive sensor. For illustration purposes, the structure shown in fig. 17A represents one sensor pixel in the sensor pixel array, and each sensor pixel includes an optical sensor 2109 and a capacitive sensor 2114 arranged adjacent to each other within the same pixel.

The optical sensor 2109 includes a photodetector 2108 and a collimator 2106 positioned over the photodetector 2108 to narrow or focus light 2124 reflected from the finger 2102 toward the photodetector 2108. One or more light sources (not shown), such as LEDs, may be placed around the collimator 2106 to emit light that is reflected by the finger as light 2124 and is directed toward the corresponding photodetector 2108 or focused toward the corresponding photodetector 2108 to capture a portion of the fingerprint image of the finger 2102. Collimator 2106 may be implemented using a fiber bundle or one or more metal layers with holes or openings. Such use of multiple optical collimators over an optical detector array may be used as a lensless optical design to capture fingerprint patterns with a desired spatial resolution for reliable optical fingerprint sensing. Fig. 17A shows a collimator 2106 implemented using one or more metal layers 2110 having holes or openings 2112. The collimator 2106 in the layer between the top structure or cover plate 2104 and the photodetector 2108 in fig. 17A includes a plurality of individual optical collimators formed by optical fibers or holes or openings in one or more layers (e.g., silicon or metal), each of which receives light 2124 along the longitudinal direction or within a small angular range of each optical collimator, as shown, which light 2124 can be captured by the top opening of each opening or hole and the tubular structure such that light incident at a large angle from the longitudinal direction of each optical collimator is rejected by each collimator to an optical photodiode on the other end of the optical collimator.

In the capacitive sensing portion of each sensing pixel, capacitive sensor 2114 includes a capacitive sensor plate 2116, which capacitive sensor plate 2116 electromagnetically couples to a portion of the finger in proximity to or in contact with the sensing pixel for capacitive sensing. More specifically, capacitive sensor plate 2116 and finger 2102 interact as two plates of one or more capacitive elements 2122 when finger 2102 is in contact with or in close proximity to an optional cover 2104 or a cover on a mobile device implementing finger sensor device 2100. The number of capacitive sensor plates 2116 may vary based on the design of the capacitive sensor 2114. Capacitive sensor plate 2116 can be implemented using one or more metal layers. Capacitive sensor plate 2116 is communicatively coupled to capacitive sensor circuit 2120 such that capacitive sensor circuit 2120 can process signals from capacitive sensor plate 2116 to obtain signals representative of a 3D fingerprint structure. Routing or shielding material may be provided between the capacitive sensor plate 2116 and the capacitive sensor circuit to the capacitive sensor plate 2116. The capacitive sensor circuit 2120 may be communicatively coupled to the capacitive sensor plate 2116 and the photodetector 2108 to process signals from the capacitive sensor plate 2116 and signals from the photodetector 2108. In fig. 17A, the capacitive sensor and the optical sensor within each hybrid sensor pixel are adjacent to each other and are displaced from each other without overlapping spatially.

In an implementation, optical sensing features such as the optical collimator design in the hybrid sensor design of fig. 17A may be used in an off-screen optical sensor module. Thus, the optical sensing with the optical collimator feature in fig. 17A may be implemented in a mobile device or electronic device capable of detecting a fingerprint by optical sensing, the mobile device or electronic device comprising: a display screen structure; a top transparent layer formed over the device screen structure as a user touch interface and as an interface for transmitting light from the display screen structure to display an image to a user; and an optical sensor module located below the display screen structure to receive light returned from the top transparent layer to detect a fingerprint. The optical sensor module includes an optical sensor array that receives the returned light and an optical collimator array to collect the returned light from the top transparent layer through the display screen structure and separate the light from different locations in the top transparent layer while directing the collected returned light through the optical collimator to the photodetectors in the optical sensor array.

Fig. 17B shows another example of a fingerprint sensor device 2130, the fingerprint sensor device 2130 structurally integrating optical sensors and capacitive sensors in each hybrid sensor pixel in a spatially overlapping configuration in an array of sensor pixels to reduce the space occupied by each hybrid sensor pixel. The fingerprint sensor device 2130 includes a semiconductor substrate 2131 of silicon or the like. A plurality of sensing elements or sensing pixels 2139 are provided on the semiconductor substrate 2131. Each sensing element or sensing pixel 2139 includes an active electronic circuit area 2132, the active electronic circuit area 2132 including CMOS switches, amplifiers, resistors, and capacitors to process the sensor signals. Each sensing element or sensing pixel 2139 includes a photodetector 2133 disposed or embedded in an active electronic circuitry area 2132. A capacitive sensor plate or sense top electrode 2134 for a capacitive sensing capacitive sensor is disposed over the photodetector 2133 and includes an aperture or opening 2138 in the sense top electrode 2134 and also acts as a collimator to direct light onto the photodetector 2133. A via 2135 filled with a conductive material is provided to electrically connect the sense top electrode 2134 to the active circuitry area 2132. By adjusting the opening or aperture and sensing the distance of the top electrode 2134 from the photodetector 2133, the light collection angle 2137 of the photodetector (e.g., photodiode) 2133 can be adjusted. The fingerprint sensor device 2130 is covered by a protective cover 2136, which protective cover 2136 comprises a hard material, such as sapphire, glass, etc. The light collection angle 2137 of the photodetector 2133 may be designed to preserve the spatial resolution of the image collected by the photodiode array. A light source 2140, such as an LED, is located below the cover plate to illuminate on the side of the fingerprint sensor device 2130, which light is reflected by the finger and directed to the photodetector 2133 to capture an image of the fingerprint. When a finger touches or is in close proximity to the protective cover, the combination of the finger and the sense top electrode 2134 forms a capacitive coupling (e.g., capacitor 2142) between the human body and the sense top electrode 2134. The fingerprint sensor device 2130 including the optical sensor and the capacitive sensor can obtain both of the light reflection image and the capacitive coupling image of the fingerprint. The sense top electrode 2134 serves a dual purpose: 1) For capacitive sensing, and 2) act as a collimator (by making one or more holes in the sensing top electrode 2134) to direct light emitted from the finger toward the photodetector 2133 or to narrow or focus toward the photodetector 2133. The reuse of the sense top electrode 2134 eliminates the need for additional metal layers or fiber bundles, thereby reducing the size of each pixel and thus the overall size of the fingerprint sensor device 2130.

In fig. 17B, the optical sensing design uses an aperture or opening 2138 formed between a protective cover plate 2136 and a photodetector 2133 as an optical collimator to select only light rays within certain light collection angles 2137 to maintain the spatial resolution of the image collected by the photodetector 2133 in the photodetector array as shown. Similar to the fiber or other tubular optical collimator in fig. 17A, the aperture or opening 2138 formed between the protective cover 2136 and the bottom array of photodetectors 2133 constitutes an optical collimator that collects the returned light from the top transparent layer via the display screen structure and separates the light from different locations in the top transparent layer while directing the collected returned light through the optical collimator to the photodetectors 2133.

Fig. 18 is a top view of an exemplary hybrid fingerprint sensor device 2200 incorporating optical and capacitive sensors into each hybrid sensing pixel. The fingerprint sensor device 2200 is implemented as a CMOS silicon chip 2221, which CMOS silicon chip 2221 includes a hybrid (incorporating optical and capacitive sensors) sensing element array 2222. Alternatively, the optical layout in fig. 18 may also be used for all optical sensing designs disclosed in this document, where the opening or aperture 2223 represents the optical collimator in fig. 17A and 17B. For example, the size or dimension of the sensing element may be in the range of 25 μm to 250 μm. The hybrid sensor device 2220 may include a support circuit array that includes amplifiers, analog-to-digital converters, and buffer memory in the side area 2224. Further, the hybrid sensor device 2200 may include regions 2225 for wire bonding or bump bonding. The top layer 2226 of the hybrid sense element array 2222 may include metal electrodes for capacitive sensing. One or more openings or holes 2223 may be made in each top metal electrode 23 to structurally act as a collimator to direct light in a vertical direction onto the photodetector below the top electrode. Thus, the top layer 2226 structure can serve the dual purpose of optical sensing and capacitive sensing. A sensor device processor may be provided to process pixel output signals from the hybrid sense pixels to extract fingerprint information.

In addition to sharing the same structure as a collimator for capacitive sensing and for focusing light in the vertical direction, one example of a sensor signal detection circuit may be shared between the optical sensor and the capacitive sensor to detect sensor signals from the photodetector and the capacitive sensor plate.

Fig. 19A illustrates an exemplary sensor pixel 2300 having capacitive sensing and optical sensing functionality for a fingerprint. The exemplary sensor pixel 2300 includes sensor signal detection circuitry 2316 to selectively switch between detecting or acquiring a sensor signal based on capacitive sensing from a sensing top electrode (e.g., top metal layer) 2308 and a sensing signal based on optical sensing from a photodetector (e.g., photodiode) 2314 to acquire a reflected optical image of a finger from the photodetector 2314 and a capacitively coupled image from the sensing top electrode 2308. In some implementations, two images from both sensing mechanisms in each hybrid sensing pixel may be processed serially by the sensor signal detection circuit. In the example shown, the switches 2310 and 2312 have: a first terminal electrically coupled to the sense top electrode 2308 and the photodetector 2314, respectively, and a second terminal coupled to a common input of the sensor signal detection circuit 2316 to provide a corresponding optical detector signal from the photodetector 2314 and a corresponding capacitive sense signal from the sense top electrode 2308 to the sensor signal detection circuit 2316. When the switch 2310 is open (cap_en=0) and the switch 2312 is closed (optical_en=1), the sensor signal detection circuit 2316 acquires an Optical detector signal representative of an Optical image of a scanned fingerprint received at a particular hybrid sensor pixel. When cap_en=1 and optical_en=0 of the switch 2310, the sensor signal detection circuit 2316 may acquire a capacitance sensing signal representing a capacitance image of a scanned fingerprint. After both the optical image and the capacitive image are acquired, the two images may be processed in downstream circuitry, either alone or in combination, to identify the fingerprint feature.

With the two imaging modalities of hybrid sensor pixels described above, fingerprint recognition performance can be enhanced by utilizing the two types of images in different ways. Such enhanced fingerprint recognition may be implemented by a sensor device processor such as sensor device processor 2321 for processing pixel output signals from the hybrid sensor pixels to extract fingerprint information. For example, the capacitive image may provide a 3D image of the depth of the ridges and valleys of the fingerprint feature. As a complement to the 3D capacitive image, the optical image may provide high resolution 2D information about the fingerprint features. Because both image information relate to ridges of the same fingerprint, an optical 2D image with higher spatial resolution can be used to restore the capacitive sensing image resolution. In some implementations, the capacitive sensing method may identify the valleys of the fingerprint more sensitively and accurately than the optical sensing method, and the spatial resolution of an image acquired using the capacitive sensing method may be degraded based on the thickness of the cover plate. This aspect of capacitive sensing can be compensated for by optical sensing. In operation, the sensor response may be fixed and the point spread function of the capacitive sensor may be fixed for all sensor positions. Higher resolution optical sensing may be used as a resolution restoration method and may be applied on the capacitive sensing image to enhance the 3D image. Partial high resolution images from optical sensing may be used to aid the restoration method. Thus, the 3D capacitive image may be enhanced by interpolation or restoration based on the high resolution 2D image to provide more information about the valleys and ridges.

The enhanced 3D image may provide improved fingerprint identification and matching. In another example, the optical image and the capacitive image may be stored together to provide two comparisons each time a fingerprint identification or match is made. The use of two types of images for comparison enhances the accuracy and security of the fingerprint sensing system.

The sensor signal detection circuit 2316 may be implemented in a variety of ways using a variety of different circuit designs. In one example, the integration circuit 2318 may be implemented to store charge caused by touching of ridges or valleys or by a cover plate of a fingerprint sensor device very close to a cover plate of a mobile device. Inclusion of the integration circuit 2318 enhances signal-to-noise ratio (SNR). The integrator sensing circuit includes an amplifier 2322 to amplify a sensor signal, such as a capacitance-related signal or an optical-related signal (e.g., a voltage signal), detected by the sensing top electrode 2308 or the photodetector 2314 of the exemplary sensor pixel 2300. The sense top electrode 2308, which includes a conductive material from among a plurality of metals, is electrically connected to the negative or inverting input 2328 of the amplifier 2322 through a switch 2310. The sensing top electrode 2308 and the partial surface 202 of the finger act as opposing plates of the capacitor Cf 2302. The capacitance of capacitor Cf2302 varies based on the distance'd' between the local surface of the finger and sensing top electrode 2308, i.e., the distance between the two plates of capacitor Cf 2302. The capacitance of capacitor Cf2302 is inversely proportional to the distance'd' between the two plates of capacitor Cf 2302. The capacitance of capacitor Cf2302 when sensing top electrode 2308 is opposite the ridge of the finger is greater than the capacitance of capacitor Cf2302 when sensing top electrode 2308 is opposite the valley of the finger.

Further, in the exemplary sensor pixel 2300, various parasitic or other capacitors can be formed between different conductive elements. For example, a parasitic capacitor CP 2304 may be formed between the sense top electrode 2308 and the device ground 2305. The device ground is closely coupled to ground. Another capacitor Cr 2324 may be formed between the output conductor of the amplifier 2322 and the negative or inverting input 2328 of the amplifier 2322 and function as a feedback capacitance of the amplifier 2322. Also, a switch 2326 may be coupled between the output of the amplifier 2322 and the negative or inverting input 2328 of the amplifier 2322 to reset the integration circuit 2318.

The positive terminal of the amplifier 2322 is electrically connected to the excitation signal Vref 2330. In each sensor pixel, the excitation signal Verf 2330 may be provided directly to the positive terminal of the dedicated amplifier. By providing the excitation signal Verf 2330 directly to the positive terminal of amplifier 2322, exemplary sensor pixel 2300 becomes an active sensor pixel. In addition, providing the excitation signal Verf 2330 directly to the positive terminal of the amplifier 2322 eliminates the need to include an excitation electrode common to all sensor pixels, thus reducing the conductive (e.g., metal) layer from the semiconductor structure of the sensor chip. In some implementations, an optional excitation Tx electrode 2306 may be implemented to enhance SNR based on the design of the sensor pixel. In addition, by providing the excitation signal Verf 2330 directly to the amplifier 2322, the excitation signal Vref is not directly applied to the finger to avoid potential irritation or injury to the finger. Furthermore, when the excitation electrode directly applying the excitation signal to the finger is not used, all components of the fingerprint sensor device may be integrated into a single package device, and the entire fingerprint sensor device may be disposed under the protective cover glass. Since the entire fingerprint sensor device is arranged under the protective cover glass, the fingerprint sensor device is protected from fingers and other external elements that may damage the fingerprint sensor.

In fig. 19A, the output signals (optical output signals and capacitive output signals) of the sensor signal detection circuit 2316 in the sensor pixel 2300 (e.g., vpo of the amplifier 2322) are electrically coupled to the switch 2320 to selectively output the output signals Vpo from the sensor pixel 2300 to the signal processing circuit including the filter. Switch 2320 may be implemented using a transistor or other switching mechanism and may be electrically coupled to a controller to control the switching of switch 2320. By controlling the switches 2320, 2310 and 2312, sensor pixels in the sensor pixel array can be selectively switched between acquiring an optical signal and acquiring a capacitive signal. In one implementation, the optical or capacitive signals of the sensor pixels of each row, column or row in the array may be acquired and then switched to acquire other types of signals for that row, column or row. Switching between optical signal acquisition and capacitive signal acquisition may be performed row by row. In another implementation, one type of signal (capacitive or optical) may be acquired for all sensor pixels or elements in the array, and then the one type of signal is switched to acquire the other type of signal for all sensor pixels or elements. Thus, switching between acquisitions of different signal types may occur across the array. Other variations of switching between acquisition of two types of sensor signals may be implemented.

Fig. 19B illustrates a circuit diagram of another exemplary hybrid fingerprint sensing element or pixel 2340. For components having the same reference numerals, the hybrid fingerprint sensing element or pixel 2340 is substantially identical to the sensor pixel 2300. For a description of common components with the same reference numerals, see the description of fig. 19A.

The hybrid fingerprint sensing element or pixel 2340 utilizes a sensing top electrode 2308 to include an aperture or opening 2342 thereon that functions as a collimator to focus or narrow reflected light 2344 toward a photodetector 2314 (e.g., a photodiode). The photodetector 2314 may be positioned or disposed below a collimator implemented using the sensing top electrode 2308 to capture reflected light 2344 focused by the aperture or opening 2342.

In some implementations, separate instances of the sensor signal detection circuitry of the optical sensor and the capacitive sensor may be included to detect sensor signals from the photodetector and the capacitive sensor board in parallel.

Fig. 19C shows a circuit diagram of an exemplary hybrid fingerprint sensing element or pixel 2350 for parallel detection of sensor signals from a photodetector and capacitive sensor plate. For components having the same reference numerals, the hybrid fingerprint sensing element or pixel 2350 is substantially identical to the hybrid fingerprint sensing element or pixel 2340. For a description of common components with the same reference numerals, see the description of fig. 19A.

To detect sensor signals from the capacitive plates and photodetectors in parallel, the hybrid fingerprint sensing element or pixel 2350 includes separate sensor signal detection circuits 2316 and 2317 communicatively coupled to the sensing top electrode 2308 and the photodetectors 2324, respectively. The sensor signal detection circuit 2317 may be implemented substantially similar to the sensor signal detection circuit 2316. In some implementations, the switches 2310 and 2312 may be configured to have first ends electrically coupled to the sense top electrode 2308 and the photodetector 2314, respectively, and second ends coupled to the sensor signal detection circuits 2316 and 2317, respectively, to provide the optical detector signal from the photodetector 2314 and the capacitive sense signal from the sense top electrode 2308 to the sensor signal detection circuits 2316 and 2317, respectively. The sensor signal detection circuits 2316 and 2317 can detect sensor signals from the sensing top electrode 2308 and the photodetector 2314 in parallel when the switch 2310 and the switch 2312 are closed and opened together. The sensor signal detection circuits 2316 and 2317 may detect sensor signals from the sensing top electrode 2308 and the photodetector 2314 in series when the switch 2310 and the switch 2312 are closed and opened out of phase with each other. In addition, sensor device processor 2321 may be communicatively coupled, directly or indirectly, to sensor signal detection circuits 2316 and 2317 through switches 2320A and 2320B to process the detected sensor signals from sensing top electrode 2308 and photodetector 2314 in parallel or in series.

In another aspect of the disclosed technology, the optical sensors described with respect to fig. 17A, 17B, 18, 19A and 19B can be used to measure a person's heart beat by measuring reflected light intensity in the finger over time caused by blood flow changes due to heart beat and pumping action. This information is contained in the received light reflected, scattered or diffused by the finger and carried by the optical detector signal. Thus, the optical sensor may be used for a variety of functions, including acquiring an optical image of a fingerprint and measuring a person's heartbeat. In an implementation, the sensor device processor is configured to process one or more optical detector signals to extract heartbeat information. Such a sensor device processor may be the same as the sensor device processor that processes pixel output signals from optically sensitive pixels or hybrid sensitive pixels to extract fingerprint information.

The following sections describe examples of various designs for fingerprint sensing using an off-screen optical sensor module that directs signal light carrying fingerprint information to an optical sensor array using an optical collimator array or a pinhole array. Such an optical collimator or pinhole is placed between the LCD display screen and the optical sensor array in the off-screen optical sensor module to couple the desired light back from the display panel while filtering the background light of the optical sensor array in optical detection. The implementation of such an optical collimator or pinhole may simplify the optical design of the optical detection of the optical sensor array, e.g. other designs disclosed in this patent document do not require the use of complex optical imaging designs, such as the imaging designs in fig. 6B, 7, 10A and 11. Furthermore, the implementation of such an optical collimator or pinhole may simplify the optical alignment of the overall optical layout to the optical sensor array and improve the reliability and performance of the optical detection of the optical sensor array. Further, such an optical collimator or pinhole may significantly simplify manufacturing and reduce the overall cost of the off-screen optical sensor module.

Fig. 20 shows an example of a smartphone with a liquid crystal display (liquid crystal display, LCD) display and an off-screen optical sensor module that includes an optical collimator array for collecting light and directing the light to an optical detector array for optical fingerprint sensing. An LCD-based display element 423 implements an optical sensing module having a photodiode array 623 beneath the display element 423.

The display assembly 423 is placed under a top transparent layer 431, the top transparent layer 431 serving as a user interface surface for various user interface operations including, for example, touch sensing operations by a user, displaying images to a user, etc., and as an optical sensing interface for accepting a finger for optical fingerprint sensing and other optical sensing operations, wherein probe light is directed from inside the device to the top transparent layer 431 to illuminate the finger. The display assembly 423 is a multi-layer display module 433 that includes a display light source 434 (e.g., LED lamp) that provides a white backlight for the display module 433; an optical waveguide plate 433c coupled to the display light source 434 to receive and guide the backlight light source; an LCD structural layer 433a (including, for example, a Liquid Crystal (LC) cell layer, an LCD electrode, a transparent conductive ITO layer, an optical polarizer layer, a color filter layer, and a touch sensing layer); a light diffuser 433b located below the LCD structural layer 433a and above the optical waveguide plate 433c to spatially diffuse the backlight light sources to illuminate LCD display pixels in the LCD structural layer 433 a; and an optical reflection film 433d positioned under the optical waveguide plate 433c to recycle the backlight source to the LCD structural layer to improve light use efficiency and display brightness; LCD module frame 433e. When the LCD cells in the sensing window are turned on, most of the LCD structural layers 433a (including liquid crystal cells, electrodes, transparent ITO, polarizers, color filters, touch sensing layers, etc.) become partially transparent, although the microstructures may deplete a portion of the probe light energy. The light diffuser 433b, light guide plate 433c, reflective film 433d, and LCD module frame are treated to hold the fingerprint sensor and provide a transparent or partially transparent sensing light path so that a portion of the reflected light from the top surface of the top transparent layer 431 can reach the photodiode array 623 with the LCD underscreen optical sensor module for fingerprint sensing and other optical sensing operations. As shown, this optical sensor module under the LCD screen includes various fingerprint sensor components, such as a collimator 617, for collimating and directing reflected probe light to the photodiode array 623, and an optical sensor circuit module receives and conditions output signals from the photodiode array 623. Collimator 617 may comprise an optical collimator and may be a waveguide-based image emitter, an array of optical fibers, an array of microlenses, or an array of pinholes. The optical collimator is used to define a Numerical Aperture (NA) of the sample image and form a corresponding picture element. Each optical collimator unit obtains a portion of an image of the touch portion of the target finger on the top transparent layer 431. The transmitted beams of all collimators together form a complete image of the target object at photodiode array 623. The photodiode array 623 may be a CMOS sensor of CMOS sensing pixels, a CCD sensor array, or a suitable optical sensor array sensitive to light.

This example shows the inclusion of an electronics module 435 for LCD display and touch sensitive operation; one or more other sensors 425, such as optical sensors for monitoring the light level of the surrounding environment; and optional side buttons 427 and 429 for controlling certain smartphone operations.

In the example of fig. 20, the light sources in the example shown include a display light source 434 and a probe light source 436. The light beam 442a from the detection light source 436 and the light beam 442b from the display light source 434 may be used as sensor detection light for illuminating a finger in contact with the top transparent layer 431 to produce a desired reflected detection light carrying a fingerprint pattern and other information to the optical sensor module.

When the LCD cells in the sensing window are turned on, most of the LCD structural layers 433a (including liquid crystal cells, electrodes, transparent ITO, polarizers, color filters, touch sensitive layers, etc.) become partially transparent, although the microstructures may deplete a portion of the probe light energy. The light diffuser 433b, the light guide plate 433c, the reflective film 433d, and the LCD module frame are processed to hold the fingerprint sensor and provide a transparent or partially transparent sensing light path.

The operation of the above-described LCD under-screen optical sensor module illustrated in fig. 20 is further described below in fig. 21. On the top transparent layer 431, the fingerprint sensing area or window is an area on the top surface of the top transparent layer 431, directly above or near the underlying optical sensor module. Since the optical sensor module is located below the LCD structure, the sensing window is part of the continuous top surface of the top transparent layer 431 and is also part of the display area of the LCD display. Thus, there may be no visible physical demarcation on the top surface to indicate the sensing window. The sensing window may be indicated to the user by other means to assist the user in placing a finger within the sensing window for fingerprint sensing and other optical sensing operations. For example, an additional designated detection light source 436 may be used to illuminate the sensing window such that the area of the sensing window is significantly different from the surrounding area on the top cover glass, which is easily visible to the user. This may be done when the LCD panel is off or when the LCD panel is on.

As shown in fig. 21, the user presses the finger against the sensing window, and the light beam 82P illuminates the finger. The finger and top transparent layer 431 reflect the probe light to form a reflected signal beam 82R. Various scattering interfaces 433S in display module 433 diffuse reflected signal beam 82R to form a diffuse beam 82D. Each collimator unit in collimator 617 selects a light component 82S and directs the selected light component 82S into a corresponding photosensitive detector of photodiode array 623. A photosensitive detector, such as a photodiode or CMOS sensing detector, generates a corresponding sensor signal containing information about the fingerprint pattern. A portion of the light source may enter the fingerprint sensor module without first passing through a finger sensing area on the top surface of the LCD panel. This portion of the light produces background noise and can be eliminated by calibration. Each collimator 617 of the collimator array selects only light transmitted along its allowed direction with relatively low light loss to a corresponding photodetector in a part of the photodiode array 623. Thus, each collimator unit in collimator 617 and its corresponding photodetector in photodiode array 623 operate together to define an effective detection optical Numerical Aperture (NA). The NA directly defines the spatial resolution of the image produced by the optical sensor module.

Based on the disclosed underscreen optical sensing design, a person's finger in direct contact with or near the LCD display screen can return returned light back into the LCD display screen while carrying a portion of the finger's information illuminated by the light output by the LCD display screen. Such information may include, for example, the spatial pattern and location of ridges and valleys of the portion to which the finger is illuminated, and the like. Thus, the optical sensor module may be integrated to capture at least a portion of such returned light to detect the spatial pattern and location of ridges and valleys of the illuminated portion of the finger through optical imaging and optical detection operations. The spatial pattern and locations of ridges and valleys of the illuminated portion of the detected finger may then be processed to construct a fingerprint pattern and fingerprint identification, for example, as part of a user authentication and device access process, to compare with stored authorized user fingerprint patterns to determine if the detected fingerprint is a matching fingerprint. Such optical sensing based fingerprint detection by using the disclosed optical sensor technology uses an LCD display screen as an optical sensing platform and can be used to replace existing capacitive fingerprint sensors or other fingerprint sensors, which are basically stand-alone sensors as "add-on" components, without using light from the display screen or using a fingerprint sensing display screen for cell phones, tablet computers and other electronic devices.

It is noted that an optical sensor module based on the disclosed optical sensor technology may be coupled to the back of an LCD display screen without requiring a designated area on the display surface side of the LCD display screen, which may occupy valuable device surface space in electronic devices such as smartphones, tablet computers, or wearable devices. Such an optical sensor module may be disposed under the LCD display screen, vertically overlapping the display screen area, and hidden behind the display screen area from the perspective of the user. Furthermore, since the optical sensing of such an optical sensor module is performed by detecting light emitted by the LCD display screen and returned from the top surface of the display area, the disclosed optical sensor module does not require a special sensing port or sensing area separate from the display screen area. Thus, in other designs including apple iPhone/iPad devices or samsung Galaxy smart phone models, etc., the fingerprint sensor is located at a specific fingerprint sensor area or port (such as a home button) on the same surface of the display screen and in a designated non-display area outside the display screen area, unlike the fingerprint sensors in the other designs described above, an optical sensor module based on the disclosed optical sensor technology can be implemented by: by using a unique optical sensing design to route light returned from the finger into the optical sensor and by providing a suitable optical imaging mechanism to achieve high resolution optical imaging sensing, fingerprint sensing is allowed at any location on the LCD display screen. In this regard, the disclosed optical sensor technology provides a unique on-screen fingerprint sensing configuration by using the same top touch sensing surface that displays an image and provides touch sensing operation without requiring a separate fingerprint sensing area or port outside the display screen area.

In addition to fingerprint detection by optical sensing, optical sensing may be used to measure other parameters. For example, the disclosed optical sensor technology is capable of measuring the pattern of a person's palm of a large touch area available across the entire LCD display screen (in contrast, some designated fingerprint sensors, such as those in the home button of an apple iPhone/iPad device, have quite small and designated off-screen fingerprint sensing areas that are highly limited in the size of the sensing area, which may not be suitable for sensing large patterns). As another example, the disclosed optical sensor technology may be used not only to capture and detect patterns of a finger or palm associated with a person using optical sensing, but also to detect whether a pattern of a fingerprint or palm captured or detected by a "live finger" detection mechanism is from a live person's hand using optical sensing or other sensing mechanisms, which may be based on the fact that: the fingers of a living person are typically moving or stretching due to the natural movement or motion (intentional or unintentional) of the person, or are typically pulsed as blood flows through the body in connection with the heartbeat. In one implementation, the optical sensor module may detect a change in light returned from the finger or palm due to a heartbeat/blood flow change, thereby detecting whether a living heartbeat is present in an object that appears as a finger or palm. User authentication may enhance access control based on a combination of optical sensing of fingerprint/palm patterns and positive determination of the presence of living persons. For another example, the optical sensor module may include a sensing function for measuring glucose level or oxygen saturation based on optical sensing of returned light from a finger or palm. As another example, when a person touches an LCD display screen, the change in touch force can be reflected in one or more ways, including fingerprint pattern distortion, a change in contact area between a finger and the screen surface, a widening of the fingerprint ridge, or a dynamic change in blood flow. These changes can be measured by optical sensing based on the disclosed optical sensor technology and can be used to calculate touch force. Such touch force sensing can be used to add more functionality to the optical sensor module than fingerprint sensing.

For useful operational or control features related to touch sensing aspects of an LCD display screen, the disclosed optical sensor technology may provide a trigger function or additional function based on one or more sensing results from the optical sensor module to perform certain operations related to touch sensing control of the LCD display screen. For example, the optical properties (e.g., refractive index) of the finger skin are typically different from other artifacts. The optical sensor module may be accordingly designed to selectively receive and detect returned light caused by a finger in contact with the surface of the LCD display screen, while returned light caused by other objects is not detected by the optical sensor module. Such object selective optical detection may be used to provide useful user control through touch sensing, such as waking up a smartphone or device only via a touch of a human finger or palm, while touches of other objects do not cause waking up of the device for energy-saving operation and prolonged use of the battery. Such operation may be achieved by control based on the output of the optical sensor module to control the wake-up circuit operation of the LCD display screen, wherein, for example, an additional light source designed for optical sensing may be included and turned on in a flash mode to intermittently flash light to the screen surface to sense any touch of a person's finger or palm while the LCD display screen may be placed in a sleep mode to save energy. In some implementations, the wake-up sensing light may be in a spectral range where infrared is not visible, so the user does not experience any visual flickering of light.

In some implementations, an optical sensor module based on the disclosed optical sensor technology may be configured as a non-invasive module that can be easily integrated into an LCD display screen without requiring changes to the design of the LCD display screen to provide desired optical sensing functions such as fingerprint sensing. In this regard, an optical sensor module based on the disclosed optical sensor technology may be independent of the design of a particular LCD display screen design, since the optical sensor module has the following properties: the optical sensing of such an optical sensor module is performed by detecting light emitted by the LCD display screen and returned from the top surface of the display area, and the disclosed optical sensor module is coupled to the back surface of the LCD display screen for receiving the returned light from the top surface of the display area, thereby eliminating the need for a specific sensing port or sensing area separate from the display screen area. Thus, such an off-screen optical sensor module may be used in combination with an LCD display screen to provide optical fingerprint sensing and other sensor functions on the LCD display screen without using a specially designed LCD display screen with hardware specifically designed to provide such optical sensing. This aspect of the disclosed optical sensor technology may allow for a wider range of LCD display screens in smartphones, tablet computers or other electronic devices with enhanced functionality from optical sensing of the disclosed optical sensor technology.

For example, for existing phone component designs that do not provide a separate fingerprint sensor, like some apple iPhone or samsung Galaxy models, such existing phone component designs may integrate an off-screen optical sensor module as described herein without changing the touch-sensitive display screen component to provide increased on-screen fingerprint sensing functionality. Because the disclosed optical sensing does not require a separate designated sensing area or port, integration of on-screen fingerprint sensing as disclosed herein does not require substantial changes to existing phone component designs or touch-sensitive display modules having a touch-sensitive layer and a display layer, like some apple iPhone/samsung Galaxy phones have front fingerprint sensors outside the display screen area, or some smartphones like hua, millet, google, or some models of association have designated rear fingerprint sensors on the back. In short, no external sensing port and external hardware buttons are required on the outside of the device for adding the disclosed optical sensor module for fingerprint sensing. The added optical sensor module and associated circuitry is under the display screen within the phone housing and can conveniently fingerprint sense on the same touch sensing surface of the touch screen.

As another example, due to the above-described nature of optical sensor modules for fingerprint sensing, smartphones incorporating such optical sensor modules can be updated with improved designs, functionality, and integration mechanisms without impacting or burdening the design or manufacture of LCD display screens to provide desired flexibility for device manufacturing and improvement/upgrades in product cycles while maintaining availability of updated versions of optical fingerprint sensors in smartphones, tablet computers, or other electronic devices that use LCD display screens. In particular, the touch sensitive layer or LCD display layer can be updated at the next product release without any significant hardware changes for the fingerprint sensing function implemented with the disclosed underscreen optical sensor module. Further, by using a new version of an off-screen optical sensor module, improved on-screen optical sensing for fingerprint sensing or other optical sensing functions, including adding additional optical sensing functions, implemented based on such an optical sensor module, can be added to a new product release without requiring significant changes to the phone component design.

Implementations of the above or other features of the disclosed optical sensor technology can provide improved fingerprint sensing and other sensing functions to new generation electronic devices, particularly for smartphones, tablets, and other electronic devices having LCD display screens, to provide various touch sensing operations and functions, and to enhance the user experience of such devices.

The optical sensor technology disclosed herein uses light in a display screen for displaying images and returned from the top surface of the device display assembly for fingerprint sensing and other sensing operations. The returned light carries information about an object (e.g., a finger) in contact with the top surface and capturing and detecting the returned light forms part of design considerations in implementing a particular optical sensor module located below the display screen. Because the top surface of the touch screen assembly serves as a fingerprint sensing area, an optical image of the touched area should be captured by an optical imaging sensor array within the optical sensor module that has high image fidelity to the original fingerprint for robust fingerprint sensing. The optical sensor module may be designed to achieve such desired optical imaging by properly configuring the optical elements for capturing and detecting the returned light.

Fig. 22A-22B illustrate an exemplary implementation of the optical collimator design of fig. 20 and 21. The optical collimator 2001 in this example includes an array of optical collimators 903 and an optical absorbing material 905 filled between the array of optical collimators 903 to absorb light to reduce cross-talk between the different optical collimators. Each optical collimator array 903 in the optical collimator 2001 may be a channel that extends or is elongated in a direction perpendicular to the display panel, and allows light to be transmitted along its axis with low loss. The optical collimator 2001 is designed to reduce optical crosstalk between different optical collimators and to maintain a desired spatial resolution in optical sensing. In some implementations, one optical collimator may correspond to only one photodetector in the photodetector array 2002. In other implementations, one optical collimator may correspond to one or more photodetectors in photodetector array 2002. As shown in fig. 22B, in some designs, the axis of each collimator unit may be perpendicular to the display screen surface and may be tilted with respect to the display surface. In operation, only light propagating along the collimator axis carries image information. For example, an appropriate light beam 82P is reflected to form light beam 82R. Beam 82R is then diffracted by the small hole of TFT 433T and expands to beam 82D. The light portion 82S is transmitted into the photodetector array 2002. The light portion 82E away from the axis is absorbed by the filler material. The reflected light on the top transparent layer 431 carries fingerprint information. The ray 901 is at an angle relative to the collimator unit axis and thus can be blocked. A portion of the reflected light (e.g., 901E) is transmitted into a corresponding optical collimator to reach the photodetector array 2002.

The optical collimator array may be fabricated by different techniques including etching holes through a planar substrate, forming an optical waveguide array, forming a microlens array that matches an optical filter, using a coreless fiber bundle, or printing collimators on a transparent sheet, etc. Desirable features of such a collimator array include: (1) A sufficient transmission contrast between the light component propagating along the axis and the component propagating away from the axis such that the collimator ensures the desired spatial resolution in the optical sensing of the fingerprint pattern at the photodetector array; (2) The allowed transmission numerical aperture is small enough to achieve the high spatial resolution desired for optical sensing.

Various optical collimator array designs may be used. Each optical collimator in the optical collimator array is configured to perform spatial filtering by transmitting light in a direction along or near an axis of the optical collimator while blocking light in other directions, and has a small transmission numerical aperture to achieve high spatial resolution by the optical collimator array. The small optically transmissive numerical aperture also reduces the amount of background light entering the optical sensor array. The collimator element aperture and spacing (e.g., the distance between two adjacent collimator elements) may be designed to achieve a desired spatial resolution of optical fingerprint sensing.

Fig. 23 shows an example of a collimator design that becomes part of a CMOS structure by using aligned holes in two different metal layers in the CMOS structure. In this particular example, each optical collimator in the array is an elongated channel in a direction perpendicular to the display panel.

Fig. 24 shows an example of an optical fingerprint sensor module under an LCD display structure that incorporates an optical sensor array and an integrated collimator array for each optical sensor pixel in capturing fingerprint information. As shown, the optical sensor array includes a photodetector array and a collimator array disposed over the photodetector array to include optically transparent through-holes and optically opaque metal structures between the through-holes as optical collimators. The illumination light is directed to illuminate the touch portion of the finger and light reflected from the finger is directed through an array of optical collimators to an array of photodetectors that capture an image of a portion of the finger's fingerprint. The optical collimator array may be implemented using one or more metal layers with holes or openings integrated by CMOS processes.

Such an optical collimator in an off-screen optical sensor module may be configured to provide direct point-to-point imaging. For example, the size of the optical collimator array and the size of the individual collimators may be designed to closely match the size of the photodetector array and the size of the individual photodetectors, respectively, to achieve one-to-one imaging between the optical collimators and the photodetectors. The entire image carried by the light received by the optical sensor module may be captured simultaneously by the photodetector array at a single photodetector without stitching.

The spatial filtering operation of the optical collimator array may advantageously reduce the amount of background light entering the photodetector array in the optical sensor module. Furthermore, due to the illumination of the light emitted from the OLED pixels, one or more optical filters may be provided in the optical sensor module to filter out background light and reduce the amount of background light at the photodetector array to improve the optical sensing of the returned light from the fingerprint sensing area. For example, the one or more optical filters may be configured as, for example, bandpass filters to allow transmission of illumination light generated for optical sensing while blocking other light components in sunlight, such as infrared light. Such optical filtering can effectively reduce the background light caused by sunlight when the device is used outdoors. The one or more optical filters may be implemented, for example, as an optical filter coating formed on one or more interfaces along an optical path to the photodetector array in the optical sensor module, or may be implemented as one or more discrete optical filters.

Fig. 25 shows an example of an optical collimator array that reduces background light reaching a photodetector array in an off-screen optical sensor module using optical filtering. This example uses an optical waveguide array as an optical collimator, and one or more filter films are coupled to the optical waveguide array to reduce unwanted background light (e.g., infrared (IR) light from sunlight) reaching a photodetector array coupled to the optical waveguide array while transmitting desired light in a predetermined spectral band of probe light for illuminating a finger. The optical waveguide may include a waveguide core with or without an external waveguide cladding. The optical waveguide may also be formed of coreless fiber bundles having different optical fibers, wherein each unit collimator is a fiber sheet having no fiber core structure. When the coreless fibers are bundled, the filler material between the fibers may include a light absorbing material to increase the absorption of stray light not guided by the coreless fibers. The final collimator may be assembled with a multi-layered array of sub-collimators.

The following sections provide different examples of different optical collimator designs and their fabrication.

Fig. 26A and 26B show an example of manufacturing a collimator by etching. In fig. 26A, a layer of suitable material for forming the optical collimators in the collimator array is formed on or supported by an optically transparent support substrate. An etch mask is formed over the layer and has a pattern for etching the underlying layer to form an optical collimator. A suitable etching process is performed to form the optical collimator. The support substrate may be combined with the collimator array, and may be formed of various optically transparent materials including silicon oxide and the like.

Fig. 26B shows an example of an optical collimator array assembled by stacking multiple layers of sub-collimator arrays via interlayer connection material (which may be adhesive, glass or suitable optically transparent material). In some implementations, the sub-collimator arrays of different layers may be stacked on top of each other without interlayer connection material. Such stacking allows for the manufacture of an optical collimator having a desired length or depth along the collimator axis to achieve a desired optical numerical aperture. The aperture of the collimator geometrically limits the viewing angle. The transmission numerical aperture is determined by the thickness and aperture of the collimator. In some applications, the holes may be filled with an optically transparent material, and may be empty in some designs.

In some implementations, the support substrate may be coated with one or more optical filters to reduce or eliminate background light (e.g., IR light from sunlight) while transmitting light desired in a predetermined spectral band of detection light for illuminating the finger.

Fig. 27 shows an array of optical spatial filters coupled with an array of microlenses, wherein each microlens is positioned relative to a respective through-hole of the optical spatial filter such that each cell collimator includes a microlens and a micro-spatial filter, e.g., a microwell. Each microlens is constructed and positioned to focus received light onto a corresponding micro-spatial filter without imaging the received light. Micropores limit the effective receive numerical aperture. The spatial filter may be printed on an optically transparent substrate or etched on a piece of silicon wafer. The microlens array may be etched by MEMS processing or chemical processing. Microlenses may also be made of graded index material, such as a sheet of graded index glass fibers cut to quarter pitch lengths. The focal length of the microlens and the diameter of the spatially filtered aperture can be used to control the emission numerical aperture of each cell. Like other designs, the collimator plate may be coated with a filter film to reduce or eliminate unused bands of light in the sensor, such as IR light from sunlight.

Fig. 28 shows one example of an integrated CMOS light detection array sensor with built-in light collimation. The collimator is constructed by carding an array of aligned holes 610 in different metal layers 609 and interleaving between the metal layers to provide separate oxide layers 608, thick oxide layers. The apertures may be aligned with a photodetector array 607 in the optical sensor array. The optical fingerprint imager is implemented with such an integrated CMOS photodetector array sensor with built-in light collimation under the LCD display module 710 and cover glass. The fingerprint of a user's finger touching the sensor window area of the cover glass may be imaged by detecting light 706 reflected from the fingerprint valleys and ridges. The light rays at the fingerprint ridge areas will be reduced because the light rays will be absorbed by the fingerprint tissue at the ridge areas, whereas the light rays at the fingerprint valley areas will be stronger compared to the fingerprint ridge areas. This difference in light level between the ridges and valleys of the fingerprint creates a fingerprint pattern at the optical sensor array.

In the above collimator-based optical sensor module designs, each collimator may be designed to be large enough to deliver imaging light to a small area on the optical detector array, or may be designed to be small enough to deliver imaging light to a large area on the optical detector array, along the thickness or length of the collimator. When each collimator in the collimator array is reduced to a certain point along the thickness or length of the collimator, for example, tens of micrometers, the field of the optical view of each collimator may be relatively large to cover a portion of the adjacent optical detectors on the optical detector array, such as a 1mm x 1mm area. In some device designs, optical fingerprint sensing may be achieved by using an array of pinholes, each pinhole in the array having an optical field of view large enough to cover a portion of adjacent optical detectors in the array of optical detectors to achieve high image resolution at the array of optical detectors when sensing a fingerprint. The pinhole array may have a thinner size and a smaller number of pinholes than designs using collimators to achieve the desired high imaging resolution without an imaging lens. Also, unlike imaging via an optical collimator, imaging with a pinhole array uses each pinhole as a pinhole camera to capture an image, an image reconstruction process based on pinhole camera operation differs from an image reconstruction process by using an optical collimator array, namely: each pinhole creates a sub-image area and the sub-image areas of different pinholes in the pinhole array are stitched together to make up the whole image. The image resolution of an optical sensor module with a pinhole array is related to the size of the sensitive elements of the detector array, so that the sensing resolution can be adjusted or optimized by adjusting the detector size.

The pinhole array can be relatively simply manufactured at a low cost based on various semiconductor patterning techniques or processes or other manufacturing methods. The pinhole array may also provide a spatial filtering operation to advantageously reduce the amount of background light entering the photodetector array in the optical sensor module. Similar to designing an optical sensor module with an optical collimator, one or more optical filters may be provided in the optical sensor module with a pinhole array to filter out background light and reduce the amount of background light at the photodetector array to increase the optical sensing of returned light from the fingerprint sensing area due to the illumination of the illumination light generated for optical sensing. For example, the one or more optical filters may be configured as, for example, bandpass filters to allow transmission of illumination light generated for optical sensing while blocking other light components in sunlight, such as infrared light. Such optical filtering can effectively reduce the background light caused by sunlight when the device is used outdoors. The one or more optical filters may be implemented, for example, as an optical filter coating formed on one or more interfaces along an optical path to the photodetector array in the optical sensor module, or may be implemented as one or more discrete optical filters.

In an optical collimator-based optical sensor module, optical imaging resolution at the optical sensor array may be improved by configuring the optical collimator in a manner that provides a pinhole camera effect. Fig. 29 shows an example of such a design.

In fig. 29, collimator cells 652 of such an optical collimator array direct light from the respective detection area cells to photodiode array 623. The apertures of the collimator unit form a field of view (FOV) 618b. If the detector in photodiode array 623 does not capture details in each unit FOV, the imaging resolution is determined by the FOV of each collimator unit. In order to improve the detection resolution, the FOV of each collimator unit needs to be reduced. However, when an optical filter film 618a is provided between each photodetector in the photodiode array 623 and the corresponding collimator unit 652, the small aperture of the collimator unit serves as a pinhole. This pinhole camera effect provides a higher imaging resolution in the image per unit FOV. When there are a plurality of detection elements in the unit FOV, as shown in the pinhole image 621a imaged by the collimator unit, the image details in the unit FOV can be identified. This means that the detection resolution is improved. In implementations, such a gap may be provided in various ways, including, for example, adding an optical filter film 618a between the collimator unit 652 and the photodiode array 623.

With the pinhole camera effect, the filling factor of the collimator plate can be optimized. For example, to detect a 10mm by 10mm sized region, a 10 x 10 collimator array may be used if each unit FOV covers a 1mm by 1mm region. If the detector can obtain 20 x 20 sharpness images in each unit FOV, the overall detection resolution is 200 x 200 or 50 microns or 500psi. This method can be applied to all types of collimation methods.

Fig. 30 shows another example of improving optical imaging resolution using a pinhole camera effect. In this example, the optical sensor module includes a plurality of layers: pad 917, collimator 617 (which may be an array of optical collimators having a sufficiently small thickness), protective material 919, photodiode array 623, and circuit board 605. The object optical distance is determined by the total material thickness from the sensing surface to the pinhole plane, including the optical thickness of the display module 433, the thickness of the gasket 917, any filter coating thickness, any gap thickness, and any adhesive material thickness. The image optical distance is determined by the total material thickness from the pinhole plane to the photodetector array, including the protective material thickness, any filter coating thickness, any gap thickness, and any adhesive material thickness. The image magnification is determined by the image optical distance compared to the object optical distance. The detection mode may be optimized by setting an appropriate magnification. For example, the magnification may be set to be less than 1, such as 0.7 or 0.5, or the like. In some device designs, the shim and pinhole array layer may be combined into a single component. In other designs, the pinhole array and the protective layer may be combined into a single component so that the center coordinates of each pinhole are predefined.

Fig. 31 shows an example of optical imaging based on pinhole camera effects. On the object side, the entire detection area 921 on the LCD display panel is divided into a plurality of sub-detection areas 923. The pinhole array 920 is arranged to image the detection area 921 resulting in an image area 931 of the entire detection area 921. Each pinhole unit in pinhole array 920 is responsible for a small field of view (FOV) 925. Each small FOV 925 covers a sub-detection area 923. As shown in fig. 31, each small FOV of one pinhole may overlap with the small FOV of its neighboring pinhole. On the image side, each sub-detection area 923 in the optical sensor array captures an image 933. As shown in fig. 31, each small FOV 925 of the pinhole has a corresponding image area 935. The magnification of the system may be optimized so that the image of each sub-detection area may be distinguished separately. In other words, the images of the small FOV do not overlap each other. In this detection mode, the center coordinates of each pinhole are predefined and the image point coordinates of each LCD display pixel may be pre-calibrated and such pre-calibration may be used during sensor operation to generate a calibration table for calibration. In this design, the image of the pinhole camera is inverted and signal processing can recover the entire image from the calibration table.

In the above example for an optical collimator, the direction of the optical collimator for directing light from a finger on top of the display screen to the optical sensor array for fingerprint sensing may be perpendicular to the top touch surface of the LCD display screen to collect probe light returned from the finger for fingerprint sensing, with the majority of the light being in the light direction perpendicular to the top touch surface. In practice, by sensing return probe light that is substantially perpendicular to the top touch surface when the touching finger is dry, the image contrast in the image detected in the optical sensor array is lower than the image contrast of the same image obtained from return probe light that is angled with respect to the perpendicular direction of the top touch surface. This is in part because the optical induction of the angled return light spatially filters out the light that is strongly returned from the top touch surface, which is mostly perpendicular to the top touch surface. In view of this aspect of optical sensing of the detection light returning from the top touch surface, the optical collimators may be oriented such that the axis of each collimator unit may be tilted with respect to the top touch surface, as shown by way of example in fig. 22B.

However, in manufacturing, the tilted collimator is more complicated and costly to manufacture. One way to achieve higher contrast in optical sensing using a perpendicular optical collimator as shown in fig. 20 and 21, while also selectively detecting the angled light returning from the top touch surface is: an optical deflection or diffraction device or layer is provided between the vertical optical collimator and the return light from the top touch surface before the light enters the vertical optical collimator. In some implementations, the optical deflection or diffraction device or layer may select between the OLED display panel and the vertical optical collimator only returned probe light at a certain tilt angle to enter the vertical optical collimator for optical detection by an optical detector array located on the other end of the vertical optical collimator while preventing or reducing the amount of probe light returned from the top touch surface perpendicular to the top touch surface from entering the optical collimator. The optical deflection or diffraction device or layer may be implemented in various forms including, for example, a prism array, an optical layer having a diffraction pattern, or other device positioned between the optical collimator and the display panel to select the angled probe light returning from the display panel to enter the optical collimator while reducing the amount of returned probe light perpendicular to the display panel and entering the optical collimator.

Fig. 32 shows an example of an optical sensor module that optically senses using an array of optical pinholes. As shown, a pinhole array 920a is formed between the display module 433 and the photodiode array 623 to image the sensing region pressed by the finger tissue 60 onto the photodiode array 623.

The thickness T of the pinhole array 920a determines the field of view (FOV). The sensing area FOVs and the imaging area FOVi are defined along with the distance from the sensing surface to the pinhole array 920a and from the imaging plane to the pinhole array 920 a. The image magnification is given by Di/Ds, where Di is the thickness of the optically transparent layer 919a between the pinhole array 920a and the photodiode array 623, and Ds is the thickness of the combined stack of layers through the spacer 917, display module 433, and top transparent layer 431. Device parameters such as pinhole layer thicknesses T, ds and Di can be optimized for a desired combination of FOV and image magnification. For example, if desired, the optical sensor module may be configured with desired parameters such that adjacent FOVs of respective adjacent pinholes in pinhole array 920a properly overlap. Similarly, adjacent fovis can also be tuned to overlap or completely separate as discrete fovis. In an optical sensor module designed such that adjacent FOVs overlap each other, some points on the sensing surface may have multiple image points. The label may be used to enhance detection.

In the example of fig. 32, a filter film for reducing background light may be formed or coated on the spacer 917, the pinhole array 920a, the optically transparent layer 919a, or the display surface. As shown, when background light 937 is projected onto the finger tissue 60, the short wavelength component is mostly absorbed and a portion of the long wavelength (e.g., red or infrared) light 939 is transmitted to the photodiode array 623. The filter film may be used to exclude those long wavelength component light 941 to improve detection of the return light signal carrying finger information.

As shown in fig. 33A, each optical collimator 2001 in the collimator array may be an extended channel along an axis perpendicular or perpendicular to the display surface. The viewing angle adapter optical layer 3210 is used to adjust the viewing angle of the probe light returning from the display panel and is located between the optical collimator 2001 and the LCD display panel to select the angled probe light returning from the display panel to enter the optical collimator 2001 while reducing the amount of returned probe light perpendicular to the display panel and entering the optical collimator 2001.

Fig. 33B shows further details of view-angle adapter optical layer 3210 and the main detection light path. For example, viewing angle adapter optical layer 3210 may be implemented as a diffraction pattern layer, such as prismatic structure 3210a. Only the returned probe light 82a and 82b from the finger having an appropriate incident angle and reflected from the display panel can be bent through the optical collimator 2001. In contrast, the returned probe light perpendicular to the display panel is directed by the viewing angle adapter optical layer 3210 away from the original direction perpendicular to the display panel and thus becomes off-axis incident light to the optical collimator 2001. This reduces the amount of returned probe light that is perpendicular to the display panel and can enter the optical collimator 2001.

When the viewing angle is properly adjusted, the received light from the different locations 63a and 63b of the fingerprint valleys carries fingerprint information. For example, under the same illumination, light 82a may be stronger than probe light 82b due to the different fingerprint contours of the skin of the fingertip and the viewing angle. This design allows the optical sensor module to obtain a certain degree of fingerprint shading. This arrangement improves detection when the finger is dry.

In designing an optical sensor module under an LCD display module, various technical features or performance of the LCD display module should be considered and incorporated into the overall optical sensor module design to improve optical sensing operation. The following sections describe several design examples.

One common component in various LCD display modules is a light diffuser, which may be a sheet that diffuses incident light into different directions to achieve a large viewing angle and spatial uniformity of the display. However, the presence of this LCD diffusion layer may reduce the optical detection of the optical sensor module under the LCD screen.

Fig. 34A and 34B show an LCD light diffuser 433B positioned between the optical waveguide plate 433c and the LCD structural layer 433 a. In some LCD assemblies, the top transparent layer 431 may be separated from the underlying light diffuser 433b by a distance (e.g., a few millimeters in some LCD devices), and the collimator 617 is separated from the light diffuser 433b by an optical waveguide plate 433c (which may be a secondary micrometer thickness). With this structure, strong diffusion in the light diffuser 433b can significantly reduce signal contrast in the signal light that passes through the display module 433 to the photodiode array 623. The light diffusion of the light diffuser 433b is required for the display operation, but it reduces fingerprint detection performance.

This undesirable effect of the light diffuser 433b can be mitigated by using different techniques. Two examples are shown in fig. 34A and 34B.

Fig. 34A shows an example in which holes 951a may be formed in a corresponding region in the portion of the light diffuser 433b above the optical sensor module or in the entire light diffuser 433b in the LCD display module to improve transmission of the returned light from the top transparent layer 431 to the photodiode array 623. The size, shape and distribution of the holes may be selected according to specific design requirements. For example, the pore size may be larger than the wavelength of the detection light to avoid intense diffraction. For example, the collimator cell aperture may be about 40 microns in diameter and the diffuser aperture size may be 5 microns, 10 microns, 30 microns, 40 microns, or 100 microns, etc. In this design, the aperture 951a is contained in the light diffuser 433b, i.e. an optical path is established for each collimator unit. The aperture of each collimator unit may have one or more apertures in the diffuser plate to provide the desired light path from the top transparent layer 431 to the photodiode array 623. If the holes of the collimator unit are discrete and have a relatively large pitch (e.g. around 1 mm), the holes in the diffuser plate may be drilled at the same pitch distance. Non-uniformities in the detection can be calibrated.

Fig. 34B illustrates another example, where the diffuser sheet may be configured to include low-diffusion optically transparent dots 951B, where light diffusion is weaker in the area above the optical sensor module to improve light transmission to the optical sensor module. The transparent dot size, shape and distribution may be selected according to specific design requirements. For example, the size of the holes may be larger than the wavelength of the detection light to avoid strong diffraction, and the dot distribution is such that each collimator unit has one or more transparent light paths to allow for efficient reception of return light from the top transparent layer 431 through the LCD display layer. If the collimator unit apertures are discrete and have a large pitch (e.g. around 1 mm), the transparent spots in the diffuser sheet can be manufactured with the same pitch. If the diffuser is made of a roughened surface material that diffracts or diffuses light, a selection material may be selectively applied to the roughened surface to provide a transparent material to reduce the original optical diffusion of the roughened surface. Examples of suitable materials include epoxy, wax, or oil, and these materials can effectively alter diffusion.

For a given LCD diffuser layer, a long wavelength light source may be selected to produce detection or illumination light such that diffuse scattering for such light is weaker so that more light may pass through the diffuser layer to the optical sensor module.

As another example, referring to fig. 35A and 35B, various LCD display modules include an optically reflective layer or film 433d in the LCD below the optical waveguide plate 433c to reflect unused light back to the LCD layer to increase display brightness. However, the presence of the reflective film 433d may block most of the light from reaching the optical sensor module under the LCD screen, and thus may adversely affect the optical fingerprint sensing. The optically reflective layer can be modified in a manner that maintains the desired optical reflection under the LCD waveguide layer in most positions while allowing the desired optical transmission at the position of the optical sensor module under the LCD screen. In some implementations, the collimator 617 of the optical sensor below the LCD screen can be fixed to contact the reflective film 433d.

Fig. 34C illustrates another example for providing a transparent light path for directing light from one or more detection light sources 436 to improve fingerprint sensing of the detection module without being significantly diffused by the diffusion layer. For example, holes 969 may be selectively formed in the light diffuser 433b to improve light transmission to the optical fingerprint sensor under the LCD screen. To avoid affecting display performance, the light path holes may be tilted to maintain a degree of light diffusing function in the region of holes 969. In addition, such apertures 969 may be designed to be small, e.g., 0.3mm or less, to further enhance diffusion of the backlight, while still providing improved optical imaging at the optical fingerprint sensor under the LCD screen. In some implementations, the light path holes may be empty, filled with air, or filled with a transparent material.

In some designs, the apertures 969 may not be limited to a certain area, but may be distributed throughout the light diffuser 433b, e.g., the apertures 969 may be uniformly distributed throughout the light diffuser 433 b. This design eliminates the undesirable spatially non-uniform illumination of the aperture 969 in some areas but not in other areas. In some designs, the apertures 969 may be distributed in a spatially gradient pattern such that any changes in LCD illumination caused by the apertures 969 are gradual and less pronounced.

Fig. 35A illustrates an example in which an optically reflective layer is modified by including or forming light transmitting apertures in regions of the optically reflective film where the optical sensor modules are located to allow optical reflection for LCD display in most locations of the optically reflective film while providing a transparent light path for collimator 617 to receive light reflected from a finger over the LCD. The size, shape, and distribution of the holes may be configured to meet optical sensing requirements. For example, the size of the holes may be larger than the wavelength of the detection light to avoid intense diffraction. For example, the collimator cell aperture may be about 40 microns in diameter, and the diffuser aperture size may be 5 microns, 10 microns, 30 microns, 40 microns, 100 microns, or the like. Each collimator unit aperture may have one or more apertures in the optically reflective layer to provide a desired optical path for optical sensing. Non-uniformities in the detection can be calibrated. If the apertures of the collimator units are discrete and have a large pitch (e.g. around 1 mm), the holes in the reflective film may be drilled with the same pitch.

Fig. 35B shows another example for modifying an optically reflective layer in an LCD, where the optical reflectivity of the optically reflective film can be modified to allow some degree of optical transmission for optical sensing by an underlying optical sensor. Various commercial LCD reflective films use flexible plastic materials as the substrate, and the light transmission of such plastic materials may be sufficient to transmit sufficient light to the optical sensor module for fingerprint sensing.

In the above-described designs for the LCD diffuser layer and LCD reflector layer, holes may be formed in the area where the one or more illumination sources are located to allow illumination light to pass through the LCD display module layer sufficiently to reach the top cover glass for illuminating a finger for optically sensing operation.

In the above design, the optical sensor module is located below the LCD display module and thus below the LCD waveguide layer, which is designed to direct backlight from the backlight source to the LCD display area. As shown in fig. 36, backlight from a display light source 434 (e.g., an LED) is directed by an optical waveguide plate 433c and diffused by the LCD diffusion layer to exit the optical waveguide plate 433c to provide the backlight required for the LCD. The light may uniformly leak out from one side of the optical waveguide plate 433c and then be diffused by the light diffuser 433 b. In some LCDs, approximately half of the diffuse light 957 may propagate toward the collimator 617 and become strong background light in photo-induced detection.

One or more detection light sources 436 may be provided to connect with the optical sensor module to illuminate the finger and provide light carrying fingerprint pattern information to the optical sensor module below the LCD screen. Due to the location of the illumination source (e.g., beside or below the reflective film 433d adjacent to the optical sensor), the light guiding function of the light guide plate 433c is not active on the light from the detection light source 436 so that the light from the detection light source 436 can reach the top surface of the LCD panel more efficiently to illuminate the finger.

In addition, the detection light source 436 may be designed to emit illumination at one or more optical wavelengths that are different from the wavelength of the LCD display illumination light from the display light source 434. The detection light source 436 may be used for fingerprint sensing and other sensing functions.

The above-described design for selecting illumination light of one or more optical wavelengths different from the optical wavelength of the backlight of an LCD display may be used to reduce power consumption. Fingerprint detection using a display backlight requires turning on the display backlight for performing optical fingerprint sensing. This design consumes more power than the above-described design, where the illumination light for optical sensing is partially different from the backlight in optical wavelength, which allows optical sensing operation without turning on the LCD backlight. The above-described design for selecting illumination light of one or more optical wavelengths different from the optical wavelength of the backlight of an LCD display enables flexible selection of the illumination light source for additional advantages. For example, infrared light may be used as the detection light source 436, making the LCD diffusion layer more transparent to the IR illumination light 959 to achieve the desired higher transmission of the IR illumination light. For another example, the illumination source may be selected to provide multiple wavelengths for other functions, such as anti-spoof living body sensing, heartbeat sensing, and the like.

In designing an underscreen optical sensor module for an LCD, the position and spatial distribution of the probe light sources 436 can be used to adjust the viewing angle to optimize the sensing quality.

Additional optical designs may be used to enhance backlight transmission from the waveguide layer to the LCD layer while maintaining adequate transmission of illumination light for optical sensing to the optical sensor module when the optical sensor module is placed under the LCD module.

Fig. 37 shows an example of an enhancement structure including two or more backlight enhancement films (e.g., 433px and 433 py) as part of the LCD layer structure shown in the LCD structural layer 433 a. Backlight enhancement films 433px and 433py are formed over the light diffuser 433 b.

In the example of fig. 37A, each of the reinforcing films 433px and 433py includes a polarizing prism structure. The prism groove directions of the two reinforcing films 433px and 433py are substantially perpendicular to each other to form a pair of reinforcing films together, so as to improve the transmission of illumination light to the LCD panel. However, if not properly configured, such a function of the enhancement film may adversely affect the optical imaging of the optical detector array unit 621U under the LCD screen.

As shown in the examples in fig. 37 (B) and 37 (C), the additional light source illumination direction 963 and the detector viewing direction 961 may be specifically configured not to be along the prism slot direction of the enhancement films 433px and 433py to reduce adverse imaging effects of the enhancement films on optical fingerprint sensing. The design is to obtain a clear image without perforations in the reinforced film. The viewing angle phi 1 and the illumination angle phi 2 should be adjusted according to the design of the enhancement film.

The example in fig. 37 (D) shows a specific design of the collimator unit 617U and the photodetector array unit 621U. Collimator unit 617U is used to provide imaging functionality implemented by microlenses, pinholes, or a combination of both. By optimizing a single detection unit design or by using multiple detection units, a larger sensing area at the photodetector array unit 621U can be achieved.

Fig. 38 shows an example of an optical waveguide layer in an LCD module that includes a portion of illumination light path 967 in detection light path 965 to allow improved light transmission of illumination light for optical sensing through the waveguide layer.

Fig. 39 shows an example of an illumination source for optical sensing designed for use in an optical sensor module under an LCD display module. In an LCD display module, the optically reflective layer enhances LCD display brightness by recycling unused backlight to the LCD layer. In this regard, defects in optical reflectivity along the optical reflective film, such as mechanical defects in the reflective film, can result in visible changes in the brightness of the LCD display, and are therefore undesirable. Fig. 39 shows design features for reducing the adverse effects of defects in the reflective layer or film.

As shown in fig. 39 (a), a micro hole 973 may be provided in the reflection film 433d at the position of the detection light source 436 of the visible light component in the illumination light. The visible light component may be used to provide illumination within a limited area of the display to display the necessary text or identification information without turning on the display backlight.

As shown in fig. 39 (B), another solution is to select the illumination source wavelength so that it no longer falls within the operating band of the reflective film, which is typically in the visible band. The detection light source 436 for optical sensing may be outside the reflective spectral range of the reflective film, for example, outside the short wavelength range below 400nm (e.g., 380 nm) or the long wavelength range beyond the visible red range (e.g., 780nm, 900nm, 940nm, etc.), so that the illumination light may pass through the reflective film or layer without forming holes in the reflective film.

Fig. 39 (C) shows another design in which the reflective film is designed to include a narrow-band transmission window 975 for transmitting illumination light for optical detection. For example, this narrow transparent or transmissive window in the reflective film may be between 525nm and 535 nm.

A portable device or other device or system such as a mobile phone based on the optical induction disclosed herein may be configured to provide additional operational features.

For example, the LCD display panel may be controlled to provide a partial flash mode to illuminate the fingerprint sensing area by operating selected LCD display pixels below view area 613. This may be provided in an optical sensor module under the LCD display panel, for example, fig. 4A and 4B based on an optical imaging design, or fig. 21 based on optical imaging by an optical collimator array. In the case of acquiring a fingerprint image, the LCD display pixels and illumination sources in the sensing window area may be temporarily turned on to produce high intensity illumination for optical sensing of the fingerprint, and at the same time the photodiode array 623 is turned on to capture a fingerprint image synchronized with the turning on of the illumination sources. The time for turning on the illumination source may be relatively short, but the emission intensity may be set to be high. For this reason, this mode for optical fingerprint sensing is a flash mode, which enables the photodiode array 623 to detect a larger amount of light to improve image sensing performance.

The optical sensor disclosed above for sensing optical fingerprints may be used to capture high quality fingerprint images to enable discrimination of small variations in captured fingerprints captured at different times. Notably, when a person presses a finger on the device, contact with the top touch surface on the display screen may change due to changes in the pressing force. When a finger contacts a sensing area on the cover glass, the change in contact force may cause several detectable changes on the optical sensor array: (1) fingerprint deformation, (2) change of contact area, (3) fingerprint ridge widening, and (4) dynamic change of blood flow of pressed area. These changes can be optically captured and used to calculate corresponding changes in touch force. Touch force sensing adds more functionality to fingerprint sensing.

Referring to fig. 40, the contact profile area increases with increasing pressing force, while the ridge embossing expands with increasing pressing force. Conversely, the contact profile area decreases with decreasing pressing force, while the ridge embossing contracts or contracts with decreasing pressing force. Fig. 40 shows two different fingerprint patterns of the same finger at different pressing forces: a lightly pressed fingerprint 3301 and a heavily pressed fingerprint 3303. The probe light returned from the selected integrated area 3305 of the fingerprint on the touch surface may be captured by a portion of the optical sensors on the optical sensor array that corresponds to the selected integrated area 3305 on the touch surface. As explained further below, the detection signals from those optical sensors are analyzed to extract useful information.

When a finger touches the sensor surface, the finger tissue absorbs optical power, and thus the received power integrated on the photodiode array is reduced. Particularly without inducing a total internal reflection pattern of low refractive index materials (water, sweat, etc.), the sensor can be used to detect if a finger touches the sensor or other object accidentally touches the sensor by analyzing the received power trend. Based on this sensing process, the sensor can determine whether the touch is a real fingerprint touch and thus can detect whether to wake up the mobile device based on whether the touch is a real finger press. Because the detection is based on integrated power detection, the light source for optical fingerprint sensing is in a power saving mode.

In the detailed fingerprint drawings, as the pressing force increases, the fingerprint ridge expands, and more light is absorbed at the touch interface by the expanded fingerprint ridge. Thus, within a relatively small selected integration area 3305, the integrated received-light power variation reflects the variation in pressing force. Based on this, the pressure can be detected.

Thus, by analyzing the received probe light power variations integrated in a small area, the time-domain evolution of the fingerprint ridge pattern deformation can be monitored. Information about the temporal evolution of the fingerprint ridge pattern deformation may then be used to determine the temporal evolution of the pressing force on the finger. In an application, the temporal evolution of the pressing force of a person's finger may be used to determine the dynamics of a user interaction through the touch of the finger, including determining whether the person presses or removes the pressing finger from the touch surface. These user interaction dynamics may be used to trigger certain operations of the mobile device or operations of certain applications on the mobile device. For example, a temporal evolution of the pressing force of a person's finger may be used to determine whether a person's touch is intended for operating the mobile device or an unexpected unintended touch, and based on such a determination, the mobile device control system may determine whether to wake the mobile device in sleep mode.

Further, a living finger in contact with the touch surface may exhibit different characteristics in terms of extinction ratios obtained at two different detection light wavelengths under different pressing forces, as explained in fig. 14 and 15. Referring back to fig. 40, a lightly pressed fingerprint 3301 may not significantly restrict blood flowing into the pressed portion of the finger and thus produce extinction ratios obtained at two different probe wavelengths indicative of living tissue. When a person presses a finger with force to produce a heavily pressed fingerprint 3303, the blood flow to the pressed finger portion may be severely reduced, and thus the corresponding extinction ratio obtained at two different probe wavelengths will be different from that of the lightly pressed fingerprint 3301. Thus, the extinction ratio obtained at two different probe light wavelengths varies with different pressing forces and different blood flow conditions. This variation is different from the extinction ratio obtained at two different wavelengths of detection light when pressed with different forces according to a false fingerprint pattern made of an artificial material.

Thus, the extinction ratio obtained at two different wavelengths of detection light can also be used to determine whether a touch is from a user's finger or other object. Such a determination may also be used to determine whether to wake the mobile device in sleep mode.

As another example, the disclosed optical sensor technology may be used to monitor the natural motion that a living finger tends to exhibit due to the natural movement or motion of a person (intentional or unintentional) or pulse related to heart beat as blood flows through the person. The wake-up operation or user authentication may enhance access control based on a combination of optical sensing of the fingerprint pattern and positive determination of the presence of a living being. As another example, the optical sensor module may include sensing functionality for measuring glucose level or oxygen saturation based on optical sensing in the returned light from the finger or palm. As yet another example, when a person touches the display screen, the change in touch force may be reflected in one or more ways, including fingerprint pattern deformation, a change in contact area between the finger and the screen surface, a widening of the fingerprint ridge, or a dynamic change in blood flow. These and other variations can be measured by optical sensing based on the disclosed optical sensor technology and can be used to calculate touch force. In addition to fingerprint sensing, such touch force sensing may be used to add more functionality to the optical sensor module.

While this patent document contains many specifics, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular inventions. Certain features that are described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

Similarly, although operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Moreover, the various individual system components in the embodiments described in this patent document should not be construed as requiring such separation in all embodiments.

This patent document describes only a few implementations and examples, and other implementations, enhancements, and variations can be made based on what is described and shown in this patent document.

Claims

1. An optical sensor device for use in an electronic device having an LCD display panel for optical detection, comprising:

one or more detection light sources for being disposed below the LCD display panel to generate detection light passing through the LCD display panel and to illuminate an object or finger above the LCD display panel, the one or more detection light sources being different from an LCD illumination light source, the one or more detection light sources being for emitting visible light to illuminate a sensing window of a display area of the LCD display panel such that the area of the sensing window is different from a surrounding area at a top of the LCD display panel such that whether the LCD display panel is off or the LCD display panel is on, it is easily visible to a user;

An optical sensor module disposed under the LCD display panel to receive the probe light passing through the LCD display panel to detect a fingerprint;

the one or more detection light sources are disposed proximate to the optical sensor module, the optical sensor module being configured to control the one or more detection light sources to intermittently emit flickering light to the LCD display panel when the LCD illumination light source is turned off to place the LCD display panel in a sleep mode without emitting light, and to detect light returned by the flickering light through the LCD display panel due to a touch of the finger, and to wake up the LCD illumination light source and the LCD display panel when the returned light is detected.

2. The optical sensor device of claim 1, further comprising: a device electronic control module coupled to the optical sensor module to receive information of a plurality of detected fingerprints obtained by sensing a touch of a finger, for measuring a change in the plurality of detected fingerprints and determining a touch force causing the change.

3. The optical sensor device of claim 2, wherein the change comprises a change in a fingerprint image due to the touch force, a change in a touch area due to the touch force, or a change in a pitch of fingerprint ridges.

4. The optical sensor device of claim 1, wherein the optical sensor module comprises an array of optical collimators that receive the probe light, the optical collimators comprising a substrate having an array of through holes formed therein as the optical collimators.

5. An electronic device, comprising:

a Liquid Crystal Display (LCD) screen providing touch sensing operation and including an LCD display panel to display an image;

a top transparent layer formed over the liquid crystal display LCD screen as a user touch interface for the touch sensing operation and as an interface for transmitting light from the LCD display panel to display an image to a user; and

the optical sensor device of any one of claims 1-4.

6. The electronic device of claim 5, wherein the LCD display panel includes a light diffusing layer that diffuses light and the light diffusing layer includes a first aperture located at a selected area above the optical sensor module to allow light to be transmitted to the optical sensor module of the optical sensor device.

7. The electronic device of claim 6, wherein each through-hole of the optical collimator array of the optical sensor module corresponds to one or more of the first holes to provide an optical path from the top transparent layer to the optical sensor module.

8. The electronic device of claim 7, wherein a second aperture is provided at the light diffusing layer corresponding above one or more detection light sources of the optical sensor device to allow detection light generated by the one or more detection light sources to pass through the LCD display panel to the top transparent layer.

9. The electronic device of claim 8, wherein the first aperture and/or the second aperture are disposed obliquely in the light-diffusing layer.

10. The electronic device of claim 5, wherein the LCD display panel includes a light diffusing layer that diffuses light, and the light diffusing layer includes selected areas over an optical sensor module of the optical sensor device that diffuse light, the selected areas allowing some light to pass through to the optical sensor module than other portions of the light diffusing layer.

11. The electronic device of claim 5, wherein the LCD display panel includes an optical reflective layer formed at a bottom region of the LCD display panel to reflect light to the LCD display panel.

12. The electronic device of claim 11, wherein an optical collimator array of the optical sensor device contacts the optically reflective layer.

13. The electronic device of claim 11, wherein the optically reflective layer includes a third aperture formed in a selected region over an optical sensor module of the optical sensor device to allow light to be transmitted to the optical sensor module.

14. The electronic device of claim 13, wherein each through-hole of the optical collimator array of the optical sensor device corresponds to one or more of the third holes.

15. The electronic device of claim 11, wherein micro-holes are provided at the corresponding optically reflective layer above one or more detection light sources of the optical sensor device to allow the detection light to pass through the LCD display panel to the top transparent layer.

16. The electronic device of claim 11, wherein the optically reflective layer comprises a selected area over an optical sensor module of the optical sensor device to allow light to be transmitted to the optical sensor module.

17. The electronic device of claim 5, further comprising: an LCD illumination source, providing backlight to the LCD display panel, for displaying an image.

18. The electronic device of claim 5, wherein the LCD display panel comprises one or more backlight-enhancing film layers, each comprising an optically polarizing prism structure having two sets of prism slots that are perpendicular to each other.

19. The electronic device of claim 18, wherein the optical sensor module receives probe light generated by one or more probe light sources of the optical sensor device in a direction different from a prism slot direction and reflected from the top transparent layer.

20. The electronic device of claim 18, wherein the backlight enhancement film layer is disposed over a light diffusing layer of the LCD display panel.

21. The electronic device of any of claims 11-18, wherein the LCD display panel includes a waveguide layer disposed between a light diffusing layer of the LCD display panel and the optically reflective layer to direct backlight to the liquid crystal display LCD screen.

22. The electronic device of claim 21, wherein the waveguide layer comprises a transparent region to allow the probe light to be transmitted through the waveguide layer.