WO2023063862A1 - Shading compensated readout - Google Patents

Shading compensated readout Download PDF

Info

Publication number
WO2023063862A1
WO2023063862A1 PCT/SE2022/050897 SE2022050897W WO2023063862A1 WO 2023063862 A1 WO2023063862 A1 WO 2023063862A1 SE 2022050897 W SE2022050897 W SE 2022050897W WO 2023063862 A1 WO2023063862 A1 WO 2023063862A1
Authority
WO
WIPO (PCT)
Prior art keywords
fingerprint sensor
optical fingerprint
image
electronic device
values
Prior art date
Application number
PCT/SE2022/050897
Other languages
French (fr)
Inventor
Petter ÖSTLUND
Original Assignee
Fingerprint Cards Anacatum Ip Ab
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fingerprint Cards Anacatum Ip Ab filed Critical Fingerprint Cards Anacatum Ip Ab
Publication of WO2023063862A1 publication Critical patent/WO2023063862A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/12Fingerprints or palmprints
    • G06V40/1347Preprocessing; Feature extraction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/12Fingerprints or palmprints
    • G06V40/13Sensors therefor
    • G06V40/1318Sensors therefor using electro-optical elements or layers, e.g. electroluminescent sensing

Definitions

  • the present invention relates to method for image shading compensation of images acquired by an optical fingerprint sensor.
  • the invention also relates to an optical fingerprint sensor, to an electronic device comprising an optical fingerprint sensor, and to a control unit configured to execute steps according to the method.
  • Biometric systems are widely used as means for increasing the convenience and security of personal electronic devices, such as mobile phones etc. Fingerprint sensing systems, in particular, are now included in a large proportion of all newly released consumer electronic devices, such as mobile phones.
  • Optical fingerprint sensors have been known for some time and may be a feasible alternative to e.g., capacitive fingerprint sensors in certain applications.
  • Optical fingerprint sensors may for example be based on the pinhole imaging principle and/or may employ micro-channels, i.e. , collimators or microlenses or camera-type lenses to focus incoming light onto an image sensor.
  • the display includes holes which allows transmission of light to the sensor.
  • the display causes a shading effect in the image on the sensor.
  • the fingerprint modulation is only a relatively small fraction of the total signal detected by the sensor. Because of the shading one needs a large dynamic range, i.e., bit depth, to extract the fingerprint modulation at different signal levels. Consequently, a relatively large memory buffer is needed for enabling the large bit depth. If one could reduce the number of bits for each pixel the buffer size and the readout time from the sensor chip, through SPI or similar interface, can be reduced. Accordingly, there is a room for improvements with regards to the buffer size of the fingerprint sensor and reducing the readout time from the sensor chip.
  • an object of the present invention to provide a method for image shading compensation of images acquired by an optical fingerprint sensor mountable under an at least partly transparent display panel that at least alleviates some drawbacks of prior art.
  • the method comprises the following steps. Acquiring, with the optical fingerprint sensor, an image having a global intensity variation. On-chip of the optical fingerprint sensor, adding or subtracting a set of compensation values to/from pixel values of the acquired image to produce a compensated image, the set of compensation values being adapted to reduce the global intensity variation of the acquired image. Providing the compensated image from the optical fingerprint sensor to the host electronic device.
  • the global intensity profile of the acquired image is flattened out by the addition or subtraction of the compensation values to/from the pixel values.
  • the global variation in intensity across the image is reduced which at the same time reduces the required bit depth for the on-chip memory.
  • Adding or subtracting a set of compensation values is performed at read-out of the pixel array, on chip of the optical fingerprint sensor.
  • a global intensity variation is meant as an intensity variation caused by shading, also known as vignetting or lens roll-off, and that affects a variation of intensity over a large distance compare to the dimension of the fingerprint modulation. Often, a global intensity variation reaches across the entire image.
  • Image shading is an effect caused by components of the entire optical system including e.g., cover glass, displays, filters, etc.
  • the compensated image is stored in an on-chip memory storage with fewer bits per pixels than a bit depth of an analogue to digital converter used for reading out the pixel values.
  • the analogue to digital converter is on chip with the pixel array.
  • the present invention is at least partly based on the realization that by performing additive or subtractive compensation on-chip to reduce the global intensity variation, the required bit depth for each pixel is reduced which also leads to reduced readout time from the image sensor.
  • a reduced bit depth allows for using a smaller on-chip buffer memory, thereby also reduced on- chip footprint for the buffer memory.
  • the compensation is additive/subtractive to not exaggerate intensity variations where the shading compensation is large, which may happen close to the edges of the image sensor, compared to a multiplicative shading compensation which is a common step in digital image processing.
  • the inventors further realized that the additive or a subtractive compensation is possible since, in fingerprint imaging, the shading profile, from different targets is very similar.
  • the target being a fingerprint
  • the parameters used for the shading compensation can be known prior to use of the optical fingerprint sensor.
  • a respective compensation value may be added or subtracted to/from each of the pixel values of the pixel array.
  • the shading compensation is applied across the entire image, to each pixel. This improves the accuracy of the shading compensation and the quality of the resulting compensated image, as well as ensures that the bit depth is reduced for all pixels.
  • the method may comprise calculating the compensation values, on chip, from a predefined formula comprising a set of coefficients.
  • the compensation values themselves are calculated on-chip.
  • the coefficients may be stored on a register memory of the optical fingerprint sensor and are retrieved for calculating the compensation values on-chip.
  • the formula may be in the form of a parameterized formula where the coefficients of the formula may be written to the sensor memory. At readout of the pixels from the pixel array the compensation values are applied to every pixel according to the parametrized formula.
  • the coefficients are written to a register memory of the optical fingerprint sensor upon starting the optical fingerprint sensor.
  • the coefficients may be constructed in a calibration step and stored on a memory of the host electronic device. Thus, upon starting the optical fingerprint sensor, the coefficients are written to the register memory of the optical fingerprint sensor.
  • the method may comprise calculating the compensation values from an interpolation between the pixel values.
  • an interpolation may be used.
  • the compensated image may have a global intensity variation that is less 50% of the global intensity variation of the acquired image. This leads to that the bit depth is reduced by at least one bit. Further, truncation of the pixel values may be used, where pixel values exceeding an upper limit value or are lower than a lower limit value are rounded off to a value within the upper and lower limits.
  • the reduction of the global intensity variation is performed to a degree so that the fingerprint modulation is not flattened out, thus the reduction is limited to a threshold where the fingerprint modulation is still detectable, including a margin for e.g., sun light or other sources that may cause stray light that may reach the image sensor, often from the side edges of the sensor.
  • the bit depth of the compensated image should be sufficient to be able to detect the fingerprint modulation.
  • the bit depth could be determined based on an expected bit depth needed to detect the fingerprint modulation.
  • the compensation is performed such that the expected bit depth is used at read out.
  • the method may comprise determining a DC level, or offset level, from an average value of a set of pixels values from center pixels of the pixel array, wherein the DC level of offset is added to the compensation values.
  • an average value of the centre pixels is determined in a calibration step.
  • the centre position, where the maximum intensity is, can be stored in the calibration step and written to the sensor upon start.
  • the DC level or offset level is the average value of the center pixels.
  • fewer bits may be needed for extracting fingerprint modulation in the compensated image compared to the number of bits needed for extracting fingerprint modulation from the acquired image. For example, if an ADC with a given number of bits per pixel is used for reading out the acquired image, that image is subject to the compensation, whereby the compensated image is read and sent to the host. The number of bits sent per pixel is then less than the number of bits per pixel of the ADC.
  • the method may comprise extracting fingerprint modulation in the compensated image with fewer number of bits compared to the number of bits required for extracting fingerprint modulation from the acquired image.
  • an optical fingerprint sensor configured to be arranged under an at least partly transparent display panel of a host electronic device, the optical fingerprint sensor comprising: an image sensor comprising photodetector pixel array configured to acquire an image of an object on the opposite side of the at least partly transparent display panel, the image having a global intensity variation; and, the optical fingerprint sensor comprising means configured to: add or subtract a set of compensation values to/from pixel values of the acquired image to produce a compensated image, the set of compensation values being adapted to reduce the global intensity variation of the acquired image, and to provide the compensated image from the optical fingerprint sensor to the host electronic device.
  • the outer surface of a display panel under which the optical fingerprint sensor is arranged may also be referred to as a sensing surface.
  • the operating principle of the described optical fingerprint sensor may be that light emitted by pixels in the display panel will be reflected by a finger placed on the sensing surface, and the reflected light is received by the light redirecting elements and subsequently redirected onto a corresponding subarray of pixels or a single pixel in the photodetector pixel array.
  • an image of a portion of a finger can be captured for each subarray.
  • the transparent display panel may comprise a color controllable light source.
  • Various types of displays can be used in accordance with embodiments. For example, display panels based on OLED, u-LED with any type of tri-stimulus emission like RGB, CMY or others. Thereby, in-display biometric imaging is enabled.
  • the optical fingerprint sensor may comprise means configured to calculate the compensation values, on chip, from a predefined formula comprising a set of coefficients.
  • the optical fingerprint sensor may comprise an on- chip register memory onto which, upon starting the optical fingerprint sensor, the predefined set of compensation values are written.
  • an electronic device comprising: an at least partly transparent display panel; the optical fingerprint sensor according to any one of the herein mentioned or derivable embodiments, and processing circuitry configured to: receive a signal from the optical fingerprint sensor indicative of a fingerprint of a finger touching the at least partly transparent display panel, perform a fingerprint authentication procedure based on information comprised in the signal.
  • the electronic device may comprise a non-volatile calibration memory for storing the coefficients of the formula, the electronic device comprising a device controller configured to write the coefficients to a register memory of the optical fingerprint sensor.
  • the electronic device may be e.g., a mobile device such as a mobile phone (e.g., Smart Phone), a tablet, a phablet, etc.
  • a mobile device such as a mobile phone (e.g., Smart Phone), a tablet, a phablet, etc.
  • control unit configured to perform the steps of any one of the herein mentioned methods.
  • Fig. 1 schematically illustrates an example of an electronic device according to embodiments of the invention
  • Fig. 2 is a schematic box diagram of an electronic device according to embodiments of the invention.
  • Fig. 3 schematically illustrates an optical fingerprint sensor according to an embodiment of the invention
  • Fig. 4A conceptually illustrates a global intensity variation across an image
  • Fig. 4B conceptually illustrates a global intensity variation across an image on which the compensation values discussed herein has been applied
  • Fig. 5 is a block diagram of an optical fingerprint sensor and a host electronic device according to embodiments of the invention
  • Fig. 6 is a flow-chart of method steps according to embodiments of the invention.
  • Fig. 7 is a flow-chart of method steps according to embodiments of the invention.
  • optical fingerprint sensor According to the present detailed description, various embodiments of the optical fingerprint sensor according to the present invention are mainly described with reference to an optical fingerprint sensor arranged under a display panel. However, it should be noted that the described imaging arrangement also may be used in other optical biometric imaging applications such as in an optical biometric arrangement located under a cover glass or the like.
  • FIG. 1 there is schematically illustrated an example of an electronic device configured to apply the concept according to the present disclosure, in the form of a mobile device 101 with an integrated in-display optical fingerprint sensor 100 and a display panel 104 with a touch screen interface 106.
  • the optical fingerprint sensor 100 may, for example, be used for unlocking the mobile device 101 and/or for authorizing transactions carried out using the mobile device 101 , etc.
  • the optical fingerprint sensor 100 is here shown to be smaller than the display panel 104, but still relatively large, e.g., a large area implementation.
  • the optical fingerprint sensor 100 may be the same size as the display panel 104, i.e. , a full display solution. Thus, in such case the user may place his/her finger anywhere on the display panel for biometric authentication.
  • the optical fingerprint sensor 100 may in other possible implementations be smaller than the depicted optical fingerprint sensor, such as providing a hot-zone implementation.
  • the mobile device 101 shown in Fig. 1 further comprises a first antenna for WLAN/Wi-Fi communication, a second antenna for telecommunication communication, a microphone, a speaker, and a phone control unit.
  • a first antenna for WLAN/Wi-Fi communication for WLAN/Wi-Fi communication
  • a second antenna for telecommunication communication for telecommunication communication
  • a microphone for telecommunication communication
  • a speaker for telecommunication communication
  • a microphone for a speaker
  • a phone control unit for controlling the mobile device 101 shown in Fig. 1
  • Further hardware elements are of course possibly comprised with the mobile device.
  • the invention may be applicable in relation to any other type of electronic devices comprising transparent display panels, such as a laptop, a tablet computer, etc.
  • Fig. 2 is a schematic box diagram of an electronic device according to embodiments of the invention.
  • the electronic device 200 comprises a transparent display panel 204 and an optical fingerprint sensor 100 conceptually illustrated to be arranged under the transparent display panel 204 according to embodiments of the invention.
  • the electronic device 200 comprises processing circuitry such as control unit 202.
  • the control unit 202 may be stand-alone control unit of the electronic device 200, e.g., a device controller.
  • the control unit 202 may be comprised in the optical fingerprint sensor 100.
  • the control unit 202 is configured to receive a signal indicative of a detected object from the optical fingerprint sensor 100.
  • the signal is indicative of a fingerprint of a finger touching the at least partly transparent display panel.
  • the received signal may comprise image data.
  • control unit 202 Based on the received signal the control unit 202 is configured to detect a fingerprint, and based on the detected fingerprint, the control unit 202 is configured to perform a fingerprint authentication procedure. Such fingerprint authentication procedures are considered perse known to the skilled person and will not be described further herein.
  • Fig. 3 schematically illustrates an optical fingerprint sensor 100 according to an embodiment of the invention.
  • the optical fingerprint sensor 100 is here arranged under an at least partially transparent display panel 301 .
  • the optical fingerprint sensor 100 may be arranged under any cover structure which is sufficiently transparent, as long as the image sensor 303 receives a sufficient amount of light to capture an image of a biometric object in contact with the outer surface of the cover structure, such as a fingerprint or a palmprint.
  • an optical fingerprint sensor 100 configured to capture an image of a finger 304 in contact with an outer surface 306 of a cover glass 302 of the display panel 301 is described.
  • the optical fingerprint sensor 100 comprises the image sensor 303 including the photodetector pixel array 307, where each pixel 312 is an individually controllable photodetector configured to detect an amount of incoming light and to generate an electric signal indicative of the light received by the detector.
  • the image sensor 303 may be any suitable type of image sensor, such as a CMOS or CCD or TFT based sensor connected to associated control circuitry.
  • a thin-film transistor (TFT) based image sensor provides a cost-efficient solution. The operation and control of such an image sensor can be assumed to be known and will not be discussed herein.
  • the optical fingerprint sensor 100 further comprises an optical stack 314 arranged to cover the image sensor 104.
  • the optical stack 314 may include various layers and components such as a transparent substrate covering the image sensor 303, a set of optical redirection elements such as at least one lens 316, opaque layers having of separate openings for the lenses, an adhesive layer to attach the display panel 301 to the optical fingerprint sensor 100, air gaps, optical filters such as color filter and IR-cut off filters, and antireflection coatings, depending on the specific implementation.
  • the at least partly transparent display panel 301 here comprises a color controllable light source 318 comprising individually controllable light emitting pixels 320.
  • the color controllable light source 318 may emit light that is reflected by the finger 304 and detected by the pixels 312 of the image sensor 303.
  • Fig. 4A conceptually illustrates a global intensity variation across an image acquired by an optical fingerprint sensor arranged under an at least partly transparent display panel.
  • the x and y axes represent pixels and the z axis represent intensity in arbitrary units.
  • the typical lens roll-off effect or shading effect is seen as the intensity decrease towards the edges of the image.
  • the intensity variation causes a relatively large signal depth, from a maximum intensity to a minimum intensity, which necessitates a large dynamic range to be able to resolve the fingerprint modulation that lies superpositioned with the global intensity variation. This further requires a relatively large number of bits in the buffer memory on the chip of the fingerprint sensor.
  • the upper boundary 402 and lower boundary 404 conceptually illustrates the bit depth b1 required for the global intensity variation shown in fig. 4A.
  • the bit depth b1 is required which also includes a margin for e.g., stray light.
  • Fig. 4B conceptually illustrates a global intensity variation across an image on which the compensation values discussed herein has been applied. Compensation values has been added to, or subtracted from, the pixel values of the image shown in fig. 4A to produce the compensated image.
  • the global intensity variation being defined as the difference between a maximum intensity and a minimum intensity is significantly smaller in the image shown in fig. 4B compared to that in fig. 4A.
  • This compensation leads to a reduced need for bit depth for detecting the fingerprint modulation and thereby also to that the buffer memory on chip of the fingerprint sensor can be reduced.
  • the upper boundary 406 and lower boundary 408 illustrates the bit depth b2 required for the global intensity variation shown in fig. 4B. Accordingly, the indicated bit depth b2 is smaller than the bit depth b1 conceptually indicated in fig. 4A.
  • Fig. 5 is a block diagram for schematically describing an optical fingerprint sensor according to embodiments of the invention.
  • the optical fingerprint sensor 100 comprises a read-out circuitry 322 controllable for converting analog sensing signals to digital signals.
  • the analog sensing signals being indicative of an image acquired by the image sensor 303 comprising the array 307 of photodetectors 312 of which not all are numbered.
  • the optical fingerprint sensor may be included in the host device 101 , 200 which may be e.g., the electronic device 200 or the mobile device 101.
  • a data transfer bus 324 is configured to transfer image data to the host device 101 , 200 and may be a serial peripheral interface (SPI) or an I3C interface although other types of data transfer busses configured to transfer data are conceivable.
  • the optical fingerprint sensor comprises a communication interface 328 having an SPI port including a serial clock input (SCLK); a master output slave input (MOSI), a master input slave output (MISO), and a slave select input (CS).
  • the sensing signal therefrom are read-out and digitalized by the read-out circuitry 322 and subsequently they are transferred, in digital form, to the host device 101 , 200 on the data transfer bus 324.
  • the read-out block 322 includes electrical components known perse, for transferring charges from the photodiodes of the image sensor pixel matrix to a voltage, e.g., an analog sensing signal, and for converting the analog signals to a digital signal, i.e. , analog to digital converters.
  • a memory storage 326 such as a frame buffer, or alternatively a first- in-first-out (FIFO) type memory if a frame buffer is not present or available may be included in the optical fingerprint sensor 100 and be configured to store data indicative of the acquired sensing signals.
  • the size of this buffer depends on the required bit depth for extracting fingerprint modulation from an acquired image. If the global intensity variation discussed with reference to figs. 4A-B can be reduced, then the size of this buffer 326 can also be reduced, and so can the read-out time via the data transfer bus 324.
  • Fig. 6 is a flow-chart of method steps according to embodiments of the invention.
  • step S102 acquiring, with an optical fingerprint sensor 100, an image having a global intensity variation.
  • the optical fingerprint sensor 100 being mountable under an at least partly transparent display panel 301 of a host electronic device 101 , 200 and is configured to acquire an image of an object on the opposite side of the at least partly transparent display panel.
  • the optical fingerprint sensor comprising an image sensor comprising photodetector pixel array.
  • step S104 performed on-chip of the optical fingerprint sensor, adding or subtracting a set of compensation values to/from pixel values of the acquired image to produce a compensated image.
  • the set of compensation values being adapted to reduce the global intensity variation of the acquired image.
  • the optical fingerprint sensor 100 comprises on-chip means, for example a control unit being part of the read-out circuitry 322, that is configured to perform the addition or subtraction of the compensation values.
  • step S106 providing the compensated image from the optical fingerprint sensor 100 to the host electronic device 101 , 200.
  • the compensated image may be transferred from the optical fingerprint sensor 100 to the host electronic device 101 , 200 via the communication interface 324.
  • the size of the buffer memory 326 can be reduced. For example, to less than 10 bits/pixel, preferably to 8 bits/pixel or less. Further, since fewer bits are needed, the data transfer time to the host device is also reduced.
  • a respective compensation value is added or subtracted to/from each of the pixel values of the pixel array.
  • all the pixel values of the image are subject to a respective compensation value.
  • the compensation values are calculated on-chip of the fingerprint sensor, from a predefined formula comprising a set of coefficients.
  • a control unit e.g., part of the read-out circuitry 322 or separate from the read-out circuitry 322 has access to a predefined formula that is can use, with the pixel values as input, to calculate the compensated pixel values, on-chip.
  • the predefined formula comprises a set of coefficients that are determined constructed in a calibration step and stored on a non-volatile calibration memory of the host electronic device 101 , 200.
  • the electronic device 101 , 200 comprises a device controller configured to write the coefficients to a register memory 325 of the optical fingerprint sensor.
  • the coefficients have been pre-stored on a register memory 325, of the optical fingerprint sensor and are retrieved for calculating the compensation values on-chip of the optical fingerprint sensor.
  • the register memory is accessible to the read-out circuitry 322 for calculating the compensation values on-chip of the optical fingerprint sensor.
  • the coefficients are determined from calibration images acquired during a prior calibration imaging setting. Since different targets look similar, such calibration is possible and is applicable subsequently acquired fingerprint images.
  • the coefficients are written to the register memory 325 of the optical fingerprint sensor. The starting of the fingerprint sensor is intended to be the initial starting at the first use of the fingerprint sensor.
  • An example predefined formula in the form of a two-dimensional polynomial may be:
  • Z(x,y) a(x - x 0 ) 2 + b(y - y 0 ) 2 + c (x - x 0 )(y - y 0 ) + d (x - x 0 ) + e (y - y 0 ) + f
  • a DC level may be determined from an average value of a set of pixels values from center pixels of the pixel array, wherein the DC level is added to the compensation values.
  • a group of center pixels may be tailored in the prior calibration but is generally used for finding the overall DC offset in the acquired image.
  • the coefficient f in the above formula represents the DC offset.
  • the DC-offset may be used for shifting the graph shown in fig. 4A to be centered at zero intensity.
  • the compensated image preferably has global intensity variation that is less 50% of the global intensity variation of the acquired image.
  • the compensation for the global intensity variation can be performed to a degree that the fingerprint modulation should be detectable with some margin for environmental light such as sunlight.
  • the ADC of the read-out circuitry 322 reads the pixel values from the array of pixels 307. Subsequently, the compensation values are applied by processing means of the of the read-out circuitry 322 such that the image is flattened out, i.e. , the global intensity variation is reduced. Subsequently, the compensated image is stored in the buffer memory 326 before it is transferred to the host device 101 , 200 on the transfer bus 324.
  • the frame buffer 326 has fewer bits than the bit-depth of the ADC of the read-out circuitry 322, and the number of bits sent on the bus 324 is fewer than the bits read out by the ADC.
  • One way of calculating the compensation values is by using a predefined formula as described above. Another possible way is to calculate the compensation values from an interpolation between the pixel values.
  • a control unit, on the chip of the fingerprint sensor may be configured to perform the steps of the methods disclosed herein.
  • a control unit configured to control the optical fingerprint sensor to acquire an image of an object having a global intensity variation.
  • the control unit is configured to add or subtract a set of compensation values to/from pixel values of the acquired image to produce a compensated image, the set of compensation values being adapted to reduce the global intensity variation of the acquired image.
  • the control unit is configured to provide the compensated image from the optical fingerprint sensor to the host electronic device.
  • a control unit may include a microprocessor, microcontroller, programmable digital signal processor or another programmable device.
  • the control unit may also, or instead, include an application specific integrated circuit, a programmable gate array or programmable array logic, a programmable logic device, or a digital signal processor.
  • the processor may further include computer executable code that controls operation of the programmable device. It should be understood that all or some parts of the functionality provided by means of the control unit (or generally discussed as “processing circuitry”) may be at least partly integrated with the optical fingerprint sensor.

Landscapes

  • Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Input (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

The present invention generally relates to a method for image shading compensation of images acquired by an optical fingerprint sensor (100) mountable under an at least partly transparent display panel (301) of a host electronic device (101, 200), the optical fingerprint sensor comprising an image sensor (303) comprising photodetector pixel array (307), the method comprising the steps: acquiring (S102), with the optical fingerprint sensor, an image having a global intensity variation; on-chip of the optical fingerprint sensor, adding (S104) or subtracting a set of compensation values to/from pixel values of the acquired image to produce a compensated image, the set of compensation values being adapted to reduce the global intensity variation of the acquired image; providing (S106) the compensated image from the optical fingerprint sensor to the host electronic device.

Description

SHADING COMPENSATED READOUT
Technical Field
The present invention relates to method for image shading compensation of images acquired by an optical fingerprint sensor. The invention also relates to an optical fingerprint sensor, to an electronic device comprising an optical fingerprint sensor, and to a control unit configured to execute steps according to the method.
Background
Biometric systems are widely used as means for increasing the convenience and security of personal electronic devices, such as mobile phones etc. Fingerprint sensing systems, in particular, are now included in a large proportion of all newly released consumer electronic devices, such as mobile phones.
Optical fingerprint sensors have been known for some time and may be a feasible alternative to e.g., capacitive fingerprint sensors in certain applications. Optical fingerprint sensors may for example be based on the pinhole imaging principle and/or may employ micro-channels, i.e. , collimators or microlenses or camera-type lenses to focus incoming light onto an image sensor.
It has recently been of interest to arrange optical fingerprint sensors under the display of electronic devices. In such arrangements, the display includes holes which allows transmission of light to the sensor. However, the display causes a shading effect in the image on the sensor.
The fingerprint modulation is only a relatively small fraction of the total signal detected by the sensor. Because of the shading one needs a large dynamic range, i.e., bit depth, to extract the fingerprint modulation at different signal levels. Consequently, a relatively large memory buffer is needed for enabling the large bit depth. If one could reduce the number of bits for each pixel the buffer size and the readout time from the sensor chip, through SPI or similar interface, can be reduced. Accordingly, there is a room for improvements with regards to the buffer size of the fingerprint sensor and reducing the readout time from the sensor chip.
Summary
In view of above-mentioned and other drawbacks of the prior art, it is an object of the present invention to provide a method for image shading compensation of images acquired by an optical fingerprint sensor mountable under an at least partly transparent display panel that at least alleviates some drawbacks of prior art.
According to a first aspect of the invention, there is provided a method for image shading compensation of images acquired by an optical fingerprint sensor mountable under an at least partly transparent display panel of a host electronic device, the optical fingerprint sensor comprising an image sensor comprising photodetector pixel array.
The method comprises the following steps. Acquiring, with the optical fingerprint sensor, an image having a global intensity variation. On-chip of the optical fingerprint sensor, adding or subtracting a set of compensation values to/from pixel values of the acquired image to produce a compensated image, the set of compensation values being adapted to reduce the global intensity variation of the acquired image. Providing the compensated image from the optical fingerprint sensor to the host electronic device.
By means of the invention, the global intensity profile of the acquired image is flattened out by the addition or subtraction of the compensation values to/from the pixel values. By flattening out the global intensity profile, the global variation in intensity across the image is reduced which at the same time reduces the required bit depth for the on-chip memory.
Adding or subtracting a set of compensation values is performed at read-out of the pixel array, on chip of the optical fingerprint sensor.
A global intensity variation is meant as an intensity variation caused by shading, also known as vignetting or lens roll-off, and that affects a variation of intensity over a large distance compare to the dimension of the fingerprint modulation. Often, a global intensity variation reaches across the entire image. Image shading is an effect caused by components of the entire optical system including e.g., cover glass, displays, filters, etc.
The compensated image is stored in an on-chip memory storage with fewer bits per pixels than a bit depth of an analogue to digital converter used for reading out the pixel values. The analogue to digital converter is on chip with the pixel array.
The present invention is at least partly based on the realization that by performing additive or subtractive compensation on-chip to reduce the global intensity variation, the required bit depth for each pixel is reduced which also leads to reduced readout time from the image sensor. A reduced bit depth allows for using a smaller on-chip buffer memory, thereby also reduced on- chip footprint for the buffer memory.
The compensation is additive/subtractive to not exaggerate intensity variations where the shading compensation is large, which may happen close to the edges of the image sensor, compared to a multiplicative shading compensation which is a common step in digital image processing.
The inventors further realized that the additive or a subtractive compensation is possible since, in fingerprint imaging, the shading profile, from different targets is very similar. Thus, the target, being a fingerprint, is known beforehand unlike a regular camera where the targets are much different between images. Thus, parameters used for the shading compensation can be known prior to use of the optical fingerprint sensor.
According to embodiments, a respective compensation value may be added or subtracted to/from each of the pixel values of the pixel array. Thus, the shading compensation is applied across the entire image, to each pixel. This improves the accuracy of the shading compensation and the quality of the resulting compensated image, as well as ensures that the bit depth is reduced for all pixels.
According to embodiments, the method may comprise calculating the compensation values, on chip, from a predefined formula comprising a set of coefficients. In other words, the compensation values themselves are calculated on-chip.
Preferably, the coefficients may be stored on a register memory of the optical fingerprint sensor and are retrieved for calculating the compensation values on-chip. The formula may be in the form of a parameterized formula where the coefficients of the formula may be written to the sensor memory. At readout of the pixels from the pixel array the compensation values are applied to every pixel according to the parametrized formula.
In embodiments, the coefficients are written to a register memory of the optical fingerprint sensor upon starting the optical fingerprint sensor.
The coefficients may be constructed in a calibration step and stored on a memory of the host electronic device. Thus, upon starting the optical fingerprint sensor, the coefficients are written to the register memory of the optical fingerprint sensor.
In embodiments, the method may comprise calculating the compensation values from an interpolation between the pixel values. Thus, as an alternative to using the formula, an interpolation may be used.
The compensated image may have a global intensity variation that is less 50% of the global intensity variation of the acquired image. This leads to that the bit depth is reduced by at least one bit. Further, truncation of the pixel values may be used, where pixel values exceeding an upper limit value or are lower than a lower limit value are rounded off to a value within the upper and lower limits. The reduction of the global intensity variation is performed to a degree so that the fingerprint modulation is not flattened out, thus the reduction is limited to a threshold where the fingerprint modulation is still detectable, including a margin for e.g., sun light or other sources that may cause stray light that may reach the image sensor, often from the side edges of the sensor. Accordingly, the bit depth of the compensated image should be sufficient to be able to detect the fingerprint modulation. The bit depth could be determined based on an expected bit depth needed to detect the fingerprint modulation. The compensation is performed such that the expected bit depth is used at read out. According to embodiments, the method may comprise determining a DC level, or offset level, from an average value of a set of pixels values from center pixels of the pixel array, wherein the DC level of offset is added to the compensation values. In other words, to set the correct DC-level or offset in the compensated image, an average value of the centre pixels is determined in a calibration step. The centre position, where the maximum intensity is, can be stored in the calibration step and written to the sensor upon start. The DC level or offset level is the average value of the center pixels.
According to embodiments, fewer bits may be needed for extracting fingerprint modulation in the compensated image compared to the number of bits needed for extracting fingerprint modulation from the acquired image. For example, if an ADC with a given number of bits per pixel is used for reading out the acquired image, that image is subject to the compensation, whereby the compensated image is read and sent to the host. The number of bits sent per pixel is then less than the number of bits per pixel of the ADC. Thus, the method may comprise extracting fingerprint modulation in the compensated image with fewer number of bits compared to the number of bits required for extracting fingerprint modulation from the acquired image.
According to a second aspect of the invention, there is provided an optical fingerprint sensor configured to be arranged under an at least partly transparent display panel of a host electronic device, the optical fingerprint sensor comprising: an image sensor comprising photodetector pixel array configured to acquire an image of an object on the opposite side of the at least partly transparent display panel, the image having a global intensity variation; and, the optical fingerprint sensor comprising means configured to: add or subtract a set of compensation values to/from pixel values of the acquired image to produce a compensated image, the set of compensation values being adapted to reduce the global intensity variation of the acquired image, and to provide the compensated image from the optical fingerprint sensor to the host electronic device.
The outer surface of a display panel under which the optical fingerprint sensor is arranged may also be referred to as a sensing surface. The operating principle of the described optical fingerprint sensor may be that light emitted by pixels in the display panel will be reflected by a finger placed on the sensing surface, and the reflected light is received by the light redirecting elements and subsequently redirected onto a corresponding subarray of pixels or a single pixel in the photodetector pixel array. In case of a subarray, an image of a portion of a finger can be captured for each subarray. By combining the images from all the light redirecting elements, an image representing the fingerprint can be formed and subsequent biometric verification can be performed.
The transparent display panel may comprise a color controllable light source. Various types of displays can be used in accordance with embodiments. For example, display panels based on OLED, u-LED with any type of tri-stimulus emission like RGB, CMY or others. Thereby, in-display biometric imaging is enabled.
In embodiments, the optical fingerprint sensor may comprise means configured to calculate the compensation values, on chip, from a predefined formula comprising a set of coefficients.
In embodiments, the optical fingerprint sensor may comprise an on- chip register memory onto which, upon starting the optical fingerprint sensor, the predefined set of compensation values are written.
Further effects and features of the second aspect of the invention are largely analogous to those described above in connection with the first aspect of the invention.
According to a third aspect of the invention, there is provided an electronic device comprising: an at least partly transparent display panel; the optical fingerprint sensor according to any one of the herein mentioned or derivable embodiments, and processing circuitry configured to: receive a signal from the optical fingerprint sensor indicative of a fingerprint of a finger touching the at least partly transparent display panel, perform a fingerprint authentication procedure based on information comprised in the signal.
In embodiments, the electronic device may comprise a non-volatile calibration memory for storing the coefficients of the formula, the electronic device comprising a device controller configured to write the coefficients to a register memory of the optical fingerprint sensor.
The electronic device may be e.g., a mobile device such as a mobile phone (e.g., Smart Phone), a tablet, a phablet, etc.
Further effects and features of the third aspect of the invention are largely analogous to those described above in connection with the first aspect and the second aspect of the invention.
There is further provided a control unit configured to perform the steps of any one of the herein mentioned methods.
Further features of, and advantages with, the present invention will become apparent when studying the appended claims and the following description. The skilled person realize that different features of the present invention may be combined to create embodiments other than those described in the following, without departing from the scope of the present invention.
Brief Description of the Drawings
These and other aspects of the present invention will now be described in more detail, with reference to the appended drawings showing an example embodiment of the invention, wherein:
Fig. 1 schematically illustrates an example of an electronic device according to embodiments of the invention;
Fig. 2 is a schematic box diagram of an electronic device according to embodiments of the invention;
Fig. 3 schematically illustrates an optical fingerprint sensor according to an embodiment of the invention;
Fig. 4A conceptually illustrates a global intensity variation across an image;
Fig. 4B conceptually illustrates a global intensity variation across an image on which the compensation values discussed herein has been applied;
Fig. 5 is a block diagram of an optical fingerprint sensor and a host electronic device according to embodiments of the invention; Fig. 6 is a flow-chart of method steps according to embodiments of the invention; and
Fig. 7 is a flow-chart of method steps according to embodiments of the invention.
Detailed Description of Example Embodiments
In the present detailed description, various embodiments of the optical fingerprint sensor according to the present invention are mainly described with reference to an optical fingerprint sensor arranged under a display panel. However, it should be noted that the described imaging arrangement also may be used in other optical biometric imaging applications such as in an optical biometric arrangement located under a cover glass or the like.
Turning now to the drawings and in particular to Fig. 1 , there is schematically illustrated an example of an electronic device configured to apply the concept according to the present disclosure, in the form of a mobile device 101 with an integrated in-display optical fingerprint sensor 100 and a display panel 104 with a touch screen interface 106. The optical fingerprint sensor 100 may, for example, be used for unlocking the mobile device 101 and/or for authorizing transactions carried out using the mobile device 101 , etc.
The optical fingerprint sensor 100 is here shown to be smaller than the display panel 104, but still relatively large, e.g., a large area implementation. In another advantageous implementation the optical fingerprint sensor 100 may be the same size as the display panel 104, i.e. , a full display solution. Thus, in such case the user may place his/her finger anywhere on the display panel for biometric authentication. The optical fingerprint sensor 100 may in other possible implementations be smaller than the depicted optical fingerprint sensor, such as providing a hot-zone implementation.
Preferably and as is apparent for the skilled person, the mobile device 101 shown in Fig. 1 further comprises a first antenna for WLAN/Wi-Fi communication, a second antenna for telecommunication communication, a microphone, a speaker, and a phone control unit. Further hardware elements are of course possibly comprised with the mobile device.
It should furthermore be noted that the invention may be applicable in relation to any other type of electronic devices comprising transparent display panels, such as a laptop, a tablet computer, etc.
Fig. 2 is a schematic box diagram of an electronic device according to embodiments of the invention. The electronic device 200 comprises a transparent display panel 204 and an optical fingerprint sensor 100 conceptually illustrated to be arranged under the transparent display panel 204 according to embodiments of the invention. Furthermore, the electronic device 200 comprises processing circuitry such as control unit 202. The control unit 202 may be stand-alone control unit of the electronic device 200, e.g., a device controller. Alternatively, the control unit 202 may be comprised in the optical fingerprint sensor 100.
The control unit 202 is configured to receive a signal indicative of a detected object from the optical fingerprint sensor 100. Preferably, the signal is indicative of a fingerprint of a finger touching the at least partly transparent display panel. The received signal may comprise image data.
Based on the received signal the control unit 202 is configured to detect a fingerprint, and based on the detected fingerprint, the control unit 202 is configured to perform a fingerprint authentication procedure. Such fingerprint authentication procedures are considered perse known to the skilled person and will not be described further herein.
Fig. 3 schematically illustrates an optical fingerprint sensor 100 according to an embodiment of the invention. The optical fingerprint sensor 100 is here arranged under an at least partially transparent display panel 301 . However, the optical fingerprint sensor 100 may be arranged under any cover structure which is sufficiently transparent, as long as the image sensor 303 receives a sufficient amount of light to capture an image of a biometric object in contact with the outer surface of the cover structure, such as a fingerprint or a palmprint. In the following, an optical fingerprint sensor 100 configured to capture an image of a finger 304 in contact with an outer surface 306 of a cover glass 302 of the display panel 301 is described.
The optical fingerprint sensor 100 comprises the image sensor 303 including the photodetector pixel array 307, where each pixel 312 is an individually controllable photodetector configured to detect an amount of incoming light and to generate an electric signal indicative of the light received by the detector. The image sensor 303 may be any suitable type of image sensor, such as a CMOS or CCD or TFT based sensor connected to associated control circuitry. A thin-film transistor (TFT) based image sensor provides a cost-efficient solution. The operation and control of such an image sensor can be assumed to be known and will not be discussed herein.
In some embodiments, the optical fingerprint sensor 100 further comprises an optical stack 314 arranged to cover the image sensor 104. The optical stack 314 may include various layers and components such as a transparent substrate covering the image sensor 303, a set of optical redirection elements such as at least one lens 316, opaque layers having of separate openings for the lenses, an adhesive layer to attach the display panel 301 to the optical fingerprint sensor 100, air gaps, optical filters such as color filter and IR-cut off filters, and antireflection coatings, depending on the specific implementation.
Moreover, for completeness, the at least partly transparent display panel 301 here comprises a color controllable light source 318 comprising individually controllable light emitting pixels 320. For acquiring an image of e.g., a fingerprint or palmprint, the color controllable light source 318 may emit light that is reflected by the finger 304 and detected by the pixels 312 of the image sensor 303. There are suitable openings or optical paths past the color controllable light source 318 so that the light beams being transmitted from the finger 304 can reach the image sensor 303.
Fig. 4A conceptually illustrates a global intensity variation across an image acquired by an optical fingerprint sensor arranged under an at least partly transparent display panel. The x and y axes represent pixels and the z axis represent intensity in arbitrary units. In fig. 4A, the typical lens roll-off effect or shading effect is seen as the intensity decrease towards the edges of the image. The intensity variation causes a relatively large signal depth, from a maximum intensity to a minimum intensity, which necessitates a large dynamic range to be able to resolve the fingerprint modulation that lies superpositioned with the global intensity variation. This further requires a relatively large number of bits in the buffer memory on the chip of the fingerprint sensor. The upper boundary 402 and lower boundary 404 conceptually illustrates the bit depth b1 required for the global intensity variation shown in fig. 4A. Thus, in order to resolve the fingerprint modulation in the global intensity variation shown in fig. 4A, the bit depth b1 is required which also includes a margin for e.g., stray light.
Fig. 4B conceptually illustrates a global intensity variation across an image on which the compensation values discussed herein has been applied. Compensation values has been added to, or subtracted from, the pixel values of the image shown in fig. 4A to produce the compensated image. The global intensity variation, being defined as the difference between a maximum intensity and a minimum intensity is significantly smaller in the image shown in fig. 4B compared to that in fig. 4A. This compensation leads to a reduced need for bit depth for detecting the fingerprint modulation and thereby also to that the buffer memory on chip of the fingerprint sensor can be reduced. Conceptually, the upper boundary 406 and lower boundary 408 illustrates the bit depth b2 required for the global intensity variation shown in fig. 4B. Accordingly, the indicated bit depth b2 is smaller than the bit depth b1 conceptually indicated in fig. 4A.
Fig. 5 is a block diagram for schematically describing an optical fingerprint sensor according to embodiments of the invention.
The optical fingerprint sensor 100 comprises a read-out circuitry 322 controllable for converting analog sensing signals to digital signals. The analog sensing signals being indicative of an image acquired by the image sensor 303 comprising the array 307 of photodetectors 312 of which not all are numbered. The optical fingerprint sensor may be included in the host device 101 , 200 which may be e.g., the electronic device 200 or the mobile device 101.
A data transfer bus 324 is configured to transfer image data to the host device 101 , 200 and may be a serial peripheral interface (SPI) or an I3C interface although other types of data transfer busses configured to transfer data are conceivable. In case of an SPI, the optical fingerprint sensor comprises a communication interface 328 having an SPI port including a serial clock input (SCLK); a master output slave input (MOSI), a master input slave output (MISO), and a slave select input (CS).
During image acquisition, after exposure of the photodetectors, the sensing signal therefrom are read-out and digitalized by the read-out circuitry 322 and subsequently they are transferred, in digital form, to the host device 101 , 200 on the data transfer bus 324.
The read-out block 322 includes electrical components known perse, for transferring charges from the photodiodes of the image sensor pixel matrix to a voltage, e.g., an analog sensing signal, and for converting the analog signals to a digital signal, i.e. , analog to digital converters.
A memory storage 326 such as a frame buffer, or alternatively a first- in-first-out (FIFO) type memory if a frame buffer is not present or available may be included in the optical fingerprint sensor 100 and be configured to store data indicative of the acquired sensing signals. The size of this buffer depends on the required bit depth for extracting fingerprint modulation from an acquired image. If the global intensity variation discussed with reference to figs. 4A-B can be reduced, then the size of this buffer 326 can also be reduced, and so can the read-out time via the data transfer bus 324.
Fig. 6 is a flow-chart of method steps according to embodiments of the invention. In step S102 acquiring, with an optical fingerprint sensor 100, an image having a global intensity variation. The optical fingerprint sensor 100 being mountable under an at least partly transparent display panel 301 of a host electronic device 101 , 200 and is configured to acquire an image of an object on the opposite side of the at least partly transparent display panel. The optical fingerprint sensor comprising an image sensor comprising photodetector pixel array.
In step S104, performed on-chip of the optical fingerprint sensor, adding or subtracting a set of compensation values to/from pixel values of the acquired image to produce a compensated image. As discussed in figs 4A-B, the set of compensation values being adapted to reduce the global intensity variation of the acquired image. The optical fingerprint sensor 100 comprises on-chip means, for example a control unit being part of the read-out circuitry 322, that is configured to perform the addition or subtraction of the compensation values.
In step S106, providing the compensated image from the optical fingerprint sensor 100 to the host electronic device 101 , 200. The compensated image may be transferred from the optical fingerprint sensor 100 to the host electronic device 101 , 200 via the communication interface 324.
By means of the embodiments of the present invention, the size of the buffer memory 326 can be reduced. For example, to less than 10 bits/pixel, preferably to 8 bits/pixel or less. Further, since fewer bits are needed, the data transfer time to the host device is also reduced.
Preferably, a respective compensation value is added or subtracted to/from each of the pixel values of the pixel array. Thus, all the pixel values of the image are subject to a respective compensation value.
Turning to the flow-chart in fig. 7, there is included a step S 103 of calculating the compensation values, on chip, from a predefined formula comprising a set of coefficients.
Accordingly, the compensation values are calculated on-chip of the fingerprint sensor, from a predefined formula comprising a set of coefficients. In other words, a control unit, e.g., part of the read-out circuitry 322 or separate from the read-out circuitry 322, has access to a predefined formula that is can use, with the pixel values as input, to calculate the compensated pixel values, on-chip. The predefined formula comprises a set of coefficients that are determined constructed in a calibration step and stored on a non-volatile calibration memory of the host electronic device 101 , 200. The electronic device 101 , 200 comprises a device controller configured to write the coefficients to a register memory 325 of the optical fingerprint sensor. In other words, during use of the optical fingerprint sensor, the coefficients have been pre-stored on a register memory 325, of the optical fingerprint sensor and are retrieved for calculating the compensation values on-chip of the optical fingerprint sensor. The register memory is accessible to the read-out circuitry 322 for calculating the compensation values on-chip of the optical fingerprint sensor.
The coefficients are determined from calibration images acquired during a prior calibration imaging setting. Since different targets look similar, such calibration is possible and is applicable subsequently acquired fingerprint images. Upon starting the optical fingerprint sensor, the coefficients are written to the register memory 325 of the optical fingerprint sensor. The starting of the fingerprint sensor is intended to be the initial starting at the first use of the fingerprint sensor.
An example predefined formula in the form of a two-dimensional polynomial may be:
Z(x,y) = a(x - x0)2 + b(y - y0)2 + c (x - x0)(y - y0) + d (x - x0) + e (y - y0) + f
This formula is added or subtracted, depending on the values of the coefficients a, b, c, d, e and fto each pixel for pixel positions x and y. To take a DC-offset into account, a DC level may be determined from an average value of a set of pixels values from center pixels of the pixel array, wherein the DC level is added to the compensation values. A group of center pixels may be tailored in the prior calibration but is generally used for finding the overall DC offset in the acquired image. The coefficient f in the above formula represents the DC offset. The DC-offset may be used for shifting the graph shown in fig. 4A to be centered at zero intensity. The compensated image preferably has global intensity variation that is less 50% of the global intensity variation of the acquired image. This reduces the bit depth by at least one bit. Further, it may be advantageous to truncate pixel values exceeding an upper threshold, and pixel values being below a lower threshold to further reduce the global intensity variation. The compensation for the global intensity variation can be performed to a degree that the fingerprint modulation should be detectable with some margin for environmental light such as sunlight.
If one compared the compensated image with the acquired raw image, fewer bits are needed for extracting fingerprint modulation in the compensated image compared to the number of bits needed for extracting fingerprint modulation from the acquired image.
Further, and now with reference to fig. 5, the ADC of the read-out circuitry 322 reads the pixel values from the array of pixels 307. Subsequently, the compensation values are applied by processing means of the of the read-out circuitry 322 such that the image is flattened out, i.e. , the global intensity variation is reduced. Subsequently, the compensated image is stored in the buffer memory 326 before it is transferred to the host device 101 , 200 on the transfer bus 324. In other words, the frame buffer 326 has fewer bits than the bit-depth of the ADC of the read-out circuitry 322, and the number of bits sent on the bus 324 is fewer than the bits read out by the ADC.
One way of calculating the compensation values is by using a predefined formula as described above. Another possible way is to calculate the compensation values from an interpolation between the pixel values.
A control unit, on the chip of the fingerprint sensor may be configured to perform the steps of the methods disclosed herein. For example, there is provided a control unit configured to control the optical fingerprint sensor to acquire an image of an object having a global intensity variation. On-chip of the optical fingerprint sensor, the control unit is configured to add or subtract a set of compensation values to/from pixel values of the acquired image to produce a compensated image, the set of compensation values being adapted to reduce the global intensity variation of the acquired image. Further, the control unit is configured to provide the compensated image from the optical fingerprint sensor to the host electronic device.
A control unit may include a microprocessor, microcontroller, programmable digital signal processor or another programmable device. The control unit may also, or instead, include an application specific integrated circuit, a programmable gate array or programmable array logic, a programmable logic device, or a digital signal processor. Where the control unit includes a programmable device such as the microprocessor, microcontroller or programmable digital signal processor mentioned above, the processor may further include computer executable code that controls operation of the programmable device. It should be understood that all or some parts of the functionality provided by means of the control unit (or generally discussed as “processing circuitry”) may be at least partly integrated with the optical fingerprint sensor.
Even though the invention has been described with reference to specific exemplifying embodiments thereof, many different alterations, modifications and the like will become apparent for those skilled in the art. Also, it should be noted that parts of the imaging device and method for manufacturing the imaging device may be omitted, interchanged or arranged in various ways, the imaging device yet being able to perform the functionality of the present invention.
Additionally, variations to the disclosed embodiments can be understood and effected by the skilled person in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

1 . A method for image shading compensation of images acquired by an optical fingerprint sensor (100) mountable under an at least partly transparent display panel (301 ) of a host electronic device (101 , 200), the optical fingerprint sensor comprising an image sensor (303) comprising photodetector pixel array (307), the method comprising the steps: acquiring (S102), with the optical fingerprint sensor, an image having a global intensity variation; on-chip of the optical fingerprint sensor and at read out of pixels values from the pixel array, adding (S104) or subtracting a set of compensation values to/from the pixel values of the acquired image to produce a compensated image that is stored in an on-chip memory storage (326) with fewer bits per pixels than a bit depth of an analogue to digital converter (322) used for reading out the pixel values, the set of compensation values being adapted to reduce the global intensity variation of the acquired image; providing (S106) the compensated image from the optical fingerprint sensor to the host electronic device.
2. The method according to claim 1 , wherein a respective compensation value is added or subtracted to/from each of the pixel values of the pixel array.
3. The method according to any one of claims 1 and 2, comprising calculating (S103) the compensation values, on chip, from a predefined formula comprising a set of coefficients.
4. The method according to claim 3, wherein the coefficients being stored on a register memory (325) of the optical fingerprint sensor and are retrieved for calculating the compensation values on-chip.
5. The method according to claim 4, comprising, upon starting the optical fingerprint sensor, writing the coefficients to a register memory of the optical fingerprint sensor.
6. The method according to any one of claims 3 to 5, wherein the coefficients are constructed in a calibration step and stored on a memory of the host electronic device.
7. The method according to any one of claims 1 and 2, comprising calculating the compensation values from an interpolation between the pixel values.
8. The method according to any one of the preceding claims, wherein the compensated image has global intensity variation that less 50% of the global intensity variation of the acquired image.
9. The method according to any one of the preceding claims, comprising determining a offset level from an average value of a set of pixels values from center pixels of the pixel array, wherein the offset level is added to the compensation values.
10. The method according to any one of the preceding claims, comprising extracting fingerprint modulation in the compensated image with fewer number of bits compared to the number of bits required for extracting fingerprint modulation from the acquired image.
11 . An optical fingerprint sensor (100) configured to be arranged under an at least partly transparent display panel (301 ) of a host electronic device (101 , 200), the optical fingerprint sensor comprising: an image sensor (303) comprising a photodetector pixel array (307) configured to acquire an image of an object (304) on the opposite side 19 of the at least partly transparent display panel, the image having a global intensity variation; and the optical fingerprint sensor comprising means configured to: add or subtract a set of compensation values to/from pixel values of the acquired image to produce a compensated image, the set of compensation values being adapted to reduce the global intensity variation of the acquired image, and to provide the compensated image from the optical fingerprint sensor to the host electronic device.
12. The optical fingerprint sensor according to claim 11 , comprising means configured to calculate the compensation values, on chip, from a predefined formula comprising a set of coefficients.
13. The optical fingerprint sensor according to claim 12, comprising an on-chip register memory onto which, upon starting the optical fingerprint sensor, set of coefficients are written.
14. An electronic device (101 , 200) comprising: an at least partly transparent display panel (301 ); the optical fingerprint sensor according to any one of claims 11 to 13, and processing circuitry (202) configured to: receive a signal from the optical fingerprint sensor indicative of a fingerprint of a finger touching the at least partly transparent display panel, perform a fingerprint authentication procedure based on information comprised in the signal.
15. The electronic device according to claim 14, comprising a non-volatile calibration memory for storing the coefficients of the formula, the electronic device comprising a device controller configured to write the coefficients to a register memory of the optical fingerprint sensor. 20
16. The electronic device according to any one of claims 14 and 15, wherein the electronic device is a mobile device.
17. A control unit (202) configured to perform the steps of any one of claims 1 to 10.
PCT/SE2022/050897 2021-10-15 2022-10-06 Shading compensated readout WO2023063862A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
SE2151265-2 2021-10-15
SE2151265 2021-10-15

Publications (1)

Publication Number Publication Date
WO2023063862A1 true WO2023063862A1 (en) 2023-04-20

Family

ID=85987606

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/SE2022/050897 WO2023063862A1 (en) 2021-10-15 2022-10-06 Shading compensated readout

Country Status (1)

Country Link
WO (1) WO2023063862A1 (en)

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150022670A1 (en) * 2013-07-22 2015-01-22 Apple Inc. Noise Compensation in a Biometric Sensing Device
US20150071502A1 (en) * 2013-09-08 2015-03-12 Apple Inc. Noise Reduction in Biometric Images
US20170068837A1 (en) * 2014-10-08 2017-03-09 Shenzhen Huiding Technology Co., Ltd. Active baseline signal cancellation in fingerprint sensors
US20180137337A1 (en) * 2017-10-09 2018-05-17 Shanghai Tianma Micro-electronics Co., Ltd. Display device, fingerprint recognition calibration method for display device and electronic apparatus
US20190377929A1 (en) * 2017-02-17 2019-12-12 Fingerprint Cards Ab Cancelling out impairment data in fingerprint images
WO2020113411A1 (en) * 2018-12-04 2020-06-11 深圳市汇顶科技股份有限公司 Fingerprint signal processing circuit, electronic device and fingerprint signal processing method
US20200285829A1 (en) * 2019-03-07 2020-09-10 Novatek Microelectronics Corp. Fingerprint signal processing circuit and method
US20200387688A1 (en) * 2019-06-05 2020-12-10 Samsung Electronics Co., Ltd. Method of setting light sources in display panel for optical fingerprint recognition and method of performing optical fingerprint recognition using the same
WO2021002791A1 (en) * 2019-07-03 2021-01-07 Fingerprint Cards Ab Sensor displacement compensation in an optical biometric imaging arrangment under a display
US20210097255A1 (en) * 2019-09-27 2021-04-01 Samsung Display Co., Ltd. Display apparatus and a method of compensating fingerprint sensing data using the same
WO2021201745A1 (en) * 2020-04-01 2021-10-07 Fingerprint Cards Ab Tft-based fingerprint sensing system with calibration circuitry

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150022670A1 (en) * 2013-07-22 2015-01-22 Apple Inc. Noise Compensation in a Biometric Sensing Device
US20150071502A1 (en) * 2013-09-08 2015-03-12 Apple Inc. Noise Reduction in Biometric Images
US20170068837A1 (en) * 2014-10-08 2017-03-09 Shenzhen Huiding Technology Co., Ltd. Active baseline signal cancellation in fingerprint sensors
US20190377929A1 (en) * 2017-02-17 2019-12-12 Fingerprint Cards Ab Cancelling out impairment data in fingerprint images
US20180137337A1 (en) * 2017-10-09 2018-05-17 Shanghai Tianma Micro-electronics Co., Ltd. Display device, fingerprint recognition calibration method for display device and electronic apparatus
WO2020113411A1 (en) * 2018-12-04 2020-06-11 深圳市汇顶科技股份有限公司 Fingerprint signal processing circuit, electronic device and fingerprint signal processing method
US20200285829A1 (en) * 2019-03-07 2020-09-10 Novatek Microelectronics Corp. Fingerprint signal processing circuit and method
US20200387688A1 (en) * 2019-06-05 2020-12-10 Samsung Electronics Co., Ltd. Method of setting light sources in display panel for optical fingerprint recognition and method of performing optical fingerprint recognition using the same
WO2021002791A1 (en) * 2019-07-03 2021-01-07 Fingerprint Cards Ab Sensor displacement compensation in an optical biometric imaging arrangment under a display
US20210097255A1 (en) * 2019-09-27 2021-04-01 Samsung Display Co., Ltd. Display apparatus and a method of compensating fingerprint sensing data using the same
WO2021201745A1 (en) * 2020-04-01 2021-10-07 Fingerprint Cards Ab Tft-based fingerprint sensing system with calibration circuitry

Similar Documents

Publication Publication Date Title
CN107636686B (en) Fingerprint acquisition method, device, chip and terminal equipment
US9224782B2 (en) Imaging systems with reference pixels for image flare mitigation
US9344649B2 (en) Floating point image sensors with different integration times
US8854244B2 (en) Imagers with improved analog-to-digital converters
WO2017110483A1 (en) Information processing device, information processing method, and program
TWI513959B (en) Colour sensor arrangement and method for colour sensor calibration
CN107820618B (en) Sensing pixel unit and optical fingerprint sensor
US10778915B2 (en) Dual-aperture ranging system
KR20140141390A (en) Image sensor and imaging device including the same
CN211062054U (en) Biometric imaging arrangement and electronic device
US8405607B2 (en) Optical navigation device and associated methods
US10970511B2 (en) System and method for correcting distortions in a biometric image
US20170078598A1 (en) Imaging apparatus and imaging method thereof using correlated double sampling
TW201427418A (en) Sensing apparatus and sensing method
US20150122973A1 (en) Sensing pixel and image sensor including the same
JP2015002469A (en) Imaging apparatus, imaging system, method for driving imaging apparatus, and method for driving imaging system
US20230232132A1 (en) Pixel and method for operating a pixel
CN111414792A (en) Fingerprint identification integrated circuit and fingerprint identification device including the same
US20240127413A1 (en) Image processing system for performing image quality tuning and method of performing image quality tuning
KR102412278B1 (en) Camera module including filter array of complementary colors and electronic device including the camera module
US8760547B2 (en) Image converter
CN111466030A (en) Image sensor with intensity normalization
US11308309B2 (en) Fingerprint sensor having an increased sensing area
WO2023063862A1 (en) Shading compensated readout
CN115170676A (en) Sensor calibration and image correction method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22881459

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE