EP4537546A2

EP4537546A2 - Imager pixel

Info

Publication number: EP4537546A2
Application number: EP23819367.6A
Authority: EP
Inventors: Karim Ali AHMED; Longyang LIN; Praveenakumar Shivappa SALAMANI; Massimo Alioto
Original assignee: National University of Singapore
Current assignee: National University of Singapore
Priority date: 2022-06-07
Filing date: 2023-06-07
Publication date: 2025-04-16
Also published as: WO2023238068A2; CN119631422A; EP4537546A4; WO2023238068A3

Abstract

An imager pixel comprising a forward-biased photo-detector with readout transistor for light-to-current conversion, and a ratiometric readout system comprising a current to pulse width converter. The readout system operates in a light mode to: generate a first current proportional to a photodetector current and a leakage current, on exposure to a photodetector voltage; and convert the first current to a first pulse width; and operates in a least significant bit (LSB) mode to: generate a second current proportional to a least significant bit (LSB) current and the leakage current, on exposure to a gate-source voltage; and convert the second current to a second pulse width. The system includes a converter circuit for converting the first pulse width to a count (CNTB) and the second pulse width to a count (CNTA), and a counting circuit for providing a radiometric readout based on a ratio of CNTB to CNTA.

Description

IMAGER PIXEL

Technical Field

The present invention relates, in general terms, to imagers and, in particular but not limited to, pixels facilitating dynamic LSB adaption and radiometric readout.

Background

Purely energy- harvested imagers enable battery suppression for low cost, small factor and long-lived sensor nodes for distributed vision systems, where image sensing and cloud Al sensemaking allow to capture visual data and generate valuable insights. To reduce power, aggressively low precisions were explored in Al accelerators (e.g., down to 4 bits). Similarly, low imager bit depths are being explored to narrow the gap. As a drawback, low bit depths N flatten visual features of interest under fluctuating lighting conditions. If the actual dynamic range in a scene is much smaller than the full-scale range FS (dark scene), most pixels are flattened into few bins with low intensity, and classification becomes hard under the comparatively large least significant bit (LSB) = FS/2N. Conversely, small and fixed LSB under scenes with large actual dynamic range (bright scene) poses the opposite issue, again misclassifying as most pixels are instead saturated to the highest value (Fig. 1).

It would be desirable to overcome or ameliorate at least one of the abovedescribed problems, or to at least provide a useful alternative.

Summary

Described herein is a novel imager design with pW peak power for purely- harvested operation. The LSB is dynamically adapted to the light intensity of the scene for aggressive bit depth down-scaling, avoiding traditional dynamic range over-margining across practical light intensities under fixed LSB. Ratiometric readout of pixel current cancels threshold voltage mismatch. A 256x256-pixel 180-nm imager shows 5-pW power at 1 fps and 4 bits, while keeping ImageNet classification accuracy drop to percentage points under 75- dB ambient light range, across original and brightness-adjusted images.

An imager with dynamic LSB is proposed to enable low-bit depth sensing and extend the range of light conditions where the accuracy drop in image classification remains marginal from dark to bright (75-dB range, Figure 1).

Disclosed is an imager pixel comprising: a forward-biased photo-detector with readout transistor for light-to- current conversion; a ratiometric readout system comprising a current to pulse width converter, for: operating in a light mode to: generate a first current proportional to a photodetector current and a leakage current, on exposure to a photodetector voltage; and convert the first current to a first pulse width; and operating in a least significant bit (LSB) mode to: generate a second current proportional to the least significant bit (LSB) current and the leakage current, on exposure to a gate-source voltage; and convert the second current to a second pulse width; a converter circuit for converting the first pulse width to a count (CNTB) and the second pulse width to a count (CNTA); and a counting circuit for providing a radiometric readout based on a ratio of CNTB to CNTA.

Also disclosed is a system comprising : a plurality of rows, each row comprising a plurality of pixels as described above, wherein the plurality of rows are separated into a plurality of clusters; a row controller for, for a frame: sequentially operating the pixels in the clusters in LSB mode, light mode and averaging mode, and then idling the pixels until a next frame; and operating in averaging mode by averaging a pixel value across the frame to produce an average pixel value; and a capacitor for each cluster for storing a voltage corresponding to the average pixel value for the cluster.

Advantageously, a forward-biased photo-detector with sub-threshold readout transistor can be provided. This enables nearly-linear light-to-current conversion.

Advantageously, embodiments provide a ratiometric readout with dual sensing mode. This mitigates threshold voltage mismatch - e.g. across the MPD.

Advantageously, embodiments provide cluster-level pipelining. This eliminates timing overhead due to dual sensing mode switching.

Brief description of the drawings

Embodiments of the present invention will now be described, by way of nonlimiting example, with reference to the drawings in which:

Figure 1 illustrates the limitations of conventional fixed LSB imagers at low bit depth;

Figure 2 schematically illustrates the dual sensing mode 12T pixel architecture and ratiometric readout scheme;

Figure 3 shows an architecture of the proposed low bit-depth imager with dynamic LSB;

Figure 4 is a timing diagram for cluster-level pipelining;

Figure 5 shows the voltage threshold (VTH) mismatch map measured from CNTB/CNTA (uniform image, with & without ratiometric readout);

Figure 6 shows the measured LSB pulse width (PWLSB) adaptation to light intensity (>75dB);

Figure 7 is an example of under harvesting: measured 1st image misclassification occurrence vs. brightness reduction; and

Figure 8 shows the classification accuracy improvement of dynamic LSB over fixed LSB.

Detailed description

Described herein are least significant bit (LSB) imagers that enable low-bit depth sensing and extends the range of light conditions where the accuracy drop in image classification remains marginal from dark to bright. It also enables operation at pW-range power for purely- harvested systems.

With reference to Figure 2, an imager pixel 200 is provided. The pixel 200 comprises a forward-biased photo-detector 202 with sub-threshold readout transistor 204 for light-to-current conversion. The pixel 200 further comprises a ratiometric readout system 206, which comprises a current-to-pulse width converter. The imager pixel operates in light sensing mode and LSB sensing mode.

For the purpose of this discussion, "light sensing mode", "light mode" and similar refer to when the photodetector is exposed to the photodetector voltage, and "LSB sensing mode", "LSB mode" and similar refer to when the photodetector is exposed to the gate-source voltage, obtained from a mismatch-free pixel exposed to a 1-LSB light intensity.

In light sensing mode, the MPD is exposed to the photodetector voltage. The MPD generates a first current (IMPD) proportional to a photodetector current (IPD) and a leakage current (Ilkg), on exposure to the photodetector voltage. MPD then converts the first current to a first pulse width PWlight, via in-pixel comparison with a current ramp (Figure 3). PWlight is in turn converted to count CNTB based on the system clock.

In LSB sensing mode, the MPD is exposed to a gate-source voltage (VGS.LSB) . This generates a second current (IMPD) proportional to a LSB current (ILSB) and the leakage current (Ii_kg). VGS.LSB is produced by a mismatch-free (i.e. threshold voltage mismatch-free) pixel exposed to a 1-LSB light intensity, derived as proper fraction of the average light level across the frame. As above, the current IMPD is converted to a pulse width PWLSB, then to count CNTA.

The pixel 200 further comprises a converter circuit 210 which converts the first pulse width PWlight to a count (CNTB) and the second pulse width PWLSB to a count (CNTA). The pixel 200 also comprises a counting circuit for providing a radiometric readout based on a ratio of CNTB to CNTA. Hence, the ratio PWlight/ PWLSB = CNTB/CNTA is independent of leakage and, accordingly, of the threshold voltage mismatch. With reference to Fig 2, the ratio is evaluated digitally without a digital divider by using count CNTA in LSB mode as modulo of the CNTB in light mode, incrementing a third counter CNTC only every CNTA cycle, provided counter B keeps incrementing. The final value of counter C (CNTC) is hence CNTB/CNTA, enabling ratiometric readout.

Disclosed here is also a system comprising a plurality of rows, with each row comprising a plurality of pixels such as imager pixels 200 as described above (i.e. are rows of pixels), with the plurality of rows separated into a plurality of clusters. The rows may be clustered by proximity - e.g. every N neighbouring rows may form a cluster. Alternatively, pixels in the rows of pixels may be clustered together based on proximity - e.g. a pixel and its surrounding eight pixels (if the pixels are in a regular grid pattern) form a cluster - and thus any row may comprise pixels from multiple clusters.

The system also comprises a row controller which sequentially operates the pixels in the clusters in LSB mode, light mode and averaging mode, then idles the pixels until the next frame. The row controller dynamically scales the LSB based on the average pixel value, generated from V_avg. With reference to Figure 4, cluster-level pipelining masks the timing overhead of LSB to light mode switching. The row controller also operates in averaging mode by averaging a pixel value across the frame to produce an average pixel value. In cluster-level pixel averaging, photodetectors are shorted by the switch LSBSEL. Their cumulative current sets the aggregated photodetector voltage Vaster to the value of a single photodetector with a current equal to their average. Vaster is then stored in a capacitor Cciuster, whose charge is progressively redistributed with the capacitors of subsequent clusters (i.e. clusters that are used subsequently in cluster-level averaging) when accessed, averaging Vaster across clusters (Figure 3). The resulting voltage VAVG at the end of frame is the average open-circuit voltage of all photodetectors across the array. The system further comprises a capacitor for each cluster for storing a voltage corresponding to the average pixel value for the cluster.

Finally, VAVG is applied to a properly sized pixel transistor replica MPD to generate the average pixel current IAVG- The LSB adaptation loop sets ILSB= IAVG/8 in 4-bit readout. The LSB adaptation loop also generates the corresponding VGS.LSB for LSB mode. Moreover, the LSB adaptation loop may scale the ramp full-scale value to 2IAVG to keep IAVG at half the dynamic range. End-of-frame VGS.LSB update transfers VAVG from a storage capacitor to a switched-current mirror (Figure 3). 180-nm testchip measurements show that ratiometric readout reduces the mismatch-induced standard deviation of the digital output across pixels by l lx (Figure 6). The LSB current ILSB (measured by PWLSB in Figure 7) linearly tracks the scene light intensity over an ambient light range spanning >75dB (average across frame from 15 to 60,000 lux), vastly exceeding the maximum 24dB value of fixed LSB at 4-bit resolution. Using an example that employs direct harvesting (35-mm² solar cell, Figure 8), dynamic LSB capture maintains correct MobileNetV2 neural network classification under lower brightness, compared to conventional fixed LSB. Iterating on a randomly sampled ImageNet dataset, dynamic LSB enables up to 24% better classification accuracy over fixed LSB in dark scenes (Figure 9).

Table I provides a performance comparison with prior imager technologies. The proposed imager with dynamic LSB enables operation at pW-range power for purely-harvested operation, thanks to the lowest 0.08 nW/pixel power that is 3.75-47.5x lower than known pixel powers that have the lowest bit depth, and by three orders of magnitude over higher-bit depth imagers. In spite of retaining linear mapping, the >75-dB ambient light range is substantially higher than the logarithmic imager by 15.7dB, without requiring imager-specific process as in the 16x10 array at 8-10 bit (ISSCC'18).

TABLE I. Performance summary (best or unique performance is in bold)

It will be appreciated that many further modifications and permutations of various aspects of the described embodiments are possible. Accordingly, the described aspects are intended to embrace all such alterations, modifications, and variations that fall within the spirit and scope of the appended claims.

Throughout this specification and the claims which follow, unless the context requires otherwise, the word "comprise", and variations such as "comprises" and "comprising", will be understood to imply the inclusion of a stated integer or step or group of integers or steps but not the exclusion of any other integer or step or group of integers or steps.

The reference in this specification to any prior publication (or information derived from it), or to any matter which is known, is not, and should not be taken as an acknowledgment or admission or any form of suggestion that that prior publication (or information derived from it) or known matter forms part of the common general knowledge in the field of endeavour to which this specification relates.

Claims

1. An imager pixel comprising: a forward-biased photo-detector with readout transistor for light- to-current conversion; a ratiometric readout system, comprising a current-to-pulse width converter, for: operating in a light mode to: generate a first current proportional to a photodetector current and a leakage current, on exposure to a photodetector voltage; and convert the first current to a first pulse width; and operating in a least significant bit (LSB) mode to: generate a second current proportional to a least significant bit (LSB) current and the leakage current, on exposure to a gate-source voltage; and convert the second current to a second pulse width; a converter circuit for converting the first pulse width to a count (CNTB) and the second pulse width to a count (CNTA); and a counting circuit for providing a radiometric readout based on a ratio of CNTB to CNTA.

2. The pixel of 1, wherein the readout transistor is a sub-threshold readout transistor.

3. The pixel of 1 or 2, wherein the first pulse width and second pulse width are proportional pulse widths generated by in-pixel comparison with a ramp current.

4. The pixel of any one of 1 to 3, wherein the gate-source voltage corresponds to a mismatch-free pixel exposed to a 1-LSB intensity light. The pixel of 4, wherein the mismatch-free pixel exposed to a 1-LSB intensity light is derived as a proper fraction of an average light level across a frame. The pixel of any one of 1 to 5, wherein the ratio is determined by using CNTA is LSB mode as modulo of CNTB in light mode, and incrementing a counter C every CNTA cycles over which CNTB increments. A system comprising: a plurality of rows, each row comprising a plurality of pixels according to any one of 1 to 6, wherein the plurality of rows are separated into a plurality of clusters; a row controller for, for a frame: sequentially operating the pixels in the clusters in LSB mode, light mode and averaging mode, and then idling the pixels until a next frame; and operating in averaging mode by averaging a pixel value across the frame to produce an average pixel value; and a capacitor for each cluster for storing a voltage corresponding to the average pixel value for the cluster. The system of 7, wherein the row controller dynamically scales the LSB based on the average pixel value. The system of 7 or 8, wherein the row controller averages the pixel value across the frame by: performing a first averaging process by averaging a pixel value within each cluster to produce a cluster-level average pixel value, and storing a voltage corresponding to the cluster-level average pixel value in the capacitor corresponding to the respective cluster; and performing a second averaging process by progressively averaging the cluster-level average pixel values across the clusters. The system of 9, wherein the row controller shorts the photodetectors of each pixel in each cluster involved in the second averaging process. The system of 9 or 10, wherein the second averaging process involves progressively averaging the cluster-level average pixel values across the clusters by progressively redistributing the voltage from each cluster across the capacitors of subsequent clusters when progressively averaging. The system of any one of 7 to 11, wherein the row controller produces an average open-circuit voltage of all photodetectors across the array (V_avg) based on the voltages, each voltage corresponding to the average pixel value for a respective said cluster. The system of any one of 7 to 12, wherein the row controller: generates an average pixel current value from V_avg; and sets the LSB current, generates the gate-source voltage, and scale a ramp current, based on the average pixel current value. The system of 13, wherein, after processing the frame, the gate-source voltage transfer V_avg to a switched-current mirror for processing a next frame.