WO2014174498A1 - A novel cmos image sensor with event/change detection and reduced data redundancy - Google Patents

A novel cmos image sensor with event/change detection and reduced data redundancy Download PDF

Info

Publication number
WO2014174498A1
WO2014174498A1 PCT/IB2014/061057 IB2014061057W WO2014174498A1 WO 2014174498 A1 WO2014174498 A1 WO 2014174498A1 IB 2014061057 W IB2014061057 W IB 2014061057W WO 2014174498 A1 WO2014174498 A1 WO 2014174498A1
Authority
WO
WIPO (PCT)
Prior art keywords
pixel
sensor
row
reference voltage
change detection
Prior art date
Application number
PCT/IB2014/061057
Other languages
French (fr)
Inventor
Gözen KÖKLÜ
Sandro Carrara
Giovanni De Micheli
Original Assignee
Ecole Polytechnique Federale De Lausanne (Epfl)
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ecole Polytechnique Federale De Lausanne (Epfl) filed Critical Ecole Polytechnique Federale De Lausanne (Epfl)
Publication of WO2014174498A1 publication Critical patent/WO2014174498A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
    • H04N25/77—Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components
    • H04N25/772—Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components comprising A/D, V/T, V/F, I/T or I/F converters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/50—Control of the SSIS exposure
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/71—Charge-coupled device [CCD] sensors; Charge-transfer registers specially adapted for CCD sensors
    • H04N25/74—Circuitry for scanning or addressing the pixel array

Definitions

  • the present invention relates to a CMOS image sensor design to be used in CMOS imaging devices and camera systems.
  • CMOS image sensors are one of the most widely used electronic devices in today's market. There is a huge research effort to satisfy the need in this field.
  • CMOS image sensor architectures have been developed in recent years. Active Pixel Sensors (APS), which has analog read-out, and Digital Pixel Sensors (DPS), which provide pixel level digital output, are two main techniques for CMOS image sensor arrays. Most of the APS architectures have current or voltage outputs at the pixel level and the corresponding pixel value is converted to digital by means of column or chip level Analog to Digital Converters (ADC). As opposed, DPS architectures convert the pixel values to digital in pixel level and avoid the requirements of extra conversion units.
  • ADC Analog to Digital Converters
  • the DPS architectures offer several advantages over analog ones by eliminating column level Fixed Pattern Noise (FPN), reducing the number of analog read-out blocks in the design and getting rid of the demands of analog design, which results in better matching with CMOS scaling.
  • FPN Fixed Pattern Noise
  • FF Fill Factor
  • Regular DPS designs use fixed exposure time intervals for every pixel similar to the APS designs and outputs a result according to the pixel value at the end of that exposure time [1]. Most of these designs suffer from large non-photosensitive area due to the pixel level memory, which gets even worse when a higher resolution sensor is required.
  • DPS architectures also provide possibility to add extra features at the pixel level, e.g., change/movement detection, data compression [2], [3], which decreases the fill factor even more.
  • DPS designs bring many advantages with additional pixel features, however it is of great importance to keep non-photosensitive area as low as possible, while benefiting from these architectures.
  • the invention provides a Digital Pixel Sensor (DPS) based CMOS camera, the CMOS camera being configured to record frames, wherein the CMOS camera comprises an event detection sensor configured to look for an event happening in a determined frame, comprising an array of event detection pixels, a first reference voltage generator, first row and column arbiters, and first row and column address encoders; the DPS based CMOS camera further comprises a change detection sensor configured to look for a change happening in between frames, comprising an array of change detection pixels, a second binary search value generator, second row and column arbiters, and second row and column address encoders.
  • DPS Digital Pixel Sensor
  • the event detection sensor (intra-frame comparison sensors) relies on a continuous search technique inside a frame that continues to search for values of every one of the event detection pixels of the array of the event detection pixels until it finds the corresponding reference voltages for each pixel, the sensor generates reference voltages that decrements at each clock cycle of the sensor by use of the first reference voltage generator, by comparing the generated reference voltage with each event detection pixel's integrated voltage until the pixel value becomes higher than the reference voltage, the reference voltage corresponding to the pixel value is found, when the pixel voltage becomes higher than the reference voltage at a specific clock cycle and a specific reference voltage, an event flag is raised by that pixel, and the address of that pixel is registered through the use of the first row and column address encoders, when multiple of the event detection pixels generate events at the same time, the first row and column arbiters are used to give priority to one pixel at a time when registering their addresses.
  • the change detection sensor relies both on a continuous search technique inside a frame to find the value of each one of the change detection pixels of the array of change detection pixels in a frame and on a continuous comparison technique that compares each change detection pixel's value in the previous frame (or the pixel's last value with event at any frame) with its current frame value, the sensor generates reference voltages that decrements at each clock cycle of the sensor by use of a second reference voltage generator, by comparing the generated reference voltage with each change detection pixel's integrated voltage until the pixel value becomes higher than the reference voltage, the reference voltage corresponding to the pixel value is found, when the pixel voltage becomes higher than the reference voltage at a specific clock cycle and a specific reference voltage, a change flag is raised by that pixel, the address of that pixel is registered through the use of the second row and column address encoders both in the sensor and the pixel memory, the memory in the pixel is used to compare
  • the invention provides a method for data compression in a change detection sensor and reduced data redundancy comprising configuring a change detection sensor such that it looks for a change happening in between frames and generating a change flag for each change detection pixel of an array only when there is a difference between a pixel's current frame value and its previous frame value (or the pixel's last value with event at any frame), whereby the method reduces the amount of data that is required to be written in a memory (data compression) and avoids the repetition of the registration of the same data multiple times (reduced data redundancy).
  • the invention provides a method for automatic generation of frames in a change detection sensor comprising automatically generating new frames only when there is a change in one or more of the change detection pixels of an array.
  • the method further comprises a use with a defined number of changed pixel threshold by providing a threshold to the sensor that defines how many change detection pixel's value should be different than their previous frame values before automatically generating a new frame.
  • the invention provides a use of a Digital Pixel Sensor (DPS) based CMOS cameras in microscopes with a change detection feature and automatic generation of frames when there is a change in the imaged sample.
  • DPS Digital Pixel Sensor
  • the CMOS camera in the used of the DPS based CMOS camera in microscopes, is the CMOS camera according to the first aspect of the invention.
  • the invention provides a use of a Digital Pixel Sensor (DPS) based CMOS camera for biomedical applications, the CMOS camera being configured to record frames, wherein the CMOS camera comprises a change detection sensor configured to look for a change happening in between frames, comprising an array of change detection pixels, a second counter, a second digital to analogue converter, second row and column arbiters, and second row and column address encoders;
  • the change detection sensor (inter-frame comparison sensor) relies both on a continuous search technique inside a frame to find the value of each change detection pixel of an array in a frame and on a continuous comparison technique that compares each change detection pixel's value in the previous frame with its current frame value, the sensor generates reference voltages that decrements at each clock cycle of the sensor by use of a second reference voltage generator, by comparing the generated reference voltage with each change detection pixel's integrated voltage until the pixel value becomes higher than the reference voltage, the reference voltage corresponding to the pixel value is found, when the pixel
  • the invention provides a use of a Digital Pixel Sensor (DPS) based CMOS camera for microscopes, the CMOS camera being configured to record frames, wherein the CMOS camera comprises a change detection sensor configured to look for a change happening in between frames, comprising an array of change detection pixels, a second counter, a second digital to analogue converter, second row and column arbiters, and second row and column address encoders;
  • the change detection sensor (inter-frame comparison sensor) relies both on a continuous search technique inside a frame to find the value of each change detection pixel of an array in a frame and on a continuous comparison technique that compares each change detection pixel's value in the previous frame with its current frame value, the sensor generates reference voltages that decrements at each clock cycle of the sensor by use of a second reference voltage generator, by comparing the generated reference voltage with each change detection pixel's integrated voltage until the pixel value becomes higher than the reference voltage, the reference voltage corresponding to the pixel value is found when the pixel voltage becomes higher
  • figure 1 shows standard implementation of a DPS
  • figure 2 illustrates one of the possible implementation of the proposed event detection pixel by using D-Latches (14) according to a preferred embodiment of the invention
  • figure 3 illustrates another method of the pixel implementation of the proposed event detection pixel by using Static Read Access Memory— SRAM— (20) for event detection, according to a preferred embodiment of the invention
  • figure 4 illustrates another method of the pixel implementation of the proposed event detection pixel by using Dynamic Read Access Memory—DRAM— (21), according to a preferred embodiment of the invention
  • figure 5 illustrates the full image sensor with MxN array of pixels, column level counters (28) and DACs (29), column arbiter (25), row arbiter (22), column address encoder (24), row address encoder (23), according to a preferred embodiment of the event detection invention
  • figure 6 illustrates the timing diagram of the first example of the invention where only one single event is shown
  • figure 7 illustrates the timing
  • the present invention relates to Digital Pixel Sensors.
  • Various designs include array of pixels, a counter per chip, a Digital to Analog Converter—DAC— per chip and row and column arbiters and encoders.
  • Each pixel includes a reset transistor, a comparator and certain number of transistors for logic functions and row and column request and acknowledges processing.
  • This invention detects events generated in each pixel possibly implemented as in Figures 1, 2 and 3.
  • the pixel includes:
  • a voltage comparator (12) to compare the photodiode integrated voltage after certain exposure time with the ramp voltage (11), which is generated by column parallel counters (28) and DACs (27);
  • An N-bit decrement counter (28) is connected to a DAC (27) and generates the ramp voltage of the pixel (11).
  • the counter and DAC can be designed per chip only, which reduces the area.
  • the resolution of the counter (represented as N in this invention) can be easily augmented depending on the application.
  • the number of bits of the counter should be in accordance with the resolution of the comparator within the pixel.
  • the technique is based on binary search where the digital control logic (counter) decrements bits at each clock cycle. After each decrement, the input voltage (which is the integrated pixel voltage in this case) is compared with the reference voltage generated by the counter block. When the input voltage is higher than the generated ramp voltage, the change detector in the pixel is triggered and changed to 1. And the pixel location is registered for the corresponding value in the counter. After the first clock cycle, when the input voltage is higher than the ramp voltage, the pixel output will stay at one for all the other clock cycles and the change detector will generate 0 since the value in the pixel will always be larger than the ramp voltage.
  • This topology allows receiving the digital output of a pixel instantaneously after achieving a good approximation between the reference and the integrated charge on the pixel capacitance. By this way, an extra A to D conversion unit is eliminated.
  • an Address Event Representation Protocol For the registration of the pixel location, an Address Event Representation Protocol — AER— is used where the active pixels that generate row and column requests are queued in the arbiters and acknowledgements are generated to each pixel depending on its previous acknowledges.
  • AER Address Event Representation Protocol
  • This invention only detects events. Even if the pixel value is the same in the next frame, it recognizes it as a new event and rewrites the address of the pixel for the corresponding digital bits in the counter. However, data compression and change detection can still be achieved with this implementation by post processing the collected pixel values or images.
  • This invention detects changes generated in each pixel, which can be implemented as in Figure 9.
  • the pixel includes:
  • a voltage comparator (12) to compare the photodiode integrated voltage after certain exposure time with the ramp voltage (11), which is generated by column parallel counters (28) and DACs (27);
  • a D-latch (14), to transfer the current value of the comparator output with ClkB; • a block of two PMOS transistors (16), to generate pixel first stage output (OUT1) high when the output of the comparator is high and its previous state was low;
  • an NMOS switch transistor (32) that is controlled by the output of the first part of the pixel (OUT1), to update the values in the shift register when OUT1 is high i.e., when activity at the pixel is detected;
  • the invention includes a P-bit decrement counter (28) is connected to a DAC (27) and as well as a serializer (38).
  • the resolution of the counter (represented as P in this invention) can be augmented depending on the application.
  • the number of bits of the counter also determines the clock of the serializer, which is a P times faster clock (ClkD) than the clock of the counter (ClkC).
  • the number of the bits of the counter should also be in accordance with the resolution of the comparator within the pixel.
  • This technique is based on binary search where the digital control logic (counter) decrements bits at each clock cycle. After each decrement, the input voltage (which is the integrated photo current) is compared with the reference voltage generated by the counter-controlled DAC block. When the input voltage is higher than the generated ramp voltage, the change detector in the pixel is triggered and changed to 1. This change allows the pixel to send a row request. Once the row receives the acknowledge, the pixel with the activity meaning that the pixel with the comparator output at high, updates its stored values in the shift register. Later, a bit-wise comparison between the previously stored values in the shift registers and the current values is made. This comparison allows the generation of a column request. The column and row encoders generate the locations of the rows and columns that are activated. Since the column request can only be generated in presence of the row acknowledge, the active column requests with the active row acknowledges determine the location of the active pixels.
  • Figure 6 illustrates the timing diagram of this single event detection example.
  • row and column arbiters give acknowledge one by one to the active pixels and register the addresses of the ones that are active.
  • the number of active pixels does not change the speed of the system since the number of clock cycles is determined by the row and column numbers. I n this example, only one row and column requests are received by the arbiters and only one row and column acknowledges are given.
  • ClkA is "Low", photodiode is no longer is in reset mode and the photodiode voltage value starts to discharge with a slope determined by the light intensity exposed on the sensor.
  • the clock of the counter is the same as pixel reset clock (ClkA).
  • Each column cell is connected to an NMOS weak pull down transistor. This transistor allows the reset of Column Request or pull down of the Column Request signal in case no event is detected or the previous values of the D- Latch or Read Access Memories are at Logic High.
  • Row Acknowledge and Column Acknowledge signals are sent to the row (23) and column (24) address encoders.
  • Address encoders may generate the addresses in serial or parallel.
  • Figure 7 illustrates the timing diagram in case two pixels on the same row are active at the same time.
  • row and column arbiters give acknowledge one by one to the active pixels and register the addresses of the ones that are active.
  • the number of active pixels does not change the speed of the system since the number of clock cycles is determined by the row and column numbers.
  • I n this example two pixels on the sa me row but at consecutive columns are active and send request signals. The acknowledges to the two consecutive columns are given one after the other by the arbiters.
  • ClkA is "Low", photodiode is no longer is in reset mode and the photodiode voltage value starts to discharge with a slope determined by the light intensity exposed on the sensor.
  • the clock of the counter is the same as pixel reset clock ClkA.
  • Each column cell is connected to an NMOS weak pull down transistor. This transistor allows the reset of Column Request or pull down of the Column Request signal in case no event is detected or the previous values of the D- Latch or Read Access Memories are at Logic High.
  • Row Request is sent from the 0 th pixel, which triggers the column request signals from the 0 th and 1 st Columns.
  • the row arbiter gives the acknowledge to the 0 th row and the column arbiter, first gives acknowledge to the 0 th column and in the next clock cycle of ClkD to the 1 st Column.
  • Row Acknowledge and Column Acknowledge signals are sent to the address encoders. Address encoders may generate the addresses in serial or parallel.
  • EXAMPLE 3 Two pixels on the 0th row and two pixels on the 1st row are active:
  • Figure 8 illustrates the timing diagram in case two pixels on one row and two other pixels on another row are active at the same time.
  • row and column arbiters give acknowledge one by one to the active pixels and register the addresses of the ones that are active.
  • the number of active pixels does not change the speed of the system since the number of clock cycles is determined by the row and column numbers.
  • two pixels on one row at consecutive columns and two other pixels at another row at the same consecutive columns are active and send request signals.
  • 0 TH row receives acknowledge from the row arbiter and the acknowledges to the two consecutive columns are given one after the other by the arbiters.
  • 1 ST row receives acknowledge from the row arbiter and the acknowledges to the two consecutive columns are given one after the other by the arbiters.
  • ClkA is "Low", photodiode is no longer is in reset mode and the photodiode voltage value starts to discharge with a slope determined by the light intensity exposed on the sensor.
  • the clock of the counter is the same as pixel reset clock ClkA.
  • Each column cell is connected to an NMOS weak pull down transistor. This transistor allows the reset of Column Request or pull down of the Column Request signal in case no event is detected or the previous values of the D- Latch or Read Access Memories are at Logic High.
  • Row Request is sent from the 0 TH row, which triggers the column request signals from the 0 TH and 1 ST Columns.
  • the row arbiter gives the acknowledge to the 0 TH row and the column, first gives acknowledge to the 0 TH column and in the next clock cycle of ClkD to the 1 ST Column.
  • Row Request is sent from the 1 ST row, which triggers the column request signals from the 0 TH and 1 ST Columns again.
  • the row arbiter gives the acknowledge to the 0 TH row and the column arbiter, first gives acknowledge to the 0 TH column and in the next clock cycle of ClkD to the 1 ST Column.
  • FIG 11 illustrates the timing diagram of this single change detection example.
  • ClkA is "Low"
  • photodiode is no longer in reset mode and the photodiode voltage value starts to discharge with a slope determined by the light intensity exposed on the sensor.
  • the pass transistor (18) is responsible in passing the row request signal from the pixels to the arbiter. When the output of the comparator is 0, the row request is set to 0.
  • the PMOS transistors (35) is responsible in making a bitwise comparison when the acknowledge is given to a certain row and when the ClkD is active. If any of the bits in the shift register is different than the previous value of it, the column request is sent.
  • ClkD is a delayed version of ClkC, which controls the Column and Row Acknowledges and gives priority.
  • the serializer is controlled by a clock divided by the number of bits of the counter in order to allow the serialization in one period of ClkC
  • Each column cell is connected to an N MOS weak pull down transistor (37).
  • Row Request signals are queued and interpreted in the row arbiter (22).
  • the arbiter cell gives priority to one row at a time by also taking into consideration the previous acknowledge of the row.
  • the arbiter cells use a faster clock than the pixel clock named as ClkC in order to switch priority from one pixel to another in a fast and effective way.
  • Row Acknowledge and Column Request signals are sent to the address encoders marked as (23) and (25). Address encoders generate the addresses in serial or parallel.
  • an adaptive clock generation system which resets the clocks when there is no activity in any of the pixels with the corresponding ramp voltage.
  • Embodiments of the present invention can be used in a wide range of cameras, including applications such as; medical and biological instruments and devices, microscopy, cell-phones, motion tracking or security surveillance systems and other applications where compression of data and low power consumption with an added feature of automatic generation of images in case of a detected change with high image precision is desirable.
  • the use of the present invention in a camera may give the user the flexibility not to be present during the experiment but after the experiment or any time that is convenient to examine the collected images with a change-detected information at each frame.
  • the change or reaction within the samples can be very slow requiring examination of the samples for minutes/hours/days or weeks, which makes this invention highly preferable.
  • the invention may also relate with the automatic generation of the images with a controllable threshold in terms of the active number of pixels to activate a new frame/image.
  • the user may have the right to define the threshold value for the number of active pixels to raise a change flag in order to generate a new frame.
  • the design may include additional features to give the user the right to decide how much change in a pixel should be considered as a change to raise flag for a new frame.
  • a novel CMOS image sensor is proposed to overcome the disadvantages of both regular APS (Active Pixel Sensor) and PWM (Pulse Width Modulation) or PFM (Pulse Frequency Modulation) image sensor architectures.
  • the pixel architecture includes a reset transistor (RS) and a comparator for comparison of the pixel photo voltage with a ramp voltage generated by a column level Digital to Analog Converter controlled by a counter.
  • RS reset transistor
  • a comparator for comparison of the pixel photo voltage with a ramp voltage generated by a column level Digital to Analog Converter controlled by a counter.
  • Two methods for event or change monitoring are proposed. With these methods, each pixel continuously monitors its photocurrent for changes and the location of the pixel with an event or change is written as an output. Thus, the address of the event or change is registered by means of an address event representation (AER) protocol.
  • AER address event representation
  • Pixels that are not stimulated meaning pixels that do not trigger the change or event flag, do not produce any output.
  • the sensitivity (minimum detectable light) of the image sensor is determined by the resolution of the comparator as well as the chip level counter where both can easily be augmented. As opposed to PWM or PFM sensors, these sensors do not suffer from time collision. Moreover, as opposed to APS architectures, they do not suffer from analog design limitations. In addition, in the change detection sensor, by having only pixels with change activation, redundant data is avoided and data compression is provided together with low power consumption.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)

Abstract

A Digital Pixel Sensor (DPS) based CMOS camera configured to record frames and comprising an event detection sensor configured to look for an event happening in a determined frame, comprising an array of event detection pixels, a first reference voltage generator, first row and column arbiters, and first row and column address encoders; the DPS based CMOS camera further comprises a change detection sensor configured to look for a change happening in between frames, comprising an array of change detection pixels, a second binary search value generator, second row and column arbiters, and second row and column address encoders.

Description

A Novel CMOS Image Sensor with Event/Change Detection and
Reduced Data Redundancy
1 BACKGROUND OF THE INVENTION
1.1 Technical Field
The present invention relates to a CMOS image sensor design to be used in CMOS imaging devices and camera systems.
1.2 Description of the Related Art
CMOS image sensors are one of the most widely used electronic devices in today's market. There is a huge research effort to satisfy the need in this field. Several CMOS image sensor architectures have been developed in recent years. Active Pixel Sensors (APS), which has analog read-out, and Digital Pixel Sensors (DPS), which provide pixel level digital output, are two main techniques for CMOS image sensor arrays. Most of the APS architectures have current or voltage outputs at the pixel level and the corresponding pixel value is converted to digital by means of column or chip level Analog to Digital Converters (ADC). As opposed, DPS architectures convert the pixel values to digital in pixel level and avoid the requirements of extra conversion units.
The DPS architectures offer several advantages over analog ones by eliminating column level Fixed Pattern Noise (FPN), reducing the number of analog read-out blocks in the design and getting rid of the demands of analog design, which results in better matching with CMOS scaling. However, these architectures come with a cost of pixel area i.e., reduced Fill Factor (FF) or increased pixel pitch.
Regular DPS designs use fixed exposure time intervals for every pixel similar to the APS designs and outputs a result according to the pixel value at the end of that exposure time [1]. Most of these designs suffer from large non-photosensitive area due to the pixel level memory, which gets even worse when a higher resolution sensor is required.
Some DPS architectures also provide possibility to add extra features at the pixel level, e.g., change/movement detection, data compression [2], [3], which decreases the fill factor even more. Hence, DPS designs bring many advantages with additional pixel features, however it is of great importance to keep non-photosensitive area as low as possible, while benefiting from these architectures.
[1] Kleinfelder, Stuart, et al. "A 10000 frames/s CMOS digital pixel sensor." Solid- State Circuits, IEEE Journal of 36.12 (2001): 2049-2059.
[2] Mallik, U., et al. "Temporal change threshold detection imager." Solid-State Circuits Conference, 2005. Digest of Technical Papers. ISSCC. 2005 IEEE International. IEEE, 2005.
[3] Posch, Christoph, Daniel Matolin, and Rainer Wohlgenannt. "A QVGA 143dB dynamic range asynchronous address-event PWM dynamic image sensor with lossless pixel-level video compression." Solid-State Circuits Conference Digest of Technical Papers (ISSCC), 2010 IEEE International. IEEE, 2010.
2 SUMMARY OF THE INVENTION
In a first aspect, the invention provides a Digital Pixel Sensor (DPS) based CMOS camera, the CMOS camera being configured to record frames, wherein the CMOS camera comprises an event detection sensor configured to look for an event happening in a determined frame, comprising an array of event detection pixels, a first reference voltage generator, first row and column arbiters, and first row and column address encoders; the DPS based CMOS camera further comprises a change detection sensor configured to look for a change happening in between frames, comprising an array of change detection pixels, a second binary search value generator, second row and column arbiters, and second row and column address encoders.
In a preferred embodiment of the digital pixel sensor based CMOS camera, the event detection sensor (intra-frame comparison sensors) relies on a continuous search technique inside a frame that continues to search for values of every one of the event detection pixels of the array of the event detection pixels until it finds the corresponding reference voltages for each pixel, the sensor generates reference voltages that decrements at each clock cycle of the sensor by use of the first reference voltage generator, by comparing the generated reference voltage with each event detection pixel's integrated voltage until the pixel value becomes higher than the reference voltage, the reference voltage corresponding to the pixel value is found, when the pixel voltage becomes higher than the reference voltage at a specific clock cycle and a specific reference voltage, an event flag is raised by that pixel, and the address of that pixel is registered through the use of the first row and column address encoders, when multiple of the event detection pixels generate events at the same time, the first row and column arbiters are used to give priority to one pixel at a time when registering their addresses.
In a further preferred embodiment of the digital pixel sensor based CMOS camera, the change detection sensor (inter-frame comparison sensor) relies both on a continuous search technique inside a frame to find the value of each one of the change detection pixels of the array of change detection pixels in a frame and on a continuous comparison technique that compares each change detection pixel's value in the previous frame (or the pixel's last value with event at any frame) with its current frame value, the sensor generates reference voltages that decrements at each clock cycle of the sensor by use of a second reference voltage generator, by comparing the generated reference voltage with each change detection pixel's integrated voltage until the pixel value becomes higher than the reference voltage, the reference voltage corresponding to the pixel value is found, when the pixel voltage becomes higher than the reference voltage at a specific clock cycle and a specific reference voltage, a change flag is raised by that pixel, the address of that pixel is registered through the use of the second row and column address encoders both in the sensor and the pixel memory, the memory in the pixel is used to compare the previous frame value (or the pixel's last value with event at any frame) of that pixel with its current frame value, when multiple of the change detection pixels generate events at the same time, the second row and column arbiters are used to give priority to one pixel at a time when registering their addresses. With this method, only the pixel addresses that raise an event flag at a reference voltage that is different than the one in the previous frame is registered.
In a second aspect the invention provides a method for data compression in a change detection sensor and reduced data redundancy comprising configuring a change detection sensor such that it looks for a change happening in between frames and generating a change flag for each change detection pixel of an array only when there is a difference between a pixel's current frame value and its previous frame value (or the pixel's last value with event at any frame), whereby the method reduces the amount of data that is required to be written in a memory (data compression) and avoids the repetition of the registration of the same data multiple times (reduced data redundancy).
In a third aspect, the invention provides a method for automatic generation of frames in a change detection sensor comprising automatically generating new frames only when there is a change in one or more of the change detection pixels of an array. In a further preferred embodiment the method further comprises a use with a defined number of changed pixel threshold by providing a threshold to the sensor that defines how many change detection pixel's value should be different than their previous frame values before automatically generating a new frame.
In a fourth aspect, the invention provides a use of a Digital Pixel Sensor (DPS) based CMOS cameras in microscopes with a change detection feature and automatic generation of frames when there is a change in the imaged sample.
In a further preferred embodiment, in the used of the DPS based CMOS camera in microscopes, the CMOS camera is the CMOS camera according to the first aspect of the invention.
In a sixth aspect, the invention provides a use of a Digital Pixel Sensor (DPS) based CMOS camera for biomedical applications, the CMOS camera being configured to record frames, wherein the CMOS camera comprises a change detection sensor configured to look for a change happening in between frames, comprising an array of change detection pixels, a second counter, a second digital to analogue converter, second row and column arbiters, and second row and column address encoders; the change detection sensor (inter-frame comparison sensor) relies both on a continuous search technique inside a frame to find the value of each change detection pixel of an array in a frame and on a continuous comparison technique that compares each change detection pixel's value in the previous frame with its current frame value, the sensor generates reference voltages that decrements at each clock cycle of the sensor by use of a second reference voltage generator, by comparing the generated reference voltage with each change detection pixel's integrated voltage until the pixel value becomes higher than the reference voltage, the reference voltage corresponding to the pixel value is found, when the pixel voltage becomes higher than the reference voltage at a specific clock cycle and a specific reference voltage, a change flag is raised by that pixel, the address of that pixel is registered through the use of the second row and column address encoders both in the sensor and the pixel memory, the memory in the pixel is used to compare the previous frame value of that pixel with its current frame value, when multiple of the change detection pixels generate events at the same time, second row and column arbiters are used to give priority to one pixel at a time when registering their addresses, in the next frame, only the pixel addresses that raises an event flag at a reference voltage that is different than the one in the previous frame is registered.
In a seventh aspect, the invention provides a use of a Digital Pixel Sensor (DPS) based CMOS camera for microscopes, the CMOS camera being configured to record frames, wherein the CMOS camera comprises a change detection sensor configured to look for a change happening in between frames, comprising an array of change detection pixels, a second counter, a second digital to analogue converter, second row and column arbiters, and second row and column address encoders; the change detection sensor (inter-frame comparison sensor) relies both on a continuous search technique inside a frame to find the value of each change detection pixel of an array in a frame and on a continuous comparison technique that compares each change detection pixel's value in the previous frame with its current frame value, the sensor generates reference voltages that decrements at each clock cycle of the sensor by use of a second reference voltage generator, by comparing the generated reference voltage with each change detection pixel's integrated voltage until the pixel value becomes higher than the reference voltage, the reference voltage corresponding to the pixel value is found when the pixel voltage becomes higher than the reference voltage at a specific clock cycle and a specific reference voltage, a change flag is raised by that pixel, the address of that pixel is registered through the use of the second row and column address encoders both in the sensor and the pixel memory, the memory in the pixel is used to compare the previous frame value of that pixel with its current frame value, when multiple of the change detection pixels generate events at the same time, second row and column arbiters are used to give priority to one pixel at a time when registering their addresses, in the next frame, only the pixel addresses that raises an event flag at a reference voltage that is different than the one in the previous frame is registered.
3 BRIEF DESCRIPTION OF THE FIGURES
The invention will be understood in the light of description of preferred embodiments and in reference to the appended figures, wherein figure 1 shows standard implementation of a DPS; figure 2 illustrates one of the possible implementation of the proposed event detection pixel by using D-Latches (14) according to a preferred embodiment of the invention; figure 3 illustrates another method of the pixel implementation of the proposed event detection pixel by using Static Read Access Memory— SRAM— (20) for event detection, according to a preferred embodiment of the invention; figure 4 illustrates another method of the pixel implementation of the proposed event detection pixel by using Dynamic Read Access Memory—DRAM— (21), according to a preferred embodiment of the invention; figure 5 illustrates the full image sensor with MxN array of pixels, column level counters (28) and DACs (29), column arbiter (25), row arbiter (22), column address encoder (24), row address encoder (23), according to a preferred embodiment of the event detection invention; figure 6 illustrates the timing diagram of the first example of the invention where only one single event is shown; figure 7 illustrates the timing diagram of the second example of the invention where two pixels (1st and 2nd Column) on the same row (1st row) are active at the same time, which allows the column arbiter to queue the events and read them out sequentially; figure 8 illustrates the timing diagram of the third example of the invention where two pixels (1st and 2nd Column) on 1st row and two pixels (1st and 2nd Column) on 2nd row are active at the same time which allows the column and row arbiter to queue the events and read them out sequentially; figure 9 illustrates one of the possible implementation of the proposed change detection pixel invention according to a preferred embodiment of the invention; figure 10 illustrates the full image sensor with MxN array of pixels, chip level counter (28) and DAC (29), column arbiter (25), row arbiter (22), column address encoder (24), row address encoder (23), according to a preferred embodiment of the invention; and figure 11 illustrates a further timing diagram.
4 DETAILED DESCRIPTION OF THE INVENTION
The present invention relates to Digital Pixel Sensors. Various designs include array of pixels, a counter per chip, a Digital to Analog Converter— DAC— per chip and row and column arbiters and encoders.
Each pixel includes a reset transistor, a comparator and certain number of transistors for logic functions and row and column request and acknowledges processing. 4.1 Description of the Event Detection Design
This invention detects events generated in each pixel possibly implemented as in Figures 1, 2 and 3.
The pixel includes:
• a photo-device (PD);
• a reset transistor (M S) (10),
• a voltage comparator (12), to compare the photodiode integrated voltage after certain exposure time with the ramp voltage (11), which is generated by column parallel counters (28) and DACs (27);
• a PMOS switch (18), to connect the comparator output to a common row request, which is active when the comparator output is high;
• an inverter (19), to control the gates of PMOS switch (18) and one of the two PMOS transistors (16);
• a weak pull-down NMOS transistor (17), to pull down the row request to low when there is no activity at the pixel;
• A D-latch (14) or an SRAM (20) or a DRAM (21), to transfer the current value of the comparator output when the pixel receives both row and column acknowledges, i.e., when the output of the AND gate (13) with inputs connected to row and column acknowledges change state from 0 to 1;
• a block of two PMOS transistors (16), to generate pixel column request when the output of the comparator is high and its previous state was low;
• a weak pull-down NMOS transistor (15), to pull down the pixel column request to low when there is no activity at the pixel;
An N-bit decrement counter (28) is connected to a DAC (27) and generates the ramp voltage of the pixel (11). The counter and DAC can be designed per chip only, which reduces the area. The resolution of the counter (represented as N in this invention) can be easily augmented depending on the application. The number of bits of the counter should be in accordance with the resolution of the comparator within the pixel.
The technique is based on binary search where the digital control logic (counter) decrements bits at each clock cycle. After each decrement, the input voltage (which is the integrated pixel voltage in this case) is compared with the reference voltage generated by the counter block. When the input voltage is higher than the generated ramp voltage, the change detector in the pixel is triggered and changed to 1. And the pixel location is registered for the corresponding value in the counter. After the first clock cycle, when the input voltage is higher than the ramp voltage, the pixel output will stay at one for all the other clock cycles and the change detector will generate 0 since the value in the pixel will always be larger than the ramp voltage.
This topology allows receiving the digital output of a pixel instantaneously after achieving a good approximation between the reference and the integrated charge on the pixel capacitance. By this way, an extra A to D conversion unit is eliminated.
For the registration of the pixel location, an Address Event Representation Protocol — AER— is used where the active pixels that generate row and column requests are queued in the arbiters and acknowledgements are generated to each pixel depending on its previous acknowledges.
This invention only detects events. Even if the pixel value is the same in the next frame, it recognizes it as a new event and rewrites the address of the pixel for the corresponding digital bits in the counter. However, data compression and change detection can still be achieved with this implementation by post processing the collected pixel values or images.
4.2 Description of the Change Detection Design
This invention detects changes generated in each pixel, which can be implemented as in Figure 9.
The pixel includes:
• a photo-device (PD);
• a reset transistor (M RS) (10), to reset the photo-device voltage;
• a voltage comparator (12), to compare the photodiode integrated voltage after certain exposure time with the ramp voltage (11), which is generated by column parallel counters (28) and DACs (27);
• a PMOS switch (18), to connect the comparator output to a common row request, which is active when the comparator output is high;
• an inverter (19), to control the gates of PMOS switch (18) and one of the two PMOS transistors (16);
• a weak pull-down N MOS transistor (17), to pull down the row request to low when there is no activity at the pixel;
• A D-latch (14), to transfer the current value of the comparator output with ClkB; • a block of two PMOS transistors (16), to generate pixel first stage output (OUT1) high when the output of the comparator is high and its previous state was low;
• a weak pull-down NMOS transistor (15), to pull down the pixel first stage output (OUT1) to low when there is no activity at the pixel;
• a shift register, to make bit wise comparison between pixel's current digital value with its last digital value, which is constructed by serially connected DFFs (34);
• an NMOS switch transistor (32) that is controlled by the output of the first part of the pixel (OUT1), to update the values in the shift register when OUT1 is high i.e., when activity at the pixel is detected;
• a PMOS switch (33) that helps the shift registers to keep their previous values by putting them into a loop if the first part of the pixel output is low, i.e., there is no activity at the pixel
• a block of PMOS transistors (35) that controls the column request signal according to the bit wise comparison between the current and previous pixel digital valu;
• two PMOS transistors (36) controlled by the inverse of row acknowledge and ClkD, to generate a column request when the pixel receives row acknowledge, detects change between its previous and current digital value during the low state of ClkD.
• a weak pull down transistor to pull down the column request low when there is no activity at the pixel or when there is no difference between the previous and current digital value of the pixel, although it detects an activity and receives row acknowledge.
At the column level or chip level, the invention includes a P-bit decrement counter (28) is connected to a DAC (27) and as well as a serializer (38). The resolution of the counter (represented as P in this invention) can be augmented depending on the application. The number of bits of the counter also determines the clock of the serializer, which is a P times faster clock (ClkD) than the clock of the counter (ClkC). The number of the bits of the counter should also be in accordance with the resolution of the comparator within the pixel.
This technique is based on binary search where the digital control logic (counter) decrements bits at each clock cycle. After each decrement, the input voltage (which is the integrated photo current) is compared with the reference voltage generated by the counter-controlled DAC block. When the input voltage is higher than the generated ramp voltage, the change detector in the pixel is triggered and changed to 1. This change allows the pixel to send a row request. Once the row receives the acknowledge, the pixel with the activity meaning that the pixel with the comparator output at high, updates its stored values in the shift register. Later, a bit-wise comparison between the previously stored values in the shift registers and the current values is made. This comparison allows the generation of a column request. The column and row encoders generate the locations of the rows and columns that are activated. Since the column request can only be generated in presence of the row acknowledge, the active column requests with the active row acknowledges determine the location of the active pixels.
5 DETAILED DESCRIPTION OF THE INVENTION - EVENT DETECTION EXAMPLE 1— Single
Figure 6 illustrates the timing diagram of this single event detection example. When both ClkA and ClkB are low, row and column arbiters give acknowledge one by one to the active pixels and register the addresses of the ones that are active. The number of active pixels does not change the speed of the system since the number of clock cycles is determined by the row and column numbers. I n this example, only one row and column requests are received by the arbiters and only one row and column acknowledges are given.
1) Global Reset is "High", providing Row and Column Acknowledges to each and every pixel which guarantees the D-Latch (14) or SRAM (20) to be transparent or the SRAM (20) or DRAM (21) to be in write mode, which sets their initial values to "LOW".
2) ClkA is "High", pixel M RS (10) is switched on and the photodiode is reset to VDD - Vth or Vset - Vth depending on the drain voltage of the M RS transistor.
3) ClkA is "Low", photodiode is no longer is in reset mode and the photodiode voltage value starts to discharge with a slope determined by the light intensity exposed on the sensor.
4) ClkB is "High"; the continuous comparison result of the ramp voltage (11) and the integrated photodiode voltage is taken.
5) The clock of the counter is the same as pixel reset clock (ClkA).
6) Each column cell is connected to an NMOS weak pull down transistor. This transistor allows the reset of Column Request or pull down of the Column Request signal in case no event is detected or the previous values of the D- Latch or Read Access Memories are at Logic High.
7) Only one row and column request signals are present and they are interpreted in the row (30) and column (25) arbiters. The arbiter cells give priority to those pixels when the clocks reach the time for their acknowledge. The arbi- ter cells use faster clocks called ClkC and ClkD in order to switch priority from one pixel to another in a fast and effective way.
Row Acknowledge and Column Acknowledge signals are sent to the row (23) and column (24) address encoders. Address encoders may generate the addresses in serial or parallel.
EXAMPLE 2— Two pixels on the same row are active:
Figure 7 illustrates the timing diagram in case two pixels on the same row are active at the same time. When both ClkA and ClkB are low, row and column arbiters give acknowledge one by one to the active pixels and register the addresses of the ones that are active. The number of active pixels does not change the speed of the system since the number of clock cycles is determined by the row and column numbers. I n this example, two pixels on the sa me row but at consecutive columns are active and send request signals. The acknowledges to the two consecutive columns are given one after the other by the arbiters.
1) Global Reset is "High", providing Row and Column Acknowledges to each and every pixel which guarantees the D-Latch (14) in Figure 1 or SRAM (20) to be transparent or the SRAM (20) in Figure 2 or DRAM (21) in Figure 3 to be in write mode, which sets their initial values to "LOW".
2) ClkA is "High", pixel MRS (10) in Figure 1 is switched on and the photodiode is reset to VDD - Vth or Vset - Vth depending on the drain voltage of the MRS transistor.
3) ClkA is "Low", photodiode is no longer is in reset mode and the photodiode voltage value starts to discharge with a slope determined by the light intensity exposed on the sensor.
4) ClkB is "High"; the continuous comparison result of the ramp voltage (11) in Figure 1 and the integrated photodiode voltage is taken.
5) The clock of the counter is the same as pixel reset clock ClkA.
6) Each column cell is connected to an NMOS weak pull down transistor. This transistor allows the reset of Column Request or pull down of the Column Request signal in case no event is detected or the previous values of the D- Latch or Read Access Memories are at Logic High.
7) Row Request is sent from the 0th pixel, which triggers the column request signals from the 0th and 1st Columns. The row arbiter gives the acknowledge to the 0th row and the column arbiter, first gives acknowledge to the 0th column and in the next clock cycle of ClkD to the 1st Column. 8) Row Acknowledge and Column Acknowledge signals are sent to the address encoders. Address encoders may generate the addresses in serial or parallel.
EXAMPLE 3— Two pixels on the 0th row and two pixels on the 1st row are active:
Figure 8 illustrates the timing diagram in case two pixels on one row and two other pixels on another row are active at the same time. When both ClkA and ClkB are low, row and column arbiters give acknowledge one by one to the active pixels and register the addresses of the ones that are active. The number of active pixels does not change the speed of the system since the number of clock cycles is determined by the row and column numbers. In this example, two pixels on one row at consecutive columns and two other pixels at another row at the same consecutive columns are active and send request signals. In the first clock cycle of ClkC, 0TH row receives acknowledge from the row arbiter and the acknowledges to the two consecutive columns are given one after the other by the arbiters. In the 2ND clock cycle of ClkC, 1ST row receives acknowledge from the row arbiter and the acknowledges to the two consecutive columns are given one after the other by the arbiters.
1) Global Reset is "High", providing Row and Column Acknowledges to each and every pixel which guarantees the D-Latch (14) to be transparent or the SRAM (20) or DRAM (21) to be in write mode, which sets their initial values to "LOW".
2) ClkA is "High", pixel MRS (10) is switched on and the photodiode is reset to VDD - Vth or Vset - Vth depending on the drain voltage of the MRS transistor.
3) ClkA is "Low", photodiode is no longer is in reset mode and the photodiode voltage value starts to discharge with a slope determined by the light intensity exposed on the sensor.
4) ClkB is "High"; the continuous comparison result of the ramp voltage (11) and the integrated photodiode voltage is taken.
5) The clock of the counter is the same as pixel reset clock ClkA.
6) Each column cell is connected to an NMOS weak pull down transistor. This transistor allows the reset of Column Request or pull down of the Column Request signal in case no event is detected or the previous values of the D- Latch or Read Access Memories are at Logic High.
7) In the 1ST clock cycle of ClkC, Row Request is sent from the 0TH row, which triggers the column request signals from the 0TH and 1ST Columns. The row arbiter gives the acknowledge to the 0TH row and the column, first gives acknowledge to the 0TH column and in the next clock cycle of ClkD to the 1ST Column.
8) In the 2ND clock cycle of ClkC, Row Request is sent from the 1ST row, which triggers the column request signals from the 0TH and 1ST Columns again. The row arbiter gives the acknowledge to the 0TH row and the column arbiter, first gives acknowledge to the 0TH column and in the next clock cycle of ClkD to the 1ST Column.
EXAMPLE 4— Single Change Detection:
Figure 11 illustrates the timing diagram of this single change detection example.
1) Global Reset is "High", provides the initial set phase for DFF (34), which sets all the outputs DFFs to 1.
2) ClkA is "High", pixel MRS (10) in Figure 10 is switched on and the photodiode is reset to VDD - Vth or Vset - Vth depending on the drain voltage of the MRS transistor.
3) ClkA is "Low", photodiode is no longer in reset mode and the photodiode voltage value starts to discharge with a slope determined by the light intensity exposed on the sensor.
4) ClkB is "High"; the continuous comparison result of the ramp voltage (11) with the photo-voltage by use of a comparator (12) is taken.
5) The pass transistor (18) is responsible in passing the row request signal from the pixels to the arbiter. When the output of the comparator is 0, the row request is set to 0.
6) When a row request is sent from a row to the arbiter, the arbiter gives acknowledge to the rows depending on their priority. When both row acknowledges and comparator outputs are active, the current value in the column level counter is shifted into the shift registers constructed by DFFs in (34). If the first stages output is low; the shift register stays in loop and keeps its current value by use of the PMOS pass transistor (33).
7) The PMOS transistors (35) is responsible in making a bitwise comparison when the acknowledge is given to a certain row and when the ClkD is active. If any of the bits in the shift register is different than the previous value of it, the column request is sent.
8) ClkD is a delayed version of ClkC, which controls the Column and Row Acknowledges and gives priority.
9) The serializer is controlled by a clock divided by the number of bits of the counter in order to allow the serialization in one period of ClkC
10) Each column cell is connected to an N MOS weak pull down transistor (37).
This transistor allows the reset of Column Request or pull down of the Column Request signal in case no event is detected or the previous values of the shift registers are the same as the new values. 11) Row Request signals are queued and interpreted in the row arbiter (22). The arbiter cell gives priority to one row at a time by also taking into consideration the previous acknowledge of the row. The arbiter cells use a faster clock than the pixel clock named as ClkC in order to switch priority from one pixel to another in a fast and effective way.
12) Row Acknowledge and Column Request signals are sent to the address encoders marked as (23) and (25). Address encoders generate the addresses in serial or parallel.
_?3j This architecture does not require a column acknowledge as opposed to the event-detection architecture.
14) In order to increase the data read-out speed of the design, an adaptive clock generation system can also be proposed, which resets the clocks when there is no activity in any of the pixels with the corresponding ramp voltage.
6 USE EXAMPLES FOR CHANGE DETECTION INVENTION
Embodiments of the present invention can be used in a wide range of cameras, including applications such as; medical and biological instruments and devices, microscopy, cell-phones, motion tracking or security surveillance systems and other applications where compression of data and low power consumption with an added feature of automatic generation of images in case of a detected change with high image precision is desirable.
For the microscopic cameras and biological chips with image sensors requiring longtime measurements and examinations, the use of the present invention in a camera may give the user the flexibility not to be present during the experiment but after the experiment or any time that is convenient to examine the collected images with a change-detected information at each frame. In some biological applications, the change or reaction within the samples can be very slow requiring examination of the samples for minutes/hours/days or weeks, which makes this invention highly preferable.
In motion tracking or security surveillance systems, data compression of a video or consecutive images including the change information only may be highly requested considering the long time of image/frame capture.
The invention may also relate with the automatic generation of the images with a controllable threshold in terms of the active number of pixels to activate a new frame/image. In these kinds of requirements, the user may have the right to define the threshold value for the number of active pixels to raise a change flag in order to generate a new frame. Similarly, the design may include additional features to give the user the right to decide how much change in a pixel should be considered as a change to raise flag for a new frame.
7 CONCLUSION
A novel CMOS image sensor is proposed to overcome the disadvantages of both regular APS (Active Pixel Sensor) and PWM (Pulse Width Modulation) or PFM (Pulse Frequency Modulation) image sensor architectures. The pixel architecture includes a reset transistor (RS) and a comparator for comparison of the pixel photo voltage with a ramp voltage generated by a column level Digital to Analog Converter controlled by a counter. Two methods for event or change monitoring are proposed. With these methods, each pixel continuously monitors its photocurrent for changes and the location of the pixel with an event or change is written as an output. Thus, the address of the event or change is registered by means of an address event representation (AER) protocol. Pixels that are not stimulated, meaning pixels that do not trigger the change or event flag, do not produce any output. The sensitivity (minimum detectable light) of the image sensor is determined by the resolution of the comparator as well as the chip level counter where both can easily be augmented. As opposed to PWM or PFM sensors, these sensors do not suffer from time collision. Moreover, as opposed to APS architectures, they do not suffer from analog design limitations. In addition, in the change detection sensor, by having only pixels with change activation, redundant data is avoided and data compression is provided together with low power consumption.

Claims

A Digital Pixel Sensor (DPS) based CMOS camera, the CMOS camera being configured to record frames, wherein the CMOS camera comprises
an event detection sensor configured to look for an event happening in a determined frame, comprising
an array of event detection pixels,
a first reference voltage generator,
first row and column arbiters, and
first row and column address encoders;
a change detection sensor configured to look for a change happening in between frames, comprising
an array of change detection pixels,
a second binary search value generator,
second row and column arbiters, and
second row and column address encoders.
The digital pixel sensor based CMOS camera of claim 1
whereby the event detection sensor (intra-frame comparison sensors) relies on a continuous search technique inside a frame that continues to search for values of every one of the event detection pixels of the array of the event detection pixels until it finds the corresponding reference voltages for each pixel, the sensor generates reference voltages that decrements at each clock cycle of the sensor by use of the first reference voltage generator, by comparing the generated reference voltage with each event detection pixel's integrated voltage until the pixel value becomes higher than the reference voltage, the reference voltage corresponding to the pixel value is found, when the pixel voltage becomes higher than the reference voltage at a specific clock cycle and a specific reference voltage, an event flag is raised by that pixel, and the address of that pixel is registered through the use of the first row and column address encoders, when multiple of the event detection pixels generate events at the same time, the first row and column arbiters are used to give priority to one pixel at a time when registering their addresses.
3. The digital pixel sensor based CMOS camera of claim 1 or claim 2
whereby the change detection sensor (inter-frame comparison sensor) relies both on a continuous search technique inside a frame to find the value of each one of the change detection pixels of the array of change detection pixels in a frame and on a continuous comparison technique that compares each change detection pixel's value in the previous frame with its current frame value, the sensor generates reference voltages that decrements at each clock cycle of the sensor by use of a second reference voltage generator, by comparing the generated reference voltage with each change detection pixel's integrated voltage until the pixel value becomes higher than the reference voltage, the reference voltage corresponding to the pixel value is found, when the pixel voltage becomes higher than the reference voltage at a specific clock cycle and a specific reference voltage, a change flag is raised by that pixel, the address of that pixel is registered through the use of the second row and column address encoders both in the sensor and the pixel memory, the memory in the pixel is used to compare the previous frame value of that pixel with its current frame value, when multiple of the change detection pixels generate events at the same time, the second row and column arbiters are used to give priority to one pixel at a time when registering their addresses, in the next frame, only the pixel addresses that raises an event flag at a reference voltage that is different than the one in the previous frame is registered.
4. A method for data compression in a change detection sensor and reduced data redundancy comprises
configuring a change detection sensor such that it looks for a change happening in between frames and generating a change flag for each change detection pixel of an array only when there is a difference between a pixel's current frame value and its previous frame value, whereby the method reduces the amount of data that is required to be written in a memory (data compression) and avoids the repetition of the registration of the same data multiple times (reduced data redundancy).
5. A method for automatic generation of frames in a change detection sensor comprises
automatically generating new frames only when there is a change in one or more of the change detection pixels of an array.
6. The method of claim 5, further comprising a use with a defined number of changed pixel threshold by providing a threshold to the sensor that defines how many change detection pixel's value should be different than their previous frame values before automatically generating a new frame.
7. Use of a Digital Pixel Sensor (DPS) based CMOS cameras in microscopes with a change detection feature and automatic generation of frames when there is a change in the imaged sample.
8. The use of claim 7, wherein the CMOS camera is the CMOS camera of claim 1.
9. Use of a Digital Pixel Sensor (DPS) based CMOS camera for biomedical applications, the CMOS camera being configured to record frames, wherein the CMOS camera comprises
a change detection sensor configured to look for a change happening in between frames, comprising
an array of change detection pixels,
a second counter,
a second digital to analogue converter,
second row and column arbiters, and
second row and column address encoders;
whereby the change detection sensor (inter-frame comparison sensor) relies both on a continuous search technique inside a frame to find the value of each change detection pixel of an array in a frame and on a continuous comparison technique that compares each change detection pixel's value in the previous frame with its current frame value, the sensor generates reference voltages that decrements at each clock cycle of the sensor by use of a second reference voltage generator, by comparing the generated reference voltage with each change detection pixel's integrated voltage until the pixel value becomes higher than the reference voltage, the reference voltage corresponding to the pixel value is found, when the pixel voltage becomes higher than the reference voltage at a specific clock cycle and a specific reference voltage, a change flag is raised by that pixel, the address of that pixel is registered through the use of the second row and column address encoders both in the sensor and the pixel memory, the memory in the pixel is used to compare the previous frame value of that pixel with its current frame value, when multiple of the change detection pixels generate events at the same time, second row and column arbiters are used to give priority to one pixel at a time when registering their addresses, in the next frame, only the pixel addresses that raises an event flag at a reference voltage that is different than the one in the previous frame is registered.
10. Use of a Digital Pixel Sensor (DPS) based CMOS camera for microscopes, the CMOS camera being configured to record frames, wherein the CMOS camera comprises
a change detection sensor configured to look for a change happening in between frames, comprising
an array of change detection pixels,
a second counter,
a second digital to analogue converter,
second row and column arbiters, and
second row and column address encoders;
whereby the change detection sensor (inter-frame comparison sensor) relies both on a continuous search technique inside a frame to find the value of each change detection pixel of an array in a frame and on a continuous comparison technique that compares each change detection pixel's value in the previous frame with its current frame value, the sensor generates reference voltages that decrements at each clock cycle of the sensor by use of a second reference voltage generator, by comparing the generated reference voltage with each change detection pixel's integrated voltage until the pixel value becomes higher than the reference voltage, the reference voltage corresponding to the pixel value is found when the pixel voltage becomes higher than the reference voltage at a specific clock cycle and a specific reference voltage, a change flag is raised by that pixel, the address of that pixel is registered through the use of the second row and column address encoders both in the sensor and the pixel memory, the memory in the pixel is used to compare the previous frame value of that pixel with its current frame value, when multiple of the change detection pixels generate events at the same time, second row and column arbiters are used to give priority to one pixel at a time when registering their addresses, in the next frame, only the pixel addresses that raises an event flag at a reference voltage that is different than the one in the previous frame is registered.
PCT/IB2014/061057 2013-04-26 2014-04-28 A novel cmos image sensor with event/change detection and reduced data redundancy WO2014174498A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201361816197P 2013-04-26 2013-04-26
US61/816,197 2013-04-26

Publications (1)

Publication Number Publication Date
WO2014174498A1 true WO2014174498A1 (en) 2014-10-30

Family

ID=50942717

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2014/061057 WO2014174498A1 (en) 2013-04-26 2014-04-28 A novel cmos image sensor with event/change detection and reduced data redundancy

Country Status (1)

Country Link
WO (1) WO2014174498A1 (en)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107635087A (en) * 2017-11-08 2018-01-26 深圳地平线机器人科技有限公司 Solid state image sensor and image-sensing method
CN107635086A (en) * 2017-10-25 2018-01-26 深圳地平线机器人科技有限公司 Solid state image sensor and image-sensing method
DE102016011913A1 (en) 2016-10-05 2018-04-05 Hensoldt Sensors Gmbh Detector unit and a method for detecting an optical detection signal
CN108089697A (en) * 2016-11-21 2018-05-29 三星电子株式会社 The sensor based on event including power control circuit
EP3386191A1 (en) * 2017-04-07 2018-10-10 Commissariat à l'Energie Atomique et aux Energies Alternatives Array sensor with time coding without arbitration
CN110677603A (en) * 2019-09-16 2020-01-10 上海集成电路研发中心有限公司 Image sensor for single-frame segmented exposure
CN110708481A (en) * 2019-09-16 2020-01-17 上海集成电路研发中心有限公司 Fixed pattern noise removing method based on difference reset
CN111713101A (en) * 2017-12-11 2020-09-25 普罗菲西公司 Event-based image sensor and method of operating the same
EP3930312A1 (en) * 2020-06-26 2021-12-29 Alpsentek GmbH Delta image sensor with digital pixel storage
EP3967027B1 (en) * 2019-05-10 2023-10-25 Sony Advanced Visual Sensing AG Dynamic vision sensor with in-pixel digital change detection
US11962926B2 (en) 2020-08-14 2024-04-16 Alpsentek Gmbh Image sensor with configurable pixel circuit and method

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6498576B1 (en) * 2000-05-09 2002-12-24 Pixim, Inc. Selective analog-to-digital conversion for a digital pixel sensor
US20070109433A1 (en) * 2005-11-16 2007-05-17 Matsushita Electric Industrial Co., Ltd. Solid-state imaging device for high-speed photography
US20100182468A1 (en) * 2006-11-23 2010-07-22 Austrian Research Centers Gmbh-Arc Method for the generation of an image in electronic form, picture element (pixel) for an image sensor for the generation of an image as well as image sensor

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6498576B1 (en) * 2000-05-09 2002-12-24 Pixim, Inc. Selective analog-to-digital conversion for a digital pixel sensor
US20070109433A1 (en) * 2005-11-16 2007-05-17 Matsushita Electric Industrial Co., Ltd. Solid-state imaging device for high-speed photography
US20100182468A1 (en) * 2006-11-23 2010-07-22 Austrian Research Centers Gmbh-Arc Method for the generation of an image in electronic form, picture element (pixel) for an image sensor for the generation of an image as well as image sensor

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
KLEINFELDER, STUART ET AL.: "A 10000 frames/s CMOS digital pixel sensor", SOLID-STATE CIRCUITS, IEEE JOURNAL OF 36.12, 2001, pages 2049 - 2059
MALLIK, U ET AL.: "Temporal change threshold detection imager", SOLID-STATE CIRCUITS CONFERENCE, 2005. DIGEST OF TECHNICAL PAPERS. ISSCC. 2005 IEEE INTERNATIONAL, 2005
POSCH, CHRISTOPH; DANIEL MATOLIN; RAINER WOHLGENANNT.: "A QVGA 143dB dynamic range asynchronous address-event PWM dynamic image sensor with lossless pixel-level video compression", SOLID-STATE CIRCUITS CONFERENCE DIGEST OF TECHNICAL PAPERS (ISSCC), 2010 IEEE INTERNATIONAL. IEEE, 2010

Cited By (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102016011913A1 (en) 2016-10-05 2018-04-05 Hensoldt Sensors Gmbh Detector unit and a method for detecting an optical detection signal
CN108089697A (en) * 2016-11-21 2018-05-29 三星电子株式会社 The sensor based on event including power control circuit
CN108089697B (en) * 2016-11-21 2023-05-09 三星电子株式会社 Event-based sensor including power control circuitry
EP3386191A1 (en) * 2017-04-07 2018-10-10 Commissariat à l'Energie Atomique et aux Energies Alternatives Array sensor with time coding without arbitration
FR3065133A1 (en) * 2017-04-07 2018-10-12 Commissariat A L'energie Atomique Et Aux Energies Alternatives MATRIX SENSOR WITH TEMPORAL CODING WITHOUT ARBITRATION
US10341587B2 (en) 2017-04-07 2019-07-02 Commissariat A L'energie Atomique Et Aux Energies Alternatives Matrix-array sensor with temporal coding without arbitration
CN107635086A (en) * 2017-10-25 2018-01-26 深圳地平线机器人科技有限公司 Solid state image sensor and image-sensing method
CN107635087A (en) * 2017-11-08 2018-01-26 深圳地平线机器人科技有限公司 Solid state image sensor and image-sensing method
CN107635087B (en) * 2017-11-08 2024-05-07 深圳地平线机器人科技有限公司 Solid-state image sensor and image sensing method
CN111713101B (en) * 2017-12-11 2022-05-13 普罗菲西公司 Event-based image sensor and method of operating the same
CN111713101A (en) * 2017-12-11 2020-09-25 普罗菲西公司 Event-based image sensor and method of operating the same
EP3967027B1 (en) * 2019-05-10 2023-10-25 Sony Advanced Visual Sensing AG Dynamic vision sensor with in-pixel digital change detection
CN110677603A (en) * 2019-09-16 2020-01-10 上海集成电路研发中心有限公司 Image sensor for single-frame segmented exposure
CN110677603B (en) * 2019-09-16 2021-09-14 上海集成电路研发中心有限公司 Image sensor for single-frame segmented exposure and data acquisition method
CN110708481A (en) * 2019-09-16 2020-01-17 上海集成电路研发中心有限公司 Fixed pattern noise removing method based on difference reset
WO2021260121A1 (en) 2020-06-26 2021-12-30 Alpsentek Gmbh Delta image sensor with digital pixel storage with event output with magnitude and direction
WO2021260105A1 (en) 2020-06-26 2021-12-30 Alpsentek Gmbh Delta image sensor with digital pixel storage using different ramps for storage and comparison
WO2021260102A1 (en) 2020-06-26 2021-12-30 Alpsentek Gmbh Delta image sensor with digital pixel storage
WO2021260116A1 (en) 2020-06-26 2021-12-30 Alpsentek Gmbh Delta image sensor with externally writeable digital pixel storage
WO2021260122A1 (en) 2020-06-26 2021-12-30 Alpsentek Gmbh Delta image sensor with digital pixel storage with long term interruptable operation
TWI810593B (en) * 2020-06-26 2023-08-01 瑞士商阿爾普勝技術有限責任公司 Delta image sensor with digital pixel storage and synchronised sampling
WO2021260106A1 (en) 2020-06-26 2021-12-30 Alpsentek Gmbh Delta image sensor with digital pixel storage
TWI821697B (en) * 2020-06-26 2023-11-11 瑞士商阿爾普勝技術有限責任公司 Delta image sensor with digital pixel storage using different ramps for storage and comparison
EP3930312A1 (en) * 2020-06-26 2021-12-29 Alpsentek GmbH Delta image sensor with digital pixel storage
US11962926B2 (en) 2020-08-14 2024-04-16 Alpsentek Gmbh Image sensor with configurable pixel circuit and method

Similar Documents

Publication Publication Date Title
WO2014174498A1 (en) A novel cmos image sensor with event/change detection and reduced data redundancy
US10547802B2 (en) Solid-state image pickup device and control method therefor, and electronic apparatus
US9197233B2 (en) Low power ADC for high dynamic range integrating pixel arrays
US8619168B2 (en) Image sensor with high dynamic range imaging and integrated motion detection
US6525304B1 (en) Circuitry for converting analog signals from pixel sensor to a digital and for storing the digital signal
JP4389190B2 (en) CMOS active pixel sensor with in-pixel local exposure control function
US6670904B1 (en) Double-ramp ADC for CMOS sensors
US8111312B2 (en) Solid-state imaging device, method of driving the same, and camera
US8586903B2 (en) Counter circuits, analog to digital converters, image sensors and digital imaging systems including the same
US20030227040A1 (en) Image sensor
US6707410B1 (en) Digital pixel sensor with a dynamic comparator having reduced threshold voltage sensitivity
WO2013131329A1 (en) Low power consumption digital domain accumulating cmos-tdi image sensor
US20170208279A1 (en) Read-out circuitry for an image sensor
US20190246049A1 (en) System and methods for dynamic pixel management of a cross pixel interconnected cmos image sensor
US8450673B2 (en) Pixel circuit, imaging integrated circuit, and method for image information acquisition
JPH08181920A (en) Active picture element sensor having difference mode and imaged system
US9848154B2 (en) Comparator with correlated double sampling scheme and operating method thereof
JP4785848B2 (en) Auto zoom tilt AD converter
JP2019526992A (en) Differential digital double sampling method and CMOS image sensor for performing the method
CN113091900A (en) Reading circuit and infrared imager
US20200021768A1 (en) Image sensor and pixel array circuit thereof
Lenero-Bardallo et al. A high dynamic range image sensor with linear response based on asynchronous event detection
US8502724B2 (en) Semiconductor device and method for driving the same
KR20160119587A (en) DDS Global Counting Apparatus Using Counting Mask and CMOS Image Sensor Thereof
Choi et al. A spatial-temporal multi-resolution CMOS image sensor with adaptive frame rates for moving objects in the region-of-interest

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14730198

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14730198

Country of ref document: EP

Kind code of ref document: A1