EP2957097A2 - Capteur d'images cmos à plage dynamique étendue ayant des propriétés anti-éblouissement, et procédés associés - Google Patents

Capteur d'images cmos à plage dynamique étendue ayant des propriétés anti-éblouissement, et procédés associés

Info

Publication number
EP2957097A2
EP2957097A2 EP14752193.4A EP14752193A EP2957097A2 EP 2957097 A2 EP2957097 A2 EP 2957097A2 EP 14752193 A EP14752193 A EP 14752193A EP 2957097 A2 EP2957097 A2 EP 2957097A2
Authority
EP
European Patent Office
Prior art keywords
row
reset
readout
image
rows
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP14752193.4A
Other languages
German (de)
English (en)
Other versions
EP2957097A4 (fr
Inventor
Jiang JUTAO
Matt BORG
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
SiOnyx LLC
Original Assignee
SiOnyx LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by SiOnyx LLC filed Critical SiOnyx LLC
Priority claimed from PCT/US2014/016979 external-priority patent/WO2014127376A2/fr
Publication of EP2957097A2 publication Critical patent/EP2957097A2/fr
Publication of EP2957097A4 publication Critical patent/EP2957097A4/fr
Withdrawn legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
    • H—ELECTRICITY
    • H01—ELECTRIC ELEMENTS
    • H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
    • H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
    • H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
    • H01L27/144—Devices controlled by radiation
    • H01L27/146—Imager structures
    • H01L27/14643—Photodiode arrays; MOS imagers
    • H01L27/14654—Blooming suppression
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/50—Control of the SSIS exposure
    • H04N25/57—Control of the dynamic range
    • H04N25/58—Control of the dynamic range involving two or more exposures
    • H04N25/587—Control of the dynamic range involving two or more exposures acquired sequentially, e.g. using the combination of odd and even image fields
    • H04N25/589—Control of the dynamic range involving two or more exposures acquired sequentially, e.g. using the combination of odd and even image fields with different integration times, e.g. short and long exposures
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/62—Detection or reduction of noise due to excess charges produced by the exposure, e.g. smear, blooming, ghost image, crosstalk or leakage between pixels
    • H04N25/621—Detection or reduction of noise due to excess charges produced by the exposure, e.g. smear, blooming, ghost image, crosstalk or leakage between pixels for the control of blooming
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/62—Detection or reduction of noise due to excess charges produced by the exposure, e.g. smear, blooming, ghost image, crosstalk or leakage between pixels
    • H04N25/621—Detection or reduction of noise due to excess charges produced by the exposure, e.g. smear, blooming, ghost image, crosstalk or leakage between pixels for the control of blooming
    • H04N25/623—Detection or reduction of noise due to excess charges produced by the exposure, e.g. smear, blooming, ghost image, crosstalk or leakage between pixels for the control of blooming by evacuation via the output or reset lines

Definitions

  • CMOS image sensors tend to have a limited dynamic range due to well capacity limitations.
  • Various approaches have been attempted to improve dynamic range using such sensors, such as, for example, multiple exposure, lateral overflow, logarithmic pixel, in-pixel delta-sigma ADC, pixel with multiple-size- photodiodes, pixel arrays with different neutral density filters, dual conversion gain, etc.
  • the multiple exposure approach is one common technique for high dynamic range (HDR) imaging.
  • pixel integration time is divided into two or more segments. In each segment, the effective pixel well capacity is varied. At the end of integration, the total accumulated charge is readout.
  • this approach normally has fixed pattern noise at each knee point (the time at which an intermediary reset voltage is applied) on the signal response curve. Lateral overflow approaches also tend to be flexible regarding dynamic range extension.
  • Delta-sigma ADC Analog to Digital Converter
  • ADC Analog to Digital Converter
  • each pixel's integration time is individually controlled. This approach in theory can provide the best HDR scene capture and is the most flexible to achieve the highest possible dynamic range.
  • the pixel size is large, complex and not appealing for most consumer applications.
  • each pixel has multiple photodiodes.
  • the effective sensitivity of two photodiodes can thus be made different by design.
  • the enhanced dynamic rage is fixed in design and it is not flexible.
  • FIG. 1 is a schematic diagram of a four transistor CMOS image sensor pixel in accordance with one aspect of the present disclosure.
  • FIG. 2 is a depiction of a method of providing blooming protection in accordance with another aspect of the present disclosure.
  • FIG. 3 is a depiction of a method of providing blooming protection in accordance with another aspect of the present disclosure.
  • FIG. 4 is a graphical representation of a 2-pointer HDR image sensor readout and shutter timing scheme, in accordance with another aspect of the present disclosure.
  • FIG. 5 is a graphical representation of a 2-pointer FfDR image sensor readout and shutter timing scheme, in accordance with another aspect of the present disclosure.
  • FIG. 6 is a graphical representation of a timing diagram for a rolling shutter readout scheme, in accordance with another aspect of the present disclosure.
  • the term “substantially” refers to the complete or nearly complete extent or degree of an action, characteristic, property, state, structure, item, or result.
  • an object that is “substantially” enclosed would mean that the object is either completely enclosed or nearly completely enclosed.
  • the term "about” is used to provide flexibility to a numerical range endpoint by providing that a given value may be “a little above” or “a little below” the endpoint.
  • the presently disclosed technology provides systems, methods, and devices for use in capturing high dynamic range (HDR) images using CMOS imagers. More specifically, blooming protection is provided through the strategic use of shuttering, particularly between images captured at different integration times. These novel techniques can reduce blooming issues that apply to any row-wise HDR CMOS imager such as, for example, rolling shutter CMOS imagers.
  • CMOS image sensors generally include an array of pixels arranged in rows and columns. Each pixel can generally include a photodiode and a transfer gate, which is used in image readout to control the transfer of collected charge from the photodiode. Imagers can further include a reset gate used to reset or shutter a plurality of pixels (i.e. one or more rows of pixels in some aspects), a row select transistor, among others. It is noted that CMOS imagers are well known in the art, and as such any known component or design incorporated into an imager using a rowwise readout scheme should be considered to be within the present scope. As examples, CMOS imagers having four transistor (4T), 5T, or more can be utilized.
  • Such a device can include a photodiode 102, a transfer transistor 104, a floating diffusion region 106, a reset 108, source follower 110, a row select 112, a power supply voltage for the pixel array 114, and the voltage out 116.
  • Row-wise readout also known as rolling shutter
  • the rolling shutter is an electronic shutter that operates at the level of the pixel, and rolls along the pixel array row-by-row.
  • a reset voltage is applied to the pixel via a reset transistor and a voltage is applied to the transfer gate, thus clearing the contents of the photodiode of the pixel and placing the pixel in a reset state. While in the reset state, the pixel does not accumulate charge from incoming photons. Once the reset voltage and the transfer gate voltage has been removed, the pixel is capable of absorbing incoming photons and accumulating the resulting charge.
  • rows of pixels in the imager can be reset in sequence, starting, for example, at the top of the array and proceeding row-by-row to the bottom of the array. It is noted, however, that rolling shutter processes can move from any point in the array to another, and that such is merely exemplary.
  • charge integration begins.
  • the readout operation begins. Rows can be read out of the pixel array in the order of the rolling shutter to maintain a constant integration time for each row.
  • the exposure time for the pixel array is controlled by the timing difference between the passing of the rolling shutter and readout of the rows of the array. This exposure time is often referred to as the integration time. Increasing the integration time increases the duration that the pixels are accumulating charge, while decreasing the integration time decreases charge accumulation.
  • a pixel has a limited charge-well capacity, and thus can only accumulate a fixed amount of charge. When this well is full, and particularly then the pixel continues to convert photons to charge, excess charge can spill over into neighboring pixels, thus corrupting the associated charge concentration in those pixels. This negative effect can also spill over into pixels multiple rows away, depending on the relative charge between the source of the blooming and the affected rows, the pixel size, the epitaxial layer thickness, etc. This can be particularly problematic for imaging scenes with bright light on a dark background, such as, for example, car headlights in the dark. Such blooming thus spreads, causing unwanted noise throughout at least portions of the image.
  • HDR images captures a greater dynamic range between the lightest and darkest regions of a given scene as compared to normal imaging technologies.
  • Common HDR scenes occur when there is a large contrast of light intensity between objects in the scene, such as illumination of a car's headlights at night.
  • multiple exposure HDR schemes multiple images are taken at different integration times for each frame for the same scene. Afterwards, the multiple images are combined to reconstruct the final image.
  • the multiple exposure HDR schemes can be frame-wise or row-wise.
  • a frame-wise HDR scheme two (or more) frames are readout sequentially, meaning the whole focal plane array is read out at least twice and combined to create the HDR image.
  • the first frame will have a predetermined integration time, also known as exposure time, and the second frame will have a predetermined integration time.
  • the frame for capturing the low lighting intensities in a scene will have a longer exposure or integration time than the frame capturing regular lighting intensities or high intensities of a given scene.
  • every row is read out multiple times in a single frame image, in which each readout has an integration time, which can, in some cases, be different integration times.
  • each row is read out three times at different integration times.
  • the raw image data from a row-wise HDR is separated into individual frame data first.
  • Both frame-wise and row-wise HDR approaches are flexible regarding extended dynamic range and can be applied to most CMOS imager with minimal design changes. However, both designs can utilize on- chip or off-chip memory to store the image data before final image re-construction.
  • One implementation of a row-wise HDR imaging scheme initiates a first image process having a longer integration time exposure using a rolling shutter, with a second image process having a shorter integration time being initiated following the readout of the first image.
  • a rolling shutter one row wide rolls across a pixel array, initiating integration.
  • each row is read out and reset with a second rolling shutter, again 1 row wide, that initiates integration of the pixels for a shorter 3 row integration time.
  • rows that have yet to be read out for the long integration time may have wells with sufficient charge to bloom, and as such can cross the second rolling shutter prior to readout of the row, causing corruption of the charge in the pixels undergoing short integration.
  • the present technology reduces or eliminates such undesirable blooming through the strategic use of shuttering to limit the blooming across the rows that are being reset or shuttered following readout of the long integration time image.
  • Such strategic shuttering can be accomplished in a variety of ways, such as, for example, variation in the number of rows shuttered, the pattern of the rows shuttered, temporal variations in the shuttering voltage, and the like.
  • the number of shuttered rows can be increased that are directly adjacent to one another.
  • readout of the long integration time is followed up by a shutter that is 2, 3, 4, or more rows wide. Following this extended shutter, the reset is removed and the pixels are allowed to integrate for the short integration time.
  • Charge from pixels having full charge wells that have not yet been read out in the long integration time region is thus precluded or limited from crossing the multiple row shutter, or in other words, the multiple rows that are in the reset state.
  • a given row is read out for the long integration time image, followed by a reset that is present for the next 3 row readouts.
  • the reset of the given row is then release, and the short integration time is initiated.
  • the last row read out is shuttered and the 4-row rolling shutter moves across the pixel array in a row-wise manner, thus providing bloom protection to the shorter integration time region. Note that the beginning of the short integration time may be delayed compared to a traditional rolling shutter due to the width of the reset.
  • the present technology is not limited to HDR imaging having only two integrations times, but the same or similar imaging process can be applied for a third, fourth, or more images having different integration times.
  • the row when the given row is read out for the second integration time, the row can be reset with a shutter having characteristics to provide bloom protection to a third integration image, and so on.
  • a variety of shutter configurations and shutter behaviors are contemplated, and can include any shutter scheme that provides blooming protection to HDR images.
  • the shutter has at least two rows of pixels that are simultaneously shuttered to provide the blooming protection.
  • the shutter can be applied across 2, 3, 4, 5, or more directly adjacent rows.
  • the shutter can be applied across 2, 3, 4, 5, or more rows that are not directly adjacent, or that contain directly adjacent rows and non-directly adjacent rows in the same shutter.
  • a 4 row shutter can be applied to rows 10, 11, 12, and 13, or a 4 row shutter can be applied to rows 10, 12, 14, and 16, or to rows 10, 13, 14, and 17, or any other combination that facilitates blooming protection.
  • the chance of blooming can vary depending on the relative charge between the source of the blooming and the affected rows, the pixel size, the epitaxial layer thickness, etc.
  • the shutter configurations can be designed to account for such factors.
  • the present technology can utilized epitaxial layer thicknesses of from about 1 to about 10 microns, and can have pixel sizes of from about 0.9 microns to about 30 microns, or from about 0.9 microns to about 6 microns, or from about 0.9 microns to about 3 microns, to name a few non- limiting examples.
  • the shutter can be up to 30 rows or more in some cases. It is, however, within the skill of those in the art to readily calculate the necessary shutter durations to account for these variations, once in possession of the present disclosure.
  • a device can have a fixed shutter scheme.
  • a shutter may be fixed at a 4 row width in a device no matter what the imaging conditions.
  • the shutter in a device can be manually set by a user for a given image or image conditions.
  • the device can include an automatic mode, where the shutter characteristics are automatically set to match lighting conditions.
  • hybrid approaches are contemplated whereby a user sets the device to behave in a desired manner, and the device automatically sets or varies the shutter characteristics to optimal or near optimal conditions within the parameters set by the user.
  • the shutter can additionally be varied within a given frame or image process.
  • the device can increase the duration of the shutter between a long integration time process and a medium integration time process, but decrease the duration of the shutter between the medium integration time process and a shorter integration time process.
  • the duration of the shutter can be increased during HDR image capture in between the image sections that may have blooming issues, but decreased in between image sections that are not as likely to experience blooming.
  • the device can contain logic to dynamically increase the shutter duration or to apply a given shutter pattern when blooming is detected or when a scene is detected where blooming may occur. So, the logic can dynamically adjust the shutter in between integration time images, or in some aspects during a given rolling shutter operation for a given integration time. If an image is being read out for a longer integration time and a potential for blooming is detected, for example, the logic can adjust the shutter to limit or preclude blooming from occurring.
  • the strength (i.e., the voltages applied) of the reset can be constant throughout the duration of a given shutter, while in other aspects the shutter strength can be variable.
  • one energy saving technique may be to hard reset a row as the shutter is applied, lower the reset voltage during the middle of the shutter, and hard reset the row again just prior to the initiation of the next integration cycle.
  • Variations in the shutter strength can be fixed in the device, set by the user, and/or dynamically controlled by the device. In the dynamically controlled case, the logic of the device can be utilized to detect blooming or the potential for blooming, and can thus increase the strength of the reset during such blooming periods and decrease the strength when blooming potential is lower.
  • one exemplary method provides blooming to a CMOS imager having a pixel array of a plurality of pixels arranged in rows and columns, where the CMOS imager is operable to capture high dynamic range images using a rolling shutter.
  • the method can include 202 selecting a readout row of pixels, 204 starting a first integration time of the pixels in the readout row, and 206 reading out charge accumulated by the pixels in the readout row to obtain a first readout.
  • the method also includes 208 applying a reset to the readout row for a reset time sufficient to allow readout and reset to occur in at least one subsequent row, 210 removing the reset and starting a second integration time of the pixels in the readout row, wherein the second integration time is shorter than the first integration time, and wherein the at least one subsequent row is a sufficient number of rows to have a combined reset to at least substantially preclude blooming effects from the pixel array in the readout row during the second integration time, and 212 reading out charge accumulated by the pixels in the readout row to obtain a second readout.
  • a subsequent row is a row that is read out or otherwise processed following the readout row.
  • applying a reset to the readout row for a reset time sufficient to allow readout and reset to occur in at least one subsequent row includes allowing the reset to continue on the reset row to at least give sufficient time for at least one subsequent row to be read out and reset.
  • the reset can be applied for the reset time sufficient such that at least two subsequent rows and the readout row are simultaneously reset or in a reset state.
  • the reset can be applied for the reset time sufficient such that at least three subsequent rows and the readout row are simultaneously reset or are in a reset state.
  • the reset time sufficient to allow readout and reset to occur in at least one subsequent row can be from about 10 nanoseconds to about 50 microseconds.
  • the reset time sufficient to allow readout and reset to occur in at least one subsequent row can be from about 0.5 microseconds to about 2 microseconds. Understandably, these reset times can vary depending on the duration of the integration and the degree of potential blooming in the array.
  • the first integration time can be from about 1 millisecond to about 1 second and the second integration time can be from about 10 nanoseconds to about 100 milliseconds.
  • the first integration time can be from about 20 microseconds to about 33 milliseconds and the second integration time can be from about 1 microsecond to about 16 milliseconds.
  • the present scope should also include shuttering or reset schemes that are sequential, non-sequential, adjacent, and non-adjacent.
  • the readout row and the at least three subsequent rows can be sequentially adjacent.
  • the readout row and the at least three subsequent rows can be sequentially non-adjacent. The same would thus also apply to any number of subsequent rows greater than one (i.e., greater than two rows, the readout row and the at least one subsequent row).
  • the method can be repeated for subsequent rows.
  • the method can be repeated on at least one subsequent row, at least two subsequent rows, at least three subsequent rows, and so on.
  • the at least one subsequent row is at least substantially all pixel rows in the pixel array.
  • the rows can be read out in various orders.
  • the method can be repeated on at least substantially all pixel rows in the pixel array in a sequential order. Such orders can include directly adjacent rows as a sequence, alternating rows as a sequence, and the like.
  • the method can be repeated on at least substantially all pixel rows in the pixel array in a non-sequential order.
  • the present technology encompasses a third, fourth, or more integration times utilized to produce an HDR image.
  • the method can further include applying the reset to the readout row following the second readout for a reset time sufficient to allow readout and reset to occur in at least one subsequent row, removing the reset and starting a third integration time of the pixels in the readout row, wherein the third integration time is shorter than the second integration time, and wherein the at least one subsequent row is a sufficient number of rows to have a combined reset to at least substantially preclude blooming effects from the pixel array in the readout row during the third integration time, and reading out charge accumulated by the pixels in the readout row to obtain a third readout. Similar steps can be accomplished for a fourth or more integration time.
  • first and second integration time may represent the first and second integrations of the image, or they may represent the third and fourth, or the second and third, or any other pair of integration operations to which blooming protection applies.
  • starting the first integration time can include specifically applying the reset to the readout row and releasing the reset from the readout row.
  • integration can be initiated by the readout and reset of the readout row from the previous integration.
  • the method can also include applying the reset at a continuous voltage level throughout the reset time duration.
  • the reset can be applied at a variable voltage level throughout the reset time duration.
  • the reset can be maintained on rows six, seven, eight, and then applied to nine, or the reset can be reapplied to rows six, seven, eight, and then applied to nine. It is thus non- limiting as to how the shutter is applied and/or maintained, provided sufficient shutter is in place to prevent or limit blooming from corrupting across the shutter.
  • the method can additionally further include, in some aspects, the formation of a HDR image from the individual images derived from the different integrations, in some cases at different integration times.
  • the first readout and the second readout can be combined to form a high dynamic range image.
  • the various integration times for each row can be combined to form the resulting image.
  • the row data from the various integration times can be formed into discrete images or image data sets and then the images or image data sets from each integration can be combined to form the HDR image.
  • two integration images, three integration images, four integration images or more can be combined into the HDR image. In some cases, all integration images are combined, while in other cases only a subset of integration images are combined.
  • a method of using a rolling shutter to provide blooming protection in a CMOS imager in high dynamic range mode and having a pixel array of a plurality of pixels arranged in rows and columns is provided.
  • Such a method can include 302 capturing a first image in the pixel array having a first integration time, 304 reading the first image out of the pixel array sequentially by rows using at least one readout row of the first image, and 306 hard resetting a plurality of rows in proximity to the at least one readout row of the first image.
  • the method also includes 308 capturing a second image in the pixel array having a second integration time, the second integration time being shorter than the first integration time, wherein the second image is protected from blooming caused by the first image by the hard resetting of the plurality of rows in proximity to the readout row of the first image, and 310 reading the second image out of the pixel array sequentially by rows using at least one readout row of the second image.
  • the present method is not limited to two images, nor should the terms "first" and "second” be construed as necessarily the first and second images taken, but rather describe an order the images are captured.
  • the method can further include hard resetting a plurality of rows in proximity to the at least one readout row of the second image, capturing a third image in the pixel array having a third integration time, the third integration time being shorter than the second integration time, wherein the third image is protected from blooming caused by the first or the second image by the hard resetting of the plurality of rows in proximity to the readout row of the second image, and reading the third image out of the pixel array sequentially by rows using at least one readout row of the third image.
  • similar processing can be utilized to generate a fourth, fifth, sixth, or more images for HDR purposes.
  • the first image and the second image can be combined to form a HDR image.
  • the first image, the second image, and the third image can be combined to form a HDR image.
  • any number of images taken can be combined, either all together or a particular subset thereof, to form a HDR image.
  • CMOS processing components, and methodologies that may be useful in the practice and/or
  • CDS correlated double sampling
  • CMOS pixel a technology that can be utilized during a readout process
  • CDS is a technique for measuring a signal that allows for removal of an undesired offset and low frequency temporal noise (e.g. kTC noise for CMOS pixel).
  • the output signal is thus measured twice: once in a known condition and once in an unknown condition.
  • the value measured from the known condition is then subtracted from the unknown condition to generate a value with a known relation to the physical quantity being measured.
  • a CDS operation can be performed on a row that is being read out from the imager.
  • a rolling shutter readout operation is shown, as a timing diagram, in FIG. 6.
  • the integration time is 1 row.
  • the floating diffusion (FD) is reset with the reset signal (RST), and the signal from the FD will be sampled with the SHR signal.
  • RST reset signal
  • TX will activate for charge transfer, and the signal will then be sampled with the SHS pulse.
  • the difference between the SHR and SHS signals will be used for the final signal, which will be CDS mode.
  • a shutter operation can be performed in which both RST and TX will be pulsed for the shutter row, or rows in some cases. This readout operation can then be repeated for other rows in the array.
  • a CDS operation can include setting the reset select transistor of the row being read to a high state and setting the power supply to a high power supply setting to set the floating diffusion regions to the high power supply setting in the row being read.
  • the power supply can be increased from 2.8 V to a high power supply setting of 3.1 V.
  • the electrical charge in the floating diffusion regions can then be read to obtain a first electrical charge value.
  • the transfer transistor of the row being read can then be set to a high state to transfer electrical charge in the photodiode to the floating diffusion regions of the row being read. Subsequently, the electrical charge in the floating diffusion regions can be read to obtain a second electrical charge value. The second charge value can then be subtracted from the first electrical charge value to obtain the CDS sampled output.
  • CMOS pixel array In rolling shutter mode, a CMOS pixel array is read out essentially a row at a time, although in some aspects more than one row may be read out simultaneously or in an overlapping fashion.
  • r c represents the readout row, or in other words, the pointer to the readout row.
  • the readout can be accomplished by any readout technique.
  • CDS mode readout which allows for removal of an undesired offset and/or temporal noise associated with reset of floating diffusion (FD).
  • CDS readout mode can be signified in the following equations by the term r CDS , which represents the readout row being read out in CDS mode.
  • r CDS which represents the readout row being read out in CDS mode.
  • shutter refers to a shutter operation or a resetting operation of the pixels.
  • shutter and “reset” can be used interchangeably, unless the context clearly indicates otherwise.
  • a reset can thus occur in a selected pixel or pixel row or rows to produce a "reset state,” or in other words a state in which the pixel is not in a charge accumulation or integration state.
  • a reset state is a state in which the pixel is not accumulating charge for the purpose of readout for imaging.
  • the reset is a reset voltage that is applied to the pixel to remove accumulated charge and to prevent or reduce further accumulation.
  • the reset voltage can be applied and/or maintained at different levels, depending on the voltage applied and/or maintained in the pixel. So, for example, a complete reset can be applied to the pixel to at least substantially erase the accumulated charge and, if maintained in the pixel, to prevent or significantly prevent further charge accumulation.
  • This initial strong or sharp voltage application can be referred to herein as a "hard reset.”
  • a lower voltage can be applied to and/or maintained in the pixel if desired. It is noted that in some cases a pixel can accumulate charge when in a reset state; however such charge accumulation is not a purposeful charge accumulation for readout of an image.
  • a shutter or reset voltage can be applied to a row of pixel to place the row in a reset state.
  • the reset voltage is removed, the row of pixels can then begin accumulating charge from incoming light and is thus in an integration state.
  • r hutter is used to refer to the physical row index of a pixel row that is being shuttered for the i th pointer operation.
  • t ⁇ nt refers to the first pointer's integration time in units of row(s) and " £ int” refers to i th pointer's integration time in units of row(s).
  • a given row is reset and allowed to integrate charge over a given integration time.
  • row r c + r shutter can t> e hard reset, where r shutter is at least one row, but can also include multiple rows in the imager array.
  • blooming protection can be provided to HDR capture through the strategic use of a shutter in between at least two of the images used in the HDR imaging process. Such blooming protection thus effectively minimizes or prevents the crossover or blooming of charge from one region of the array to another.
  • the term "p" is used to describe the blooming protection row count (or purge row count) in numbers of rows that is provided by this strategic shutter. It is noted that the rows may or may not be sequential, that various numbers of rows or patterns of rows can be used as blooming protection, and that such variations are intended to be within the present scope. Those of ordinary skill in the art would recognize appropriate variations in the equations and methodologies disclosed herein to account for such variations in design. In one aspect, however, p can be greater than one row, or more. Any number of rows or patterns of rows that can reduce or eliminate blooming are contemplated and included in the present scope.
  • Equation (3) a shutter is applied to a row or rows of the array to provide protection from blooming.
  • a larger-than-l-row purge row count can enable the HDR imager device to have reduced blooming effects.
  • This blooming purge count p provides additional shifts between different pointer operation regarding readout and shutter row index. The shift can be adjusted to achieve the proper anti-blooming properties given a particular scene.
  • the total shutter count of p times is allowed, and in this case provides the blooming protection.
  • a (p+1) shutter can be executed sequentially after a readout such as CDS.
  • the valid row address starts at row 1 and will end at row N.
  • the physical row readout for the first pointer will start on row 1 (CDS: 1). Since the first integration time is 100 row units, row 101 is shuttered (S: 101) as row 1 is read out. It is noted in the timing scheme another row (CDS: -2) is also read out. Since the physical row -2 is non-existent no row data will be readout. Once read out, row 1 is shuttered (S: l) in preparation for the second integration.
  • the 2 nd physical row (CDS: 2) is read out in CDS mode. Since the integration time is 100 rows, row 102 is shuttered (S: 102) and row 101 begins integration. Row 2 is shuttered (S:2) in preparation for the second integration of row 2, and the shutter is removed from row 1, which begins the second integration (tf nt ). Because the shutter is only one row wide, blooming can cross into the second integration time.
  • the readout sequence for the first integration or the 1 st pointer for row 1 is: CDS: 1 ; Shutter: 101, CDS: -5; Shutter: -2, - 1, 0, 1. Since physical row index starts at 1, the CDS of row -5 will not produce any data since -5 is not a valid physical row address. For the 2 nd pointer shutter operation, a total of 4 rows are being reset: -2, -1, 0, 1.
  • the resultant terms will be CDS: 7; Shutter: 107, 108, 109, 110; CDS: 1, Shutter: 4, 5, 6, 7.
  • the 1 st one (for row 4) is used to start integration of row 4 for the 2 nd pointer. Additional bloom purge shutter 5, 6, 7 are used to provide blooming protection as has been described.
  • rows 5, 6, and 7 are shuttered (reset), they change from a saturation state to an unsaturation state.
  • rows 5, 6, and 7 will not reach saturation and blooming during row 4's integration for 2 nd pointer or second integration time.
  • the integration for row 4 will not be corrupted by blooming.
  • sequential shutters can include 4, 5, or 4, 5, 6 in this case.
  • additional blooming purge shutters can provide better blooming protection.
  • CMOS imager design a register can be used for additional bloom purge shutter count.
  • the value of this blooming purge count can be controlled by the user directly or other by image processing algorithms that detect blooming issues on the image.
  • the row driver control logic can also be designed based on Equations (l)-(4) as shown above.
  • the additional bloom purge shutter can be applied sequentially to each pointer's shutter operation.
  • the application of the additional bloom purge shutter should apply only to a particular pointer operation. This can be beneficial if the additional blooming purge shutter has a negative impact on frame rate. For example, for a 2-pointer row-wise HDR image sensor, if 100 row, and 3 row, adding additional bloom purge shutter for the 1 st pointer operation might not be necessary since the integration time is already 100 rows.
  • the additional purge shutter can be applied to the 2 nd pointer in order to correctly expose the brighter portions of a scene.
  • the additional blooming purge shutter count of 3 can be applied only to the 3 rd pointer shutter operation.
  • Equations (5)-(10) 3-pointer row-wise image sensor readout and shutter algorithms for a third integration time are shown in Equations (5)-(10). Similar equations can readily be derived therefrom for fourth, fifth, and greater integration times.
  • the resultant parameters are the following: (CDS: 4; Shutter: 104; CDS: 1; Shutter: 4).
  • the raw data from row 4 is read out; then a shutter is applied to rows 104, 105, 106, and 107; data from row 1 is read out, noting that the data readout of row 1 is after a 3 row integration time; and subsequently a shutter operation is applied to rows 4, 5, 6 and 7. It should be pointed out that row 4 will not be read out again until row 7 is read out.
  • row 4 After readout of row 4 for the 1 st pointer, the image data will be in a row-wise interlaced pattern in which an odd row will be associated with one integration time and even rows will be associated with a different integration time.
  • row 4 starts integration for its second integration time. If a high intensity light spot is being integrated by rows 5, 6, and 7, the intensity of this light spot and can cause the pixel to reach saturation after about five (5) rows of integration time. Since rows 5, 6, and 7 are still in their first pointer operation (which has integration time of 100 rows), all those rows are saturated (since those rows previously integrated 99, 98, 97 rows already for physical rows 5, 6, 7, respectively) and start blooming into their neighboring rows.
  • an idle row's reset (RST) and transfer (TX) gate are set to high and remain high during the entire idle period to provide blooming protection.
  • An idle row refers to a physical row in the sensor array that is not in integration or in shutter operation. Additionally, idle rows could include all those rows that are outside the addressed image output windows.
  • the top 25 rows and the bottom 25 rows can be considered in an idle row operation or mode.
  • a latch-based row decoder design is typically required. Therefore, an image sensor running such "anti-blooming" timing cannot protect the rows in a second pointer from blooming corruption.
  • Various benefits of the present technology include the reduction in die size due to not needing a latch-based row driver, flexible to handle 2-pointer, 3 -pointer, or more pointer row-wise HDR operation, the reduction in hot pixel impact on final images since the pixels will reset multiple times in average, among other things.
  • the present scope additionally includes systems and devices utilizing the present technology.
  • Such should include any CMOS device, system, architecture, and/or design that allows HDR imaging using a rolling shutter.
  • CMOS device system, architecture, and/or design that allows HDR imaging using a rolling shutter.
  • those skilled in the art can readily design and produce systems and devices incorporating the present technology once in possession of the present disclosure. In some cases, this technology can be implemented into current HDR imagers using minimal row driver digital design modifications.
  • CMOS imager having a pixel array of a plurality of pixels arranged in rows and columns, where the CMOS imager is operable to capture high dynamic range images using a rolling shutter.
  • Such a method can include selecting a readout row of pixels, starting a first integration time of the pixels in the readout row, reading out charge accumulated by the pixels in the readout row to obtain a first readout, applying a reset to the readout row for a reset time sufficient to allow readout and reset to occur in at least one subsequent row, removing the reset and starting a second integration time of the pixels in the readout row, wherein the second integration time is shorter than the first integration time, and wherein the at least one subsequent row is a sufficient number of rows to have a combined reset to at least substantially preclude blooming effects from the pixel array in the readout row during the second integration time, and reading out charge accumulated by the pixels in the readout row to obtain a second readout.
  • the method of example 1 can be repeated on at least one subsequent row.
  • the at least one subsequent row is at least substantially all pixel rows in the pixel array.
  • the method of example 1 can be repeated on at least substantially all pixel rows in a sequential order.
  • the method of example 1 can be repeated on at least substantially all pixel rows in a non-sequential order.
  • the first readout and the second readout are combined to form a high dynamic range image.
  • the method of example 1 can further include applying the reset to the readout row following the second readout for a reset time sufficient to allow readout and reset to occur in at least one subsequent row, removing the reset and starting a third integration time of the pixels in the readout row, wherein the third integration time is shorter than the second integration time, and wherein the at least one subsequent row is a sufficient number of rows to have a combined reset to at least substantially preclude blooming effects from the pixel array in the readout row during the third integration time, and reading out charge accumulated by the pixels in the readout row to obtain a third readout.
  • the method can be repeated for a fourth or more integration time.
  • starting the first integration time further includes applying the reset to the readout row and releasing the reset from the readout row.
  • applying the reset to the readout row for the reset time sufficient to allow readout and reset to occur in at least one subsequent row includes applying the reset at a continuous voltage level throughout the reset time duration.
  • applying the reset to the readout row for the reset time sufficient to allow readout and reset to occur in at least one subsequent row includes applying the reset at a variable voltage level throughout the reset time duration.
  • the reset is applied for the reset time sufficient such that at least two subsequent rows and the readout row are simultaneously reset.
  • the reset is applied for the reset time sufficient such that at least three subsequent rows and the readout row are simultaneously reset.
  • the readout row and the at least three subsequent rows are sequentially adjacent.
  • the readout row and the at least three subsequent rows are sequentially non-adjacent.
  • the reset time sufficient to allow readout and reset to occur in at least one subsequent row is from about 10 nanoseconds to about 50 microseconds.
  • the first integration time is from about 1 millisecond to about 1 second and the second integration time is from about 10 nanoseconds to about 100 milliseconds.
  • a method of using a rolling shutter to provide blooming protection in a CMOS imager in high dynamic range mode and having a pixel array of a plurality of pixels arranged in rows and columns can include capturing a first image in the pixel array having a first integration time, reading the first image out of the pixel array sequentially by rows using at least one readout row of the first image, hard resetting a plurality of rows in proximity to the at least one readout row of the first image, capturing a second image in the pixel array having a second integration time, the second integration time being shorter than the first integration time, wherein the second image is protected from blooming caused by the first image by the hard resetting of the plurality of rows in proximity to the readout row of the first image, and reading the second image out of the pixel array sequentially by rows using at least one readout row of the second image.
  • the first image and the second image are combined to form a high dynamic range image.
  • the method can further include hard resetting a plurality of rows in proximity to the at least one readout row of the second image, capturing a third image in the pixel array having a third integration time, the third integration time being shorter than the second integration time, wherein the third image is protected from blooming caused by the first or the second image by the hard resetting of the plurality of rows in proximity to the readout row of the second image, and reading the third image out of the pixel array sequentially by rows using at least one readout row of the third image.
  • the first image, the second image, and the third image are combined to form a high dynamic range image.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Power Engineering (AREA)
  • Electromagnetism (AREA)
  • Condensed Matter Physics & Semiconductors (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)
  • Solid State Image Pick-Up Elements (AREA)

Abstract

La présente invention se rapporte à un procédé qui permet de protéger contre l'éblouissement un capteur d'images CMOS comportant un réseau de pixels constitué d'une pluralité de pixels disposés en rangées et en colonnes, le capteur d'images CMOS servant à capturer des images à plage dynamique étendue à l'aide d'un obturateur déroulant. Ce procédé peut comprendre la lecture de la charge accumulée par les pixels dans une rangée de lecture d'un premier temps d'intégration, l'application d'une réinitialisation sur la rangée de lecture pendant un temps de réinitialisation suffisant pour que la lecture et la réinitialisation se produisent dans au moins une rangée ultérieure, et le début d'un second temps d'intégration des pixels dans la rangée de lecture, le second temps d'intégration étant plus court que le premier temps d'intégration, et ladite rangée ultérieure correspondant à un nombre de rangées suffisant pour obtenir une réinitialisation combinée en vue d'éviter les effets d'éblouissement en provenance du réseau de pixels au cours du second temps d'intégration.
EP14752193.4A 2013-02-15 2014-02-18 Capteur d'images cmos à plage dynamique étendue ayant des propriétés anti-éblouissement, et procédés associés Withdrawn EP2957097A4 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201361765599P 2013-02-15 2013-02-15
PCT/US2014/016979 WO2014127376A2 (fr) 2013-02-15 2014-02-18 Capteur d'images cmos à plage dynamique étendue ayant des propriétés anti-éblouissement, et procédés associés

Publications (2)

Publication Number Publication Date
EP2957097A2 true EP2957097A2 (fr) 2015-12-23
EP2957097A4 EP2957097A4 (fr) 2016-07-27

Family

ID=54540268

Family Applications (1)

Application Number Title Priority Date Filing Date
EP14752193.4A Withdrawn EP2957097A4 (fr) 2013-02-15 2014-02-18 Capteur d'images cmos à plage dynamique étendue ayant des propriétés anti-éblouissement, et procédés associés

Country Status (1)

Country Link
EP (1) EP2957097A4 (fr)

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8013919B2 (en) * 2006-06-27 2011-09-06 Samsung Electronics Co., Ltd. CMOS image sensor with increased dynamic range based on multiple exposure periods of varying lengths
JP4243870B2 (ja) * 2007-02-08 2009-03-25 ソニー株式会社 固体撮像装置及び撮像装置
JP5163935B2 (ja) * 2007-05-17 2013-03-13 ソニー株式会社 イメージセンサ

Also Published As

Publication number Publication date
EP2957097A4 (fr) 2016-07-27

Similar Documents

Publication Publication Date Title
US9762830B2 (en) High dynamic range CMOS image sensor having anti-blooming properties and associated methods
US10091434B2 (en) System and method for capturing digital images using multiple short exposures
US10136090B2 (en) Threshold-monitoring, conditional-reset image sensor
US8981277B2 (en) High dynamic range image sensor with in pixel memory
EP3151546B1 (fr) Systemes d'imagerie a attenuation de papillotement et a grande gamme dynamique
US8873882B2 (en) Blooming filter for multiple exposure high dynamic range image sensors
US8514322B2 (en) Systems and methods for adaptive control and dynamic range extension of image sensors
EP3721616B1 (fr) Capteur d'image haute dynamique
US9282256B1 (en) System and method for HDR imaging
CA3082607A1 (fr) Attenuation de papillotement de del d'images animees
US20110050965A1 (en) Image capture device and control method thereof
KR20130008030A (ko) 디지털 촬상을 위한 상관해제된 채널 샘플링
JP2007174032A (ja) 撮像装置
EP2957097A2 (fr) Capteur d'images cmos à plage dynamique étendue ayant des propriétés anti-éblouissement, et procédés associés
JP2009159596A (ja) Ｃｃｄ信号処理装置および撮像装置
KR102106372B1 (ko) 이미징 장치 및 그 구동방법
KR102047136B1 (ko) 이미징 장치 및 그 구동방법
KR102141107B1 (ko) 이미징 장치 및 그 구동방법
JP2011142434A (ja) 固体撮像装置
Mochizuki et al. An ultra-high-speed compressive multi-aperture CMOS image sensor
JP2010171958A5 (fr)
Ogi et al. Smart image sensor for wide dynamic range by variable integration time
KR20140136291A (ko) 이미징 장치 및 그 구동방법
KR20140136287A (ko) 이미징 장치 및 그 구동방법
Faramarzpour et al. High resolution, high bandwidth global shutter CMOS area scan sensors

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20150904

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: SIONYX, LLC

DAX Request for extension of the european patent (deleted)
A4 Supplementary search report drawn up and despatched

Effective date: 20160628

RIC1 Information provided on ipc code assigned before grant

Ipc: H04N 5/359 20110101ALI20160622BHEP

Ipc: H04N 3/14 20060101AFI20160622BHEP

Ipc: H04N 5/353 20110101ALI20160622BHEP

Ipc: H04N 5/355 20110101ALI20160622BHEP

17Q First examination report despatched

Effective date: 20171115

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20180526