TW201929534A - Digital pixel image sensor - Google Patents

Abstract

Disclosed herein are techniques for digital imaging. A digital pixel image sensor includes a digitizer in each pixel of a plurality of pixels, where the digitizer digitizes analog output signals from a photodiode of the pixel using a comparator, a global reference ramp signal, and a clock counter. In some embodiments, the comparator includes a pre-charging circuit, rather than a constant biasing circuit, to reduce the power consumption of each pixel. In some embodiments, each pixel includes a digital or analog correlated double sampling (CDS) circuit to reduce noise and provide a higher dynamic range.

Description

Digital pixel image sensor

The present invention relates to a digital pixel image sensor, and more particularly to a digital pixel image sensor having a digitizer in each pixel.

Image sensors are used in many different applications. For example, an image sensor can be found in a digital imaging device (eg, a digital camera, a smart phone, etc.) for capturing digital images. In another example, in a virtual reality system, such as a virtual reality (VR) system, an augmented reality (AR) system, and a mixed reality (MR) system, The image sensor can be used to capture an image of the physical environment in which the user is located, and the captured image can then be used to control or influence the operation of the artificial reality system, such as controlling or affecting the display content of the artificial reality system. . For many of these applications, including artificial reality systems, image sensors with high speed, high sensitivity, high dynamic range, low noise, high density, high resolution, and low power consumption may be required.

The invention relates to a digital pixel image sensor. More specifically, the techniques disclosed herein relate to a digital pixel image sensor having a digitizer (eg, an ADC) in each pixel, wherein the digitizer utilizes a comparator, a reference ramp signal, and a counter. To digitize the analog output of the photodiode from the pixel. In some embodiments, each pixel may contain a digital or analog correlation double sampling (CDS) circuit to reduce noise and provide a higher dynamic range. In some embodiments, the comparator can include a pre-charge circuit instead of a constant bias circuit, thereby reducing the power consumption of each digital pixel.

In some embodiments, the digital pixel image sensor can include a plurality of pixels. Each pixel includes a photodiode for generating a charge in response to the optical signal, and a charge storage device for storing the charge generated by the photodiode, wherein the stored charge can generate a voltage signal on the charge storage device. Each pixel can also include a pixel memory and a digitizer. The digitizer can include a comparator for receiving the ramp signal and the voltage signal, wherein the voltage level of the ramp signal is increased or decreased after each cycle of the clock signal. The comparator can be further used to change the output state of the comparator after the voltage level of the ramp signal reaches the voltage level of the voltage signal. The digitizer may further include a digital output generating circuit for receiving a first quantity when the output state of the comparator changes, which corresponds to a time point when the ramp signal starts and a time point when the output state of the comparator changes. The total number of cycles of the clock signal, and the digital output generating circuit is configured to store the first quantity to the pixel memory, wherein the first quantity corresponds to a digital value generated by digitizing the voltage level of the voltage signal.

In some embodiments, the digital pixels of the image sensor may include a photodiode for generating a charge in response to the optical signal, and a charge storage device for storing the charge generated by the photodiode, wherein the stored charge A voltage signal can be generated on the charge storage device. Each pixel can also include a pixel memory and a digitizer. The digitizer can include a comparator for receiving the ramp signal and the voltage signal, wherein the voltage level of the ramp signal is increased or decreased after each cycle of the clock signal. The comparator can be further used to change the output state of the comparator after the voltage level of the ramp signal reaches the voltage level of the voltage signal. The digitizer may further include a digital output generating circuit for receiving a first quantity when the output state of the comparator changes, which corresponds to a time point when the ramp signal starts and a time point when the output state of the comparator changes. The total number of cycles of the clock signal, and the digital output generating circuit is configured to store the first number to the pixel memory, wherein the first number corresponds to the digitized digit value of the voltage level based on the voltage signal.

In some embodiments, a method of digital imaging is disclosed. The method can include receiving, by the photodiode of the pixel in the image sensor, the optical signal during the exposure period, and converting the optical signal into a voltage signal on the charge storage device of the pixel by the pixel. The method may further include starting a clock counter to calculate the number of clock cycles of the clock signal, and comparing the voltage signal and the ramp signal by a comparator of the pixel, wherein the voltage level of the ramp signal is linear with the number of clock cycles Increase or decrease. The method may also include changing, by the comparator, the output state of the comparator when the voltage level of the ramp signal reaches the voltage level of the voltage signal, and storing the clock period corresponding to the current pulse when the output state of the comparator changes. The first number of quantities to the pixel memory of the pixel is used as the first digit value of the voltage signal.

The Summary is not intended to identify key or essential features of the claimed subject matter, and is not intended to be used alone to determine the scope of the claimed subject matter. The subject matter should be understood by reference to the entire specification, any or all of the accompanying drawings and the appended claims. The foregoing, as well as other features and examples, are described in more detail in the description, claims, and drawings.

This article reveals image sensors with high speed, high resolution, high dynamic range, high sensitivity and low power consumption. In various embodiments, a digital pixel image sensor that includes a digitizer per pixel can be used to achieve the desired performance. In some embodiments, each digital pixel in the digital pixel image sensor can include a photodiode, a transfer gate, an analog storage device (eg, an explicit or parasitic capacitance), a digitizer (eg, an ADC), and a digital memory. . The light diode converts the optical signal into a signal. The transfer gate can be used to transfer an electrical signal (eg, accumulated charge) from the photodiode to an analog storage device, and the digitizer can convert the electrical signal at the analog storage device into a digital bit. The digital memory can store the digits before the digits are read. In an example, the digitizer in each digital pixel can include a comparator. The comparator can compare the electrical signal of the analog storage device with the reference ramp signal. The reference ramp signal can be, for example, a global signal generated by a DAC. A clock counter (such as a global clock counter) can continuously calculate the number of clock cycles in each video frame. When the reference ramp signal reaches the level of the electrical signal at the analog storage device, the output state of the comparator in the digital pixel can be changed (eg, flipped, toggled, or ramped up/down). The count value at the time point when the output of the comparator in the digital pixel changes state can then be latched in the memory of the digital pixel by switching the output of the comparator.

In some embodiments, the digital pixel may include a correlated double sampling (CDS) circuit (eg, a digital CDS circuit) to reduce random noise (eg, 1/f noise, thermal kT/C noise on the capacitor) And fixed pattern noise (FPN) (eg caused by a mismatch in comparator thresholds between pixels). The CDS circuit can be digitized at the reset level of the analog storage device (eg, to m bits) and the level of the electrical signal (eg, to n bits) of the analog storage device transferred from the photodiode. The two digitized values can be stored in the pixel memory of (n+m) bits. The difference between the two digitized values can be used as a digital output of a digital pixel of an image frame.

In one embodiment, for each image frame, the photodiode can be exposed to the optical signal and activated to integrate (integrate) the converted electrical signal. The analog storage device can be reset at the end of the exposure period (or integration period) or near the end (eg, about 100 microseconds ago). The reset level can be digitized by a digitizer. After the exposure period, the integrated electrical signal can be transferred from the photodiode to the analog storage device and digitized by the digitizer. Thus, for most of the frame period (eg, 33 milliseconds) (eg, 95%, 99%, or more), the digital pixels can operate in a low power mode (eg, an integration mode).

In some embodiments, the comparator of the digitizer can include circuitry capable of minimizing static (direct current) power consumption. For example, the precharge circuit can be used to precharge the internal node of the comparator to, for example, a low level before performing the digitization, instead of using a DC bias circuit to set the comparator to an active state. The pre-charging can be maintained for a short period of time, such as a few microseconds. During the remainder of the frame period, the comparator can have little or no static power. Therefore, the total power consumption of the comparator can be significantly reduced.

In the following description, for the purposes of illustration However, it will be apparent that various examples may be practiced without these specific details. For example, the devices, systems, structures, components, methods, and other elements may be presented in the form of blocks to avoid obscuring the examples in unnecessary detail. In other instances, well-known details of well-known devices, procedures, systems, structures, and techniques are not presented to avoid obscuring the examples. The drawings and the description are not intended to limit the disclosure. The use of the terms and expressions in the present disclosure are to be construed as a description of the terms of the invention, and are not intended to be limiting.

The image sensor can include an array of light sensors. Each photosensor can be a photodiode that can convert photons into electrical charges (eg, electrons or holes) to sense incident light by utilizing the photoelectric effect of some optoelectronic materials. The light sensor can also include an analog storage device, such as a capacitive device (eg, parasitic capacitance), to collect (eg, accumulate or integrate) the charge generated by the photodiode during the exposure period. The collected charge can cause a voltage change in the capacitive device. The change in voltage that reflects the amount of charge stored in the capacitive device during the exposure period may be related to the intensity of the incident light. The voltage level of the capacitive device can be buffered and fed to an analog digital converter (ADC) or other digitizer that can convert the voltage level to a digital value representative of the intensity of the incident light. The image frame can be generated according to the intensity data provided by an array of light sensors, wherein each of the light sensors forms a pixel of the image sensor, which corresponds to a pixel of the image frame. The pixel array of the image sensor can be arranged in a plurality of columns and rows, wherein each pixel produces a voltage representative of the intensity of the pixel at a particular location in the image. The plurality of pixels included in the array can determine the resolution of the generated image frame.

Embodiments of the image sensor disclosed herein may be implemented in or with an artificial reality system. Artificial reality is a form of reality that has been adjusted in some way before being presented to the user, and may include, for example, virtual reality (VR), augmented reality (AR), mixed reality. Mixed reality (MR) or some combination and/or derivative thereof. Artificial reality content can contain only computer-generated content or computer-generated content combined with captured content (such as images of real-world objects). The artificial reality content may include video, audio, tactile feedback, or some combination thereof, and any of them may be presented in a single channel or in multiple channels (eg, a stereoscopic video that produces a three-dimensional effect to the viewer). Moreover, in some embodiments, the artificial reality may be linked to an application, product, accessory, service, or some combination thereof, such as for creating content in a human reality and/or otherwise in an artificial reality. Use (for example, perform activities in it). Artificial reality systems that provide artificial reality content can be implemented on a variety of platforms, including head-mounted displays (HMDs), stand-alone HMDs, shifting devices or computer systems connected to the host computer system, or Any other hardware platform that can provide artificial reality content to one or more viewers.

1A is a perspective view of a near-eye display 100 including a simplified example of multiple sensors, in accordance with certain embodiments. 1B is a cross-sectional view of a near-eye display including a simplified example of a plurality of sensors 100, in accordance with some embodiments. Examples of media presented by near-eye display 100 include one or more images, video, audio, or some combination thereof. In some embodiments, the audio is presented via an external device (eg, a speaker and/or earphone) from the near-eye display 100, console, or both, and the audio material is rendered in accordance with audio information. The near-eye display 100 can be used to operate as a virtual reality (VR) display. In some embodiments, the near-eye display 100 can be used to operate as an augmented reality (AR) display and/or a hybrid reality (MR) display.

The near-eye display 100 can include a frame 105 and a display 110. One or more optical elements can be coupled or embedded in the frame 105. The display 110 can be used for a user to view the content presented by the near-eye display 100. Display 110 can include an electronic display and/or an optical display. For example, in some implementations, display 110 can include a waveguide display assembly to direct light from one or more generated or real images to the user's eyes.

The near-eye display 100 can include one or more image sensors 120a, 120b, 120c, and 120d. Each image sensor 120a, 120b, 120c, and 120d can include a pixel array for generating image data representing different fields of view in different directions. For example, image sensors 120a and 120b can be used to provide image data representing two fields of view along direction A along the Z-axis, and image sensor 120c can be used to provide directions along the X-axis. The image data of the field of view on B, and the image sensor 120d can be used to represent image data of the field of view in the direction C along the X axis.

In some embodiments, image sensors 120a-120d may be configured as input devices to control or influence the display content of near-eye display 100 to provide an interactive VR/AR/MR experience to a user of near-eye display 100. For example, the image sensors 120a-120d can generate physical image data of the physical environment in which the user is located. The physical image data can be provided to a location tracking system to track the location and/or movement path of the user in the physical environment. The system can then update the image data provided to display 110 based on the location or orientation of the user, to provide an interactive experience. In some embodiments, the location tracking system can perform a simultaneous localization and mapping (SLAM) algorithm to track a physical environment and a user's field of view as the user moves within the physical environment. Group of objects. The location tracking system can construct and update a map of the physical environment based on the set of objects described above, and track the location of the users within the map. By providing image data corresponding to a plurality of fields of view, image sensors 120a-120d can provide a more comprehensive physical environment to the location tracking system that can include more objects in the construction and update of the map. With this arrangement, the accuracy and effectiveness of tracking the location of the user in the physical environment can be improved.

The near-eye display 100 can further include one or more illuminators 130 to project light to a physical environment. The projected light can be related to different spectra (eg, visible light, infrared light, ultraviolet light, etc.) and can serve a variety of purposes. For example, the illuminator 130 can project light in a dark environment (or in a low-intensity infrared, ultraviolet, etc. environment) to assist the image sensors 120a-120d in capturing images of different objects in a dark environment, such as Ability to perform location tracking by the user. Illuminator 130 may project certain indicia (eg, structured light patterns) onto objects within the environment to assist the location tracking system in performing object recognition for map construction or updating.

In some embodiments, illuminator 130 can also perform stereoscopic imaging. For example, one or more of image sensors 120a and 120b can include a first array of pixels for visible light sensing and a second array of pixels for infrared (IR) light sensing. The first pixel array can be covered by a color filter (eg, a Bayer filter), wherein each pixel of the first pixel array is used to measure the light intensity associated with a particular color (eg, red, green, or blue). The second pixel array (for IR light sensing) may also be covered by a filter for IR light to pass through, wherein each pixel of the second pixel array is used to measure the intensity of the IR light. The pixel array can generate RGB images and IR images of the object, where each pixel of the IR image is mapped to each pixel of the RGB image. The illuminator 130 can project a set of IR markers onto the object, the image of which can be captured by the IR pixel array. Depending on the distribution of the IR markers of the object shown in the image, the system can estimate the distance between different portions of the object and the IR pixel array and generate a three-dimensional (3D) image of the object based on the distance. Based on the 3D image of the object, the system can determine, for example, the relative orientation of the object to the user, and can update the image data provided to the near-eye display 100 based on the relative orientation to provide an interactive experience.

As noted above, the near-eye display 100 can operate in an environment that is associated with a relatively wide range of light intensities. For example, near-eye display 100 can operate in an indoor or outdoor environment, and/or at different times of the day. The near-eye display 100 can also operate with or without the illuminator 130 turned on. As such, the image sensors 120a-120d may need to have a wide dynamic range, high sensitivity, and low noise levels to be able to operate normally at various light intensities in a wide range associated with different near-eye display 100 operating environments ( For example, an output related to the intensity of incident light is generated).

In addition, image sensors 120a-120d may need to be able to produce an output at high speed to track the movement of the eyeball. For example, the user's eyeball can move very fast (such as a saccade movement), and can quickly jump from one eyeball orientation to another. In order to track the rapid movement of the user's eyeballs, the image sensors 120a-120d may need to produce an image of the eyeball at high speed. For example, the rate at which the image sensor generates the image frame (frame rate) needs to match at least the speed of movement of the eyeball. The frame rate requires that the total exposure time of all image sensor pixels participating in the generation of the image frame is short, and high speed is also required to convert the sensor output value to a digital value for image generation. In addition, image sensors may also need to be able to operate with low power consumption.

2A is a front elevational view of a near-eye display 200 including a simplified example of multiple sensors, in accordance with certain embodiments. 2B is a cross-sectional view of a near-eye display 200 including a simplified example of multiple sensors, in accordance with certain embodiments. The near-eye display 200 can approximate the near-eye display 100 and can include a frame 205 and a display 210. One or more image sensors 250a and 250b can be coupled to or embedded in the frame 205. 2A depicts one side of a near-eye display 200 that faces the eyeball 235 of a user of the near-eye display 200. As shown in Figures 2A and 2B, the near-eye display 200 can include a plurality of illuminators 240a, 240b, 240c, 240d, 240e, and 240f. The near-eye display 200 can further include a plurality of image sensors 250a and 250b. Illuminators 240a, 240b, and 240c can emit light in a particular frequency range (eg, NIR) in direction D (opposite direction A of FIGS. 1A and 1B). The emitted light can be associated with a particular pattern and can be reflected by the user's left eyeball. Image sensor 250a may include an array of pixels to receive the reflected light and produce an image of the reflected pattern. Similarly, illuminators 240d, 240e, and 240f can emit NIR light with a particular pattern. The NIR light can be reflected by the user's right eye and received by image sensor 250b. Image sensor 250b may also include an array of pixels to produce an image of the reflected pattern. Based on the images of the reflected patterns from image sensors 250a and 250b, the system can determine the visual dwell point of the user and update the image data provided to the near-eye display 200 based on the determined visual dwell point to provide an interactive experience. To the user.

In order to avoid harm to the user's eyeballs, the illuminators 240a, 240b, 240c, 240d, 240e, and 240f are typically used to illuminate at very low intensity. In the case where the image sensors 250a and 250b include the same sensing devices as the image sensors 120a-120d, the image sensors 250a and 250b may need to be capable of generating incident light intensity when the intensity of the incident light is very weak. A related output, which may further increase the dynamic range requirements of the image sensor.

3 is a simplified block diagram of an image sensor 300 with an example of analog pixels. In some embodiments, image sensor 300 can be an active pixel sensor (APS). The image sensor 300 can include a pixel array 310, an ADC interface 320, a digital analog conversion (DAC) and support circuit 330, and a control circuit 340. Pixel array 310 can include multiple AOS pixels. Each pixel in pixel array 310 can include a light sensor, such as a photodetector or photodiode, that can generate a voltage or current signal that corresponds to the intensity of the optical signal that illuminates the pixel. For example, each pixel can convert the optical signal on the pixel into a current. Each pixel in pixel array 310 can also include an analog storage device, such as a capacitive device that can integrate current to generate and store a voltage signal, which can be referred to as an analog detection signal that represents grayscale/color information for the pixel.

Control circuitry 340 can include column decoder and driver circuitry and/or row decoder and driver circuitry at the edge of pixel array 310 for selectively enabling one or more pixels (eg, a column of pixels) to transmit the analogy The detection signal is sent to the ADC interface 320.

The ADC interface 320 can include multiple ADC devices. In some embodiments, the ADC devices can individually correspond to a row of pixels and can be used to convert the analog detection signals from the pixels into digital image data one column at a time. Each ADC device can contain two inputs, one for the reference signal and the other for the analog detection signal. The reference signal can be generated by, for example, a digital analog conversion (DAC) and support circuit 330. The ADC device can convert the analog detection signal from each pixel into digital data according to the reference signal. The digital data from each column of pixels can be stored as a digital image data file to form an image frame.

In some embodiments, each ADC may include an internal offset correction circuit and a correlated double sampling (CDS) circuit for reducing noise, such as shaped noise caused by pixel-to-pixel parameter differences. (fixed pattern noise, FPN). The CDS circuit can be a separate unit other than the ADC interface 320. For example, the CDS operation can be performed by sampling and maintaining a reference or reset signal; sampling and maintaining the analog detection signal; and subtracting the reference signal from the analog detection signal to generate a correlation analog detection signal. The ADC can then convert the associated analog detection signal into digital image data.

In some embodiments, each pixel in pixel array 310 can comprise, for example, a four-transistor (4T) APS pixel or a three-transistor (3T) APS pixel. For example, each 3T pixel in the pixel array can include a photodetector (eg, a fixed photodiode), a reset gate, a select gate, a source-coupled amplifying transistor, and a capacitive device (eg, located at the source) The parasitic capacitance of the gate of the pole-coupled amplifying transistor). The reset gate can be turned on to remove the charge stored on the capacitor device. During exposure, the charge generated by the photodetector can be stored on the capacitive device to produce an analog detection signal (eg, a voltage signal). When a corresponding selection gate is activated by using, for example, a column select signal to select a pixel, the analog detection signal at the capacitive device is amplified by the source-supplied amplifier transistor and transmitted to the read bus (readout bus). For example, a column line is converted into digital image data by the ADC to the corresponding line. In some embodiments, multiple pixels may share some gates to reduce the total number of gates used by the image sensor.

4 illustrates an exemplary four-electrode (4T) active pixel 400 in a CMOS active pixel sensor (APS). The 4T active pixel 400 can include a photodetector (eg, a fixed photodiode ( PD) 410), transfer gate 420, capacitor storage device (eg, floating diffusion (FD) capacitor 430), reset gate 440, source follower amplifying transistor 450, and select gate 460. The fixed photodiode 410 can convert the optical signal into an electrical signal and store the electrical signal as a charge to the capacitive device, such as the parasitic capacitance 412 of the fixed photodiode 410. The stored charge can be transferred to the FD capacitor 430 through the transfer gate 420. The reset gate 440 can be used to reset the FD capacitor 430 to a known voltage level. The gate of the select gate 460 can be controlled by a select signal, such as a column select signal, to selectively couple the FD capacitor 430 through the source follower amplifying transistor 450 that can amplify the voltage signal at the FD capacitor 430 to read Take the bus (for example, line 480).

During operation of the active pixel 400, for example, a shutter signal can be used to clear or discharge the charge stored in the parasitic capacitance 412 before each pixel is exposed, and the reset gate 440 can be turned off to clear the FD capacitor 430. Charge. Alternatively, the voltage level (ie, reset level) on the FD capacitor 430 after reset can be read. During the exposure, the charge generated by the photodetector can be stored on the parasitic capacitance 412 of the photodiode 410. At the end of the exposure, the charge can be transferred to the FD capacitor 430 through the transfer gate 420. The fixed photodiode 410 can have a low dark current (dark current) and a good blue response, and can be fully transferred from the fixed photodiode 410 to the FD capacitor 430 when coupled to the transfer gate. The charge can cause a voltage change of the FD capacitor 430. When the pixel is selected by enabling the corresponding selection gate 460, the voltage signal (ie, the analog detection signal) located in the FD capacitor 430 can be amplified by the source-supplied amplifier transistor 450 and transmitted to the line circuit. 480. The ADC connected to line 480 can then convert the amplified voltage signal into digital image data. In some embodiments, the use of charge in the pixel from the photodetector to the floating diffusion capacitor can reduce noise by enabling correlated double sampling (CDS).

In many image sensors, since the number of ADCs in the image sensor is limited due to limitations such as wafer size and/or power, pixels in the image sensor need to alternately access the ADC to generate digital image data. , for example, one column of pixels at a time. In general, a set of ADCs (eg, one pixel per line) can be used to simultaneously convert the voltage signals generated by the pixels in a column into digital image data. However, pixel cells in adjacent columns may need to access the set of ADCs in turn. In one example, a rolling electronic shutter can be used on a CMOS image sensor, wherein the pixel columns are sequentially exposed to the incident light to generate a charge, and one column of pixels in the image sensor can be selected and read at a time, such that The pixels of the image sensor can be selected and read column by column to produce an image frame. In one embodiment, each column of pixels of the image sensor can be individually exposed to incident light for an exposure period. During the exposure period, the pixels in the column can each generate a voltage signal according to the charge generated by the photodiode, and transmit the voltage signal to the ADC of the corresponding row. All rows of ADCs can produce a set of digital image data representing the intensity of the incident light received by that row of pixels. During the next exposure period, the next column of pixels can be exposed to incident light to produce another set of digital image data until all columns of pixels have been exposed to incident light and a digital image of an image frame has been output. In another example, the exposure times of adjacent pixel columns may have some overlap, but the pixels of each column may still need to convert the voltage signals generated by the photo charges into digital image data in turn. The image frame can be generated according to the digital image data of each column of pixels in the image sensor.

FIG. 5A illustrates an example state of pixels of different columns in image sensor 510 using a rolling shutter at a first time instant. At a first instant in time, pixels on column 520 of image sensor 510 can be reset, and pixels on single or multiple columns 530 can be exposed to optical signals to accumulate charge on each pixel, and from column 540. The voltage signal of the upper pixel can be read by a group of ADCs and converted into a digital signal. The resetting of the pixels in image sensor 510 can be disabled at the instant of the first time without consuming any electrical energy. A window containing a reset pixel column (eg, column 520), a single column or columns of pixels exposed to the VLC optical signal (eg, column 530), and a read pixel column (eg, column 540) may be moved down one column at a time To generate an image frame.

FIG. 5B illustrates an example state of pixels of different columns in image sensor 510 using a rolling shutter at a second time instant. The second moment is later than the first moment. In FIG. 5B, the reset pixel column (eg, column 520), the single or multiple columns of pixels exposed to the optical signal (eg, column 530), and the compared to the location of the first time instant shown in FIG. 5A The read pixel column (eg, column 540) can be moved down.

As noted above, image sensors with high speed (e.g., high frame rate), high sensitivity, high dynamic range, high resolution, and low power consumption are desirable for applications such as virtual reality or augmented reality devices. However, the speed and sensitivity of the image sensor using the rolling shutter described above may be limited due to the limited number of ADCs shared by pixels on different columns and the limited exposure period for each column of pixels.

In some embodiments, a digital pixel image sensor having a digitizer for each pixel can be used to achieve a high frame rate. Each digital pixel in a digital pixel image sensor can include a photodetector (eg, a photodiode), a transfer gate, an analog storage device (eg, an explicit or parasitic capacitor), a digitizer (eg, an ADC), and a digital memory. body. The photodiode can convert the optical signal into a telecommunication signal (such as a charge or current) and/or integrate the electrical signal. The transfer gate can be used to transfer the (integrated) electrical signal from the optical diode to the analog storage device, and the digitizer can convert the electrical signal in the analog storage device into a digital bit. The digital memory can store the digits before the digits are read from each pixel. Since each pixel in the digital pixel image sensor has its own ADC, all pixels of the digital pixel image sensor can be exposed to the optical signal during the same exposure time of the image frame, and from the digital pixel image sensor. The voltage signals of all pixels can be converted into digital image data at the same time. Therefore, the global shutter can be used to control the exposure of all pixels in the image sensor, and the frame rate of the image sensor can be significantly improved compared to the aforementioned rolling shutter image sensor.

6 is a simplified block diagram of an example global shutter digital pixel image sensor 600 in accordance with some embodiments. The digital pixel image sensor 600 can include a digital pixel array 610 and other support circuits, such as a column drive and global signal drive circuit 620, a global counter 630, one or more count buffers 640, and a ramp generation and buffer circuit 650. The digital pixel image sensor 600 can include a sense amplifier 660, a sense amplification bias circuit 670, and a line memory 680 for reading digital data from each column of digital pixels to form a digital image. The digital pixel image sensor 600 can also include other circuitry, such as a digital block 690, a power conditioning circuit 695, and/or a mobile industry processor interface (MIPI) circuit 698.

The digital pixel array 610 can include a two-dimensional array of pixels. Each pixel can include a photodetector (eg, a photodiode) and a digitizer. The analog voltage signal generated by the photodiode collecting charges in response to the incident light signal can be converted by the digitizer inside each pixel. Thereby, each pixel can output digital data instead of analog voltage signals, which correspond to the intensity and/or color of the incident light. In addition, the analog voltage signals on all of the pixels of the digital pixel array 610 can be simultaneously converted, allowing global shutter operation without the use of additional masking analog storage nodes in the pixels to store analog voltage signals. The column drive and global signal drive circuit 620 can control the operation of the pixel, including charge integration, comparator operation, digital write, digital output, and the like.

The global counter 630 can be used to provide a global count value to all pixels of the digital pixel array 610. One or more count buffers 640 can transmit the global count value from the global counter 630 to each pixel. The ramp generation and buffer circuit 650 can generate a global reference signal to all pixels, such as a ramp signal (tilt up or down) or a triangle signal. The digitizer in each pixel can use the global count value and the global reference signal to determine the digital data corresponding to the analog voltage signal generated by the pixel.

In some embodiments, the voltage representing the digital bit within the pixel can be oscillated without rail-to-rail, so the sense amplifier can be used to regenerate the digital value (in-pixel digital bit). Sense amplifier 660 can read the digital value (the voltage representing the digital bit within the pixel) in the pixel resulting from the analog to digital conversion. The sense amplifier 660 can read the digital value (voltage) in a column of pixels at a time. Each sense amplifier 660 can be coupled to a pixel digital output line to read digital data in each pixel of a column. The sense amplification bias circuit 670 can be used to provide a bias voltage and current to the sense amplifier 660. Line memory 680 can temporarily hold digital data read from a column of pixels.

The digital block 690 can include logic to control the operation of the image sensor, including the timing of the image sensor. The power conditioning circuit 695 can generate analog power and voltage sources of different levels (eg, 3.3 volts, 1.8 volts, and 1.2 volts) to the image sensor and manage the power of the image sensor, including power to turn each module on or off. The MIPI circuit 698 can be used to transfer digital data in the MIPI output format to the memory.

By providing a digitizer (eg, an ADC) in each pixel, a plurality of pixels of the pixel array can be exposed to incident light and simultaneously produce a digital representation of the intensity of the incident light received by the plurality of pixels, respectively, to Provides full shutter operation. For high-speed motion capture, a global shutter is advantageous because it avoids the problem of motion distortion, which is caused by images of multiple columns of pixels capturing different images of moving objects at different times. Rolling shutter operation. Further, the method in which the pixels of the plurality of columns are alternately exposed to generate image data indicating the light intensity can reduce the overall time for using the pixels to generate the image frame. Thus, the techniques of the present disclosure can increase the operating speed of an image sensor. Furthermore, since all pixels are simultaneously exposed, the average exposure time of each pixel of the present disclosure can be increased compared to using a rolling shutter. Thereby, the sensitivity of the image sensor can also be improved.

7 is a simplified block diagram of an example digital pixel 700 of an exemplary global shutter digital pixel image sensor in accordance with some embodiments. The digital pixel 700 can be a partial digital pixel array in a coefficient pixel image sensor, like the digital pixel array 610 in the coefficient bit pixel image sensor 600. The digital pixel 700 can generate digital image data corresponding to the intensity of a pixel in the image frame. As shown in FIG. 7, the digital pixel 700 can include an optical diode 702, an integrating capacitor 703, a transfer gate 704, a reset switch 718, a measurement capacitor 706, an optical buffer 710, and a pixel quantizer 750. In some embodiments, the digital pixel 700 can include a shutter switch 726 that is controlled by a global shutter signal.

In some embodiments, the photodiode 702 can comprise a PN diode or a PIN diode. Each of the shutter switch 726, the transfer gate 704, and the reset switch 718 may include a transistor. For example, the transistor may include a metal-oxide-semiconductor field-effect transistor (MOSFET), a bipolar junction transistor (BJT), or the like. The shutter switch 726 can function as an electronic shutter to control the exposure period of the digital pixel 700. The shutter switch 726 can be enabled (conducted) to reset the integrating capacitor 703 prior to the exposure period. During the exposure period, the shutter enable signal 724 can be disabled (turned off) by the exposure enable signal 724, which can cause the charge generated by the photodiode 702 to move to the integrating capacitor 703 and/or the measurement capacitor 706. The reset switch 718 can be disabled (turned off) by resetting the signal 720, which can cause the measurement capacitor 706 to store the charge generated by the photodiode 702 and form a voltage signal that is related to the amount of stored charge. The voltage signal of the measurement capacitor 706 can then be converted to digital data. Upon completion of the conversion of the voltage signal to the measurement capacitor, the reset switch 718 can be enabled to clear the charge stored in the measurement capacitor 706 to the charge slot 722 so that the measurement capacitor 706 can be used for the next measurement.

The integrating capacitor 703 can be a parasitic capacitance of the photodiode 702 and other circuits connected to the photodiode 702, and can store the charge generated by the photodiode 702. For example, the integrating capacitor 703 can include a bonding capacitance at the P-N diode bonding interface or other parasitic capacitance connected to the photodiode 702. Since the integrating capacitor 703 is adjacent to the photodiode 702, the charge generated by the photodiode 702 can be accumulated in the integrating capacitor 703. The measurement capacitance 706 can be at a parasitic capacitance of the floating diffusion node (eg, the floating end of the transfer gate 704), a metal capacitance, a MOS capacitance, or any combination thereof. Measurement capacitance 706 can be used to store the amount of charge, which can be measured by pixel quantizer 750 to provide a digital output indicative of the intensity of the incident light. The charge stored in the measurement capacitor 706 may be the charge transferred from the integration capacitor 703 through the transfer gate 704. The transfer gate 704 can be controlled by the measurement control signal 708 to control the transfer of charge from the integration capacitor 703 to the measurement capacitance 706. The total amount of charge accumulated in the integrating capacitor 703 and/or the measuring capacitor 706 may reflect the total charge generated by the photodiode 702 during the exposure period, which reflects the intensity of light incident on the photodiode 702 during the exposure period.

The charge stored in the measurement capacitor 706 can be sensed by a selection sense amplifier or optical buffer 710 to produce a copy of the analog voltage signal at analog output node 712 (but with greater drive strength). The analog voltage signal generated at analog output node 712 can be converted by pixel quantizer 750 into a set of digital data (eg, including logic 1 and 0). After the exposure period, the analog voltage signal formed on the measurement capacitor 706 can be sampled and a digital output can be generated.

The pixel digitizer 750 can include a comparator 754 and a digital output generator 760, which can include pixel memory 764. The pixel quantizer 750 can use the count value generated by the clock counter 762, which can be a global clock counter for all pixels in the coefficient pixel image sensor. The clock counter 762 can generate a set of count values based on the clock signal 780. In some embodiments, the clock counter 762 can also be used to generate a global reference signal by the reference signal generator 770. The reference signal generator 770 can include a digital analog converter (digital-to-analog) capable of generating an arbitrary reference signal. Converter, DAC), or a ramp or triangle waveform generator that uses the count value. For example, after the start of digitization, the DAC 772 can be programmed to generate a ramp reference signal 752 corresponding to the count output 766 from the clock counter 762, which can be tilted up or down depending on the implementation state. Comparator 754 can compare the analog voltage signal from buffer 710 with reference signal 752 from reference signal generator 770. When the analog voltage signal from buffer 710 and reference signal 752 cross each other, the output of comparator 754 can change state. Digital output generator 760 can use the output of comparator 754 to latch the current value of count output 766 from clock counter 762 to pixel memory 764. The current count output 766 can correspond to the total number of quantization steps used to digitize the analog voltage signal, the quantization error being less than the voltage level representing the quantization step (also known as the least significant bit (least significant bit). LSB)). Thus, the count output 766 is represented by a digital representation of the amount of charge stored in the measurement capacitor 706 and a digital representation of the incident light intensity. The digital data in the pixel memory 764 can be read through a set of pixel output bus bars 790 to, for example, the line memory 680 or external memory to store the digital video frame.

The digital pixel 700 can also include or be connected to other control circuitry (not shown in FIG. 7) to control the timing and intensity of the exposure enable signal 724, the measurement control signal 708, and the reset signal 720 to control the integration capacitor 703 and The charge accumulation operation in the capacitance 706 is measured for the measurement of the light intensity. It should be understood that these control circuitry may be externally coupled to digital pixel 700 and may be, for example, part of column drive and global signal driver circuit 620 and/or digital block 690 of FIG.

8 illustrates example operations of an example digital pixel (eg, digital pixel 700) of an exemplary global shutter digital pixel image sensor, in accordance with certain embodiments. In the example shown in FIG. 8, the quantization process can be performed using a uniform quantization step, wherein the reference signal (eg, reference signal 752 of FIG. 7) will be applied to each clock cycle of the clock signal (eg, clock signal 780). Raise (or drop) the same amount. The amount of reference signal rise (or fall) may correspond to the quantization step (ie, LSB). As described above, the reference signal 752 is coupled to one input of the comparator, and the analog voltage signal to be quantized is coupled to the other input of the comparator. As the number of clock cycles increases, the reference signal can be increased and reached a quantization step of the analog voltage signal, at which point the output of the comparator can change state, for example, from a low level to a high level. The flipping of the comparator output can latch the current count value to the pixel memory as a digital data representing the analog voltage signal at the pixel.

Figure 8 shows an analog voltage signal at two pixels, an analog voltage signal 810 at pixel 1 and an analog voltage signal 820 at pixel 2. FIG. 8 also shows the clock count value 870 of the clock signal 860 and the pulse counter, which calculates the number of cycles of the clock signal 860. At time t0 (when the digitization starts), the clock counter can begin to calculate the number of cycles of the clock signal 860. For example, the clock count value 870 can be incremented by one after each cycle of the clock signal 860. As described above, the clock counter can be a global clock counter shared by a plurality of pixels, whereby the size of each pixel can be reduced. When the pulse count value increases, the voltage level of the reference signal 830 can be increased. For example, reference signal 830 can be generated by the DAC based on clock count value 870. Since the reference signal 830 is lower than the analog voltage signal 810 at the pixel 1 and the analog voltage signal 820 at the pixel 2, the output 840 of the comparator in the pixel 1 and the output 850 of the comparator in the pixel 2 can be at a low (or high) ) Level.

At time t1, the reference signal 830 can reach the analog voltage signal 810 at pixel 1 (eg, within one of its LSBs), and the output 840 of the comparator in pixel 1 can be flipped from a low level to a high level. The inversion of the output 840 of the comparator in pixel 1 may form a clock count value 870 having a digital value D1 at time point t1 for storage to the pixel memory of pixel 1. The clock counter can continue to calculate the number of cycles of the clock signal 860, and the voltage level of the reference signal 830 can continue to increase. At time t2, the reference signal 830 can reach the analog voltage signal 820 at pixel 2 (eg, within one of its LSBs), whereby the output 850 of the comparator in pixel 2 can be flipped from a low level to a high level. The flipping of the output 850 of the comparator in pixel 2 may form a clock count value 870 having a digital value D2 at time point t2 for storage to the pixel memory of pixel 2.

In this way, analog voltage signals at different pixel pixels can be simultaneously converted to digital values by using a global clock count value and a global reference signal 830 (eg, a ramp signal) by a comparator in each pixel, which is represented by different pixels. Light intensity, not by complex ADCs in each pixel or in each row of pixels. Therefore, the size of the digital pixels and the power consumption of the digital pixels can be significantly reduced, which enables the image sensor to have higher resolution, higher density, smaller size, and lower power consumption.

Various noises or errors may affect the measurable lower limit of the light intensity of the image sensor (often referred to as the minimum resolvable signal level). For example, the charge collected at the floating node may contain noise charges that are independent of light intensity. One source of the noise charge is a dark current, which may be caused by, for example, crystallographic defects in the PN junction of a photodiode connected to a capacitor and the PN interface of other semiconductor devices (eg, transistors). Current. The dark current may flow into the capacitor causing a voltage change that is independent of the intensity of the incident light. The dark current generated in the photodiode is typically smaller than the dark current generated in other semiconductor devices. Some of the noise charge may be caused by capacitive coupling with other circuitry. For example, when the digitizer performs a read operation to determine the amount of charge stored in the floating node, the digitizer may introduce a noise charge into the floating node through capacitive coupling.

In addition to the noise charge, the digitizer may also cause measurement errors when determining the amount of charge. The measurement error may reduce the correlation between the number of digits and the intensity of the incident light. One source of measurement error is the quantization error. In the quantization process, a set of discrete levels can be used to represent a set of consecutive voltage signals, each of which represents a predetermined range of voltage signal levels. Therefore, when there is a difference between the voltage level indicated by the quantization level and the input analog voltage approximate to the quantization level, quantization error may occur. The quantization error of the digitizer shown in FIG. 7 can be achieved by using a smaller quantization step size (eg, an increase or decrease of the reference signal 830 in each order or each clock cycle) and/or a fast clock signal. Come down. Other sources of measurement error include, for example, random noise (such as thermal kT/C noise on a capacitor), device noise (such as device noise for ADC circuitry), and comparator offset, which is stored in a capacitor. Uncertainty is added to the measurement of the amount of charge.

The noise charge and the digitizer measurement error can determine the measurable lower limit (sensitivity) of the image sensor. The upper limit of the measurable light intensity of the image sensor can be determined by the intensity of the light that causes the charge (ie, photocurrent) generated by the photodiode to be saturated per unit time. The ratio between the upper and lower limits can generally be referred to as the dynamic range, which can determine the range of operational light intensities of the image sensor.

In some cases, unintended offsets in the comparator and changes in other components or parameters of the digital pixels may cause fixed pattern noise (FPN). In some embodiments, the digital pixels in the digital pixel image sensor may include correlated double sampling (CDS) circuitry to reduce offset errors, thereby reducing FPN. The CDS circuit can measure the reset level of the analog storage device (eg, measurement capacitance 806) and the signal level of the analog voltage signal of the analog storage device after exposure, and utilize the measured signal level and reset. The difference between the levels determines the actual signal value of the pixel. Since the measured signal level includes a reset level component (which is caused by different differences or changes in other parameters or devices among the pixels of the image sensor), the measured signal level and reset The difference between the levels can more accurately represent the actual voltage change caused by the charge generated by the light illumination of the pixel.

9 is a simplified block diagram of an example digital pixel 900 including an analog correlation double sampling (CDS) circuit, in accordance with some embodiments. The digital pixel 900 can include a fixed photodiode 910, a transfer gate 920 controlled by the control signal TX, a floating diffusion node 930, a reset gate 940 controlled by the control signal RST, and a selection gate 950 controlled by the control signal SEL. The in-pixel source-carrying buffer stage including transistors 942 and 952, and the gate 950 are selected to ensure high pixel conversion gain. The digital pixel 900 can include a comparator 970 and a pixel memory 980. The operation of these circuits can be the same as the operation of active pixel 400 or digital pixel 700 described above.

As shown in FIG. 9, the digital pixel 900 can further include an analog CDS circuit 960. In some embodiments, the analog CDS circuit 960 can include two CDS capacitors, two gates, and a differential amplifier. The analog voltage level (ie, the reset level) after the reset of the measurement capacitor (ie, FD node 930) can be stored in a CDS capacitor through a gate, and the analog voltage level after the charge transfer is measured. The quasi (ie signal level) can be stored in another CDS capacitor through another gate. The difference between the two analog voltage levels can be generated using a differential amplifier and can then be digitized by comparator 970. Some other implementations of analog CDS can also be utilized. For example, in some embodiments, some circuit simplifications of the analog CDS circuit 960 can be applied to reduce the total number of transistors. However, in order to make the thermal noise (kT/C noise) of the capacitor meet the requirements of the image sensor, the analog CDS may use a relatively large area to build a CDS capacitor for one or more pixels. Moreover, using the source follower buffer stage to drive the one or more analog CDS capacitors and the area of the CDS capacitor may limit the ability to scale down the pixels and may also increase the power consumption of the pixels.

10 is a simplified block diagram of an exemplary digital pixel 1000 including a digital CDS circuit, in accordance with some embodiments. The digital pixel 1000 can include a 3T or 4T light sensor, a comparator 1050, and a pixel memory block as described above. The photo sensor may include a fixed photodiode 1010, a transfer gate 1020, a reset gate 1040, and a floating diffusion capacitor 1030.

During or prior to exposure of the digital pixel 1000, the floating diffusion capacitor 1030 can be reset to a reset level (eg, 0 volts or other DC level) by turning on the reset gate 1040 with the reset signal RST. The voltage level at the floating diffusion capacitor 1030 can be digitized and stored to the m-bit memory block 1060 of the pixel memory. Since the reset level is typically low, the time of digitization performed using the digitizer described above can be short, and the reset level can be digitized into a fractional value that can be represented by a small number of bits.

After the digital pixel 1000 is exposed, the charge generated by the fixed photodiode 1010 and accumulated in the photodiode 1010 can be transferred to the floating diffusion capacitor 1030 by turning on the transfer gate 1020 by using the transfer control signal TX. The voltage level at floating diffusion capacitor 1030 can be digitized and stored into n-bit memory block 1070 of pixel memory, where n can be greater than m. Since the voltage level after the charge transfer of the floating diffusion capacitor 1030 can be higher than the reset level, the above-mentioned digitization performed by the digitizer may be longer, and the voltage level may be digitized into a larger number of bits. The larger value represented by the yuan.

The m-bit data indicating the reset level and the n-bit data indicating the signal level can be read, and the difference between the n-bit data and the m-bit data can represent the detected photocurrent or The voltage signal generated by the charge, thereby indicating the intensity of the light detected at the pixel. In this way, errors or noises caused by mismatches or changes in the device or parameters of the pixels in the digital image data (eg, deviation error, comparator threshold mismatch, capacitance mismatch, etc.) can be reduced. In some embodiments, the subtraction of n-bit data and m-bit data can be performed at the pixel level before the output of the pixel is read, so the total number of digits read by the pixel can be small.

Since no additional CDS capacitors or differential amplifiers are used in digital CDS, digital pixels containing digital CDS circuits can use significantly less germanium area and power than digital pixels using analog CDS circuits.

11 illustrates an example digital pixel 1100 that includes a digital CDS circuit in accordance with some embodiments. Digital pixel 1100 can be an exemplary implementation of coefficient bit pixel 1000. The digital pixel 1100 can include a photo sensor 1105 that includes a fixed photodiode 1110, a transfer gate 1120, a reset gate 1140, and a floating diffusion capacitor 1130. The digital pixel 1100 can also include a comparator 1150, such as a comparator 754, to perform analog voltage level digitization.

The digital pixel 1100 can further include a write logic gate, which can include a gate (or inverse gate) 1160. The input of the gate (or reverse gate) 1160 can be connected to the output of the comparator 1150 and the reset enable signal ENABLE_RST (for example, a reset signal for turning on the reset gate 1140 or a signal synchronized with the reset signal) . The output of the gate (or reverse gate) 1160 can be coupled to the write enable input WR of the m-bit cell 1180 (eg, a D-type flip-flop). When the reset level is digitized, the reset enable signal ENABLE_RST can be asserted (eg, set to a high level). When the output of the comparator 1150 changes state, such as when the reference signal (for example, the ramp signal) reaches the reset level and changes from "0" to "1", the gate (or reverse gate) 1160 will be from "0". Flips to "1" which can cause, for example, the current count value from the global clock counter to be locked to m-bit 1180, which can be read later.

The write logic gate of the digital pixel 1100 can further include a gate (or reverse gate) 1170. The input of the gate (or reverse gate) 1170 can be coupled to the output of the comparator 1150 and to the reset enable signal ENABLE_RST (eg, a select signal for selecting a pixel). The output of the gate (or reverse gate) 1170 can be coupled to the write enable input WR of the n-bit cell 1190 (eg, a D-type flip-flop). When the pixel signal level is digitized, the read enable signal ENABLE_SIG can be asserted (eg, set to a high level). When the reference signal (such as the ramp signal RAMP) reaches the pixel signal level, the output of the comparator 1150 may be reversed, such as changing from "0" to "1", so the output of the gate (or reverse gate) 1170 may be "0" flips to "1", which causes the current count value from the global clock counter to be locked to n-bit 1190, which can be read later.

In this way, errors or noises caused by mismatches and changes in the device and parameters of the pixel in the digital image data (eg, deviation error, comparator threshold mismatch, capacitance mismatch, etc.) can be reduced. The digital pixel 1100 can use m additional bit cells, such as a D-type flip-flop, which is a digital circuit and can use a smaller area than a digital pixel that may not include a digital CDS circuit. As noted above, by using digital pixels, the image sensor can operate the global shutter to increase frame rate and sensitivity (eg, due to longer exposure times per pixel). During the time period of the image frame, the reset level of the digital pixels can be digitized at different times.

12A illustrates an example timing cycle during operation of a global shutter digital pixel sensor having a digital CDS circuit during operation of a global shutter digital pixel sensor, in accordance with some embodiments. All of the pixels in the digital pixel sensor can operate in the same manner at the same time. For a digital pixel, a time period 1200 for outputting a set of output data of a frame image may include a shutter period 1210, an integration period 1212, a reset period 1214, a reset level transition period 1216, a charge transfer period 1218, The signal level conversion period 1220 and the data output period 1222.

For a digital pixel, the time period for generating a set of output data for an image frame may start at a shutter period 1210 during which the voltage level of the photodiode may be reset and may be set for pixel integration. Time period. At the end of the shutter period 1210, each digital pixel can begin to perform signal integration (eg, charge accumulation) in the integration period 1212, during the integration period 1212, the parasitic capacitance of the photodiode of the digital pixel (eg, the integration capacitance 703, which can The parasitic capacitance including the photodiode 702 and other circuitry connected to the photodiode 702 can collect charge (eg, photoelectrons) that is generated by the photodiode in response to incident light at the digital pixel. At a short period of time before the end of the integration period 1212 (eg, less than 100 microseconds), the measurement capacitance of each pixel (eg, measurement capacitance 706 or floating diffusion capacitance 1130) may be weighted during the reset period (RST) 1214. The analog-to-digital conversion (AtoD) is then performed on the reset voltage level (V_RST) during the reset level transition period 1216. The digitized pixel reset level can be stored in the pixel memory, such as in the m-bit cell 1180. After the pixel reset level is converted, during the charge transfer period (TG) 1218, the charge accumulated in the integrated capacitance of the photodiode of each pixel can be transferred to the measurement capacitance (eg, redistributed to the photodiode) Performing an analog-to-digital conversion of the pixel signal level (V_SIG) (ie, the voltage level on the measured capacitance after charge transfer) during the signal level transition period 1220) (AtoD). The digitized pixel signal level of each pixel can be stored in the pixel memory, such as in n-bit cell 1190. After the digital pixel reset level and the signal level are stored in the pixel memory, the data output period 1222 can be started, and the pixel reset level and the signal level digit value of each pixel can be from the pixel array. Read it out in a column. These digital values can be read more quickly than the analog voltage.

Since the previously accumulated charge is transferred to the measurement capacitor, the digital pixel can start accumulating the charge of the next image frame, and the frame image period T_Frame of the digital image sensor can be equal to the shutter period 1210 (T_Shutter), the integration period. 1212 (T_Int), reset period 1214 (RST) and reset level conversion period 1216 (collectively referred to as T_Rst) and charge transfer period 1218 (TG). As mentioned above, the reset level is usually a low level, so the time for digitizing the reset level can be shorter. The charge transfer period 1218, the signal level transition period 1220, and the data output period 1222 may also be shorter than the integration period. Therefore, the frame recurring expenditure (non-integration time) in the frame time period can be small. In an example, the frame time period can be about 33 milliseconds (ie, the frame rate is 30 frames per second), and frame integration recurring expenses (which can include a reset period 1214, a reset level transition period 1216, and a charge). The transition period 1218, and in some embodiments, may further include a signal level transition period 1220 and a data output period 1222) may be approximately 100 microseconds. Thus, during most (eg, about 95%, 99%, or more) of the time period of the frame, the digital pixels can operate in a low power mode (eg, an integration mode). Since during the shutter period 1210 and the integration period 1212, the digital pixels can consume extremely low power in the low power mode, and the shutter period 1210 and the integration period 1212 occupy most of the frame time period, so the overall work of the digital pixels The consumption can be very low.

12B is a timing diagram of the operation of a digital pixel having a digital CDS circuit in a global shutter digital pixel image sensor, in accordance with some embodiments. In FIG. 12B, the first reset pulse 1252 of the reset signal 1250 and the first transfer control pulse 1262 of the transfer control signal 1260 can be simultaneously generated (for example, at the end of the shutter period like the shutter period 1210), so the integration capacitor can be And measure the capacitor reset or discharge. After the first reset pulse 1252 and the first transfer control pulse 1262, integration of the pixels can begin. At the end of the integration period, the second reset pulse 1254 can reset the measurement capacitance (eg, floating diffusion capacitance) to a reset level, such as 0 volts or other DC level. The reset level of the measurement capacitor can be digitized using the comparator, clock counter, and ramp signal 1270. As described above, during the digitization of the reset level, the ramp signal 1270 can be gradually increased or decreased as indicated by ramp 1272. The clock counter can count the number of clock cycles. The clock count value 1280 when the ramp signal 1270 reaches the reset level can be stored as a reset count value 1282 in the pixel memory (eg, m-bit reset level memory). Next, the second transfer control pulse 1264 can be asserted to transfer charge from the integrated capacitance of the photodiode to the measurement capacitance. The voltage level on the measurement capacitor (referred to as the signal level) can be digitized using the comparator, ramp signal 1270, and clock counter as described above. During the digitization of the signal level, the ramp signal 1270 can be gradually increased or decreased as indicated by ramp 1274. The clock count value 1280 when the ramp signal 1270 reaches the reset level can be stored as a signal count value 1284 in the pixel memory (eg, n-bit signal level memory). The reset count value 1282 and the signal count value 1284 can be read from the digital pixel memory and sent out as output data 1292 in the MIPI format on the MIPI interface 1290. Since the reset value of the measurement capacitor is usually small, the reset count value representing the reset level can occupy less bits than the signal count value representing the signal level.

Figures 12A and 12B show the reset level measured at the end of the integration period. In some embodiments, the reset level can be measured at other times. For example, since the measurement capacitance and the integration capacitance can be isolated by the transfer gate, the reset level can be measured and digitized during the shutter period, at the beginning of the integration period, or during the intermediate period of the integration period. The examples shown in FIGS. 12A and 12B can be compared with the operation and timing components of other kinds of digital pixels, and can reduce the noise of digital pixels (such as noise caused by dark current and 1/f noise). good performance.

For example, as shown in FIGS. 12A and 12B, since the measurement capacitance is reset before the charge transfer and signal level digitization, the dark current will be accumulated on the measurement capacitor before the signal level is digitized. Short, so the noise or error caused by dark current will be smaller. In contrast, if the measurement capacitance is reset at the beginning of the integration period or during the shutter period, the dark current will accumulate charge on the measurement capacitor for the entire or most of the integration period, so the dark current The noise or error caused will be larger.

In addition, since the measurement capacitance is reset before the charge transfer and signal level digitization, the CDS circuit can perform effective switching at high frequencies because of the short time between the reset of the measurement capacitance and the charge transfer. Therefore, the 1/f noise of the digital pixels (ie, flicker noise or pink noise) can be reduced due to the higher operating frequency. Conversely, if the measurement capacitance is reset, for example, at the beginning of the integration period, the CDS circuit can be effectively switched at low frequencies due to the long time between the reset of the measurement capacitance and the charge transfer. Therefore, in such a data conversion process, the noise or error caused by the 1/f noise is large.

As described above, since each digital pixel includes a digitizer, the overall power consumption of the digital pixel image sensor is high when all the pixels operate in parallel. For example, if each digitizer consumes 1 microwatt, an image sensor with one million digit pixels consumes at least 1 watt, which may not be suitable for mobile devices or wearable devices like HMDs. . Therefore, there is a need to reduce the power consumption of digitizers, buffers, and other circuits in digital pixels. In some of the above examples, such as the example of FIG. 7, the digitizer can use a comparator to compare the signal level with a reference level and latch the clock count value to the pixel memory. Since the latch and pixel memory can consume less power, reducing the power consumption of the comparator can more effectively reduce the power consumption of the digitizer and the digital pixel.

FIG. 13 depicts an example comparator 1300 that includes a DC bias circuit. The comparator 1300 can include a differential amplifier that can include P-channel transistors 1310 and 1320, N-channel transistors 1330, 1340, and 1350. The analog voltage signal VFD from the measurement capacitor can be connected to the gate of one of the transistor 1330 and the transistor 1340, and the ramp signal VRAMP from the reference signal generator (eg, the reference signal generator 770) can be connected to the transistor. The gate of the other of 1330 and transistor 1340. The difference between the ramp signal VRAMP and the analog voltage signal VFD from the measurement capacitor may cause the current through the transistors 1310, 1320, 1330, and 1340 to be different. Therefore, the voltage level at node 1325 will depend on the difference between the ramp signal VRAMP and the analog voltage signal VFD from the measurement capacitor. The inverter 1360 connected to the node 1325 can convert the voltage level at the node 1325 to a "high" or "low" signal.

In comparator 1300, transistor 1350 can apply a bias voltage to the transistor of the differential amplifier to a level at which the transistor is suitable for DC operation. During operation of the comparator, the drop in current through transistor 1330 may correspond to an increase in current through transistor 1340, and the DC bias current through transistor 1350 may maintain a constant value. Thus, the differential amplifier of comparator 1300 can consume DC power to apply a bias voltage to the transistor. As noted above, if the DC power used to apply the bias is 1 microwatt per pixel, an image sensor with one million digit pixels will consume at least 1 watt.

FIG. 14 illustrates an example comparator 1400 including a pre-charge circuit in accordance with some embodiments. In the comparator 1400, the analog voltage signal VFD from the measuring capacitor can be connected to the gate of the transistor 1410, while the reference signal VRAMP can be connected to the source or (drain) of the transistor 1410. In some embodiments, the transistor 1410 can be a p-channel metal-oxide-semiconductor (PMOS) transistor without biasing the transistor 1410 using a DC bias circuit. Instead, a precharge circuit including precharged transistor 1420 is used to precharge the output node COMP of the comparator before the digitization of the analog voltage signal VFD begins. During digitization, pre-charged transistor 1420 can be turned off without current flowing through pre-charged transistor 1420. Thereby, the DC bias current as well as the static power consumption can be reduced or minimized.

Comparator 1400 can also include one or more inverters. For example, as shown in FIG. 14, the first inverter may include transistors 1430 and 1440 to generate a write signal Write, and the second inverter may include transistors 1450 and 1460 to generate a write b signal Writeb. The write signal Write and the write b signal Writeb can be used to latch the clock count value into the pixel memory. In some embodiments, transistors 1410 and 1420 can have a thick gate oxide and operate at 3.3 volts, while one or both of the two inverters can include a thin gate oxide and operate at a lower voltage. The transistor (for example, 1.8 volts or 1.2 volts) can make the switching boundary of the write signal Write and/or the write B signal Writeb sharp, and the level of the write signal Write and/or the write B signal Writeb can be More consistent with the operating voltage of a digital circuit comprising a logic gate combination (eg, gates 1160 and 1170) and a memory device. One of the motivations for using thin oxide transistors is that thin oxide transistors can occupy less area in the pixel and have lower power consumption than thick oxide transistors.

15 is a timing diagram 1500 of the operation of the example comparator of FIG. 14, such as comparator 1400 shown in FIG. 14, in accordance with some embodiments. 15 shows a precharge signal 1510 for controlling the precharge transistor 1420, a ramp signal 1520 for connecting to the source of the p-channel transistor 1410, a voltage signal 1530 that can drive the gate of the transistor 1410, and a transistor 1410. The voltage signal of the drain (ie, the voltage signal of the node COMP) 1540, the write signal 1550 at the output of the first inverter of the comparator 1400, and the write of the output of the second inverter of the comparator 1400 b Signal 1560. As shown in FIG. 15, before the start of the digitizing operation, a high level pulse is applied to the gate of the precharge transistor 1420 to turn on the precharge transistor 1420, whereby the node COMP can be precharged to a low level (eg, Vss). After the precharge operation, the precharge signal 1510 can be set to a low level to turn off the precharge transistor 1420 and analog to digital conversion can begin. During the analog to digital conversion, the ramp signal 1520 can be gradually increased. When the ramp signal 1520 is less than the sum of the voltage signal 1530 and the threshold voltage Vtp of the transistor 1410, the transistor 1410 may be turned off and the voltage signal 1540 of the node COMP may be at a low level. Therefore, the write signal 1550 at the output of the first inverter may be at a high level, and the write b signal 1560 at the output of the second inverter may be at a low level. At time t1, the ramp signal 1520 becomes greater than the sum of the voltage signal 1530 and the threshold voltage Vtp of the transistor 1410. At this time, the transistor 1410 may be turned on to charge the node COMP, so the voltage signal 1540 of the node COMP may increase to a high level. The level can cause the first inverter to flip. As a result, after a gate delay, the write signal 1550 at the output of the first inverter may become a low level, and the write b signal 1560 at the output of the second inverter may Will become a high standard.

As shown in Figure 15, the precharged transistor will only be turned on for a short period of time (e.g., about 1 microsecond) before the digitization of the analog voltage signal of the capacitor is measured. The comparator consumes very little or no static power during the rest of the frame time period. Therefore, the static (DC) power consumption generated by the comparator can be reduced or minimized. As a result, the overall power consumption of the digital pixel image sensor can be reduced or minimized.

Thus, the techniques disclosed herein can increase the frame rate by using a global shutter and a digitizer for each pixel, as well as reading digital values instead of analog signals for each pixel. The techniques disclosed herein can also use CDS circuits to reduce random and fixed noise (eg, offset error, dark current, and 1/f noise), thereby increasing image sensor sensitivity, signal-to-noise ratio (SNR). ), dynamic range, etc. The techniques disclosed herein can also reduce the size of each digital pixel by using a comparator (rather than a complex ADC) and a digital CDS, thereby increasing the density and resolution of the digital pixel image sensor. The techniques disclosed herein can also reduce the number of pixels by reducing the time period in which the digital pixels operate in the high power mode and by reducing the static (DC) power consumption produced by the comparators in the digitizer of the digital pixels. Power consumption.

16 is a flow chart of an exemplary digital imaging method in accordance with some embodiments. For example, the method can be performed by digital pixel image sensor 600 and/or digital pixels 700, 900, 1000, or 1100. This method can be used to capture digital video frames at a high frame rate with high sensitivity, low noise level, and low power consumption. All of the digital pixels in the digital pixel image sensor can perform the method in parallel.

In block 1610, the photodiodes of the pixels in the image sensor can receive optical signals during the exposure period. In some embodiments, the photodiode of the pixel can be reset by, for example, an electronic shutter signal when the optical signal is received, or the electronic shutter signal controls the reset of all pixels in the image sensor. Therefore, the photodiodes of all the pixels in the image sensor can be simultaneously reset by the same electronic shutter signal (ie, the global electronic shutter). The photodiode can be reset to a DC voltage level (eg, 0 volts) to vent all of the charge on the parasitic capacitance associated with the photodiode. In some embodiments, a charge storage device (ie, a measurement capacitor) such as an external or parasitic capacitance (eg, a floating diffusion node) can also be reset. In some embodiments, the reset signal (eg, reset switch 718 or reset gate 1040) and the transfer gate (eg, transfer gate 704 or 1020) may be used to reset the signal (eg, reset signal 720 in FIG. 10 or The RST signal) and the transfer gate control signal (such as the measurement control signal 708 or the TX signal in FIG. 10) are used to reset the photodiode of the pixel.

In block 1620, the pixels can convert the optical signals to voltage levels on a charge storage device (eg, an FD node). During the exposure period, the photodiode can be disconnected from the charge storage device, for example by turning off the transfer gate. The photodiode can generate a charge (such as a photoelectron or a hole) or a photocurrent in response to the reception of the optical signal. For example, for a brighter optical signal, the photodiode can generate a larger photocurrent, resulting in more charge. The charge generated by the photodiode during the exposure period can be concentrated on the integrating capacitor or integrated (integrated) by the integrating capacitor. In some embodiments, the integrating capacitor can be a parasitic capacitance associated with the photodiode and/or a circuit photodiode connected to the photodiode. After the exposure period, the charge storage device can be coupled to the photodiode, for example by turning on a transfer gate, thereby transferring at least a portion or all of the accumulated charge from the integrating capacitance to the charge storage device. In some embodiments, the charge storage device can be reset before the end of the exposure period, such as 100 microseconds before the end of the exposure period. The transferred charge can cause the voltage signal (or voltage level charge) on the charge storage device (eg, measurement capacitor 706 or FD capacitor 1030) to grow. The voltage signal level can be related to the amount of charge stored on the charge storage device, and thus can also be related to the brightness or intensity of the optical signal. The voltage signal level can also depend on the capacitance value of the charge storage device. After charge transfer, the charge storage device can be disconnected from the photodiode, for example by turning off the transfer gate.

In block 1630, the pixel (specifically, the comparator in the digitizer of the pixel) can receive a count value from the clock counter, which calculates the number of clock cycles of the clock signal. The clock counter can be a global clock counter that provides a plurality of count values to a plurality or all of the pixels of the image sensor. The clock counter can start counting the clock cycle at the beginning of the digitizing period, and can be reset to a preset value (for example, a value of 0 or greater than 0) before the start of the digitizing period or after the end of the digitizing period. .

In block 1640, the comparator of the pixel can compare the voltage level on the charge storage device to the ramp signal. As the number of clock cycles increases, the voltage level of the ramp signal can rise linearly (ie, tilt up) or fall (ie, tilt down). In some embodiments, the ramp signal can be generated by a global reference signal generator. For example, in some embodiments, the DAC can use the count value of the clock counter (the calculated number of clock cycles) as an input to generate a ramp signal, and the comparator of multiple or all pixels of the image sensor can Use this ramp signal as a reference signal. Before the voltage level on the charge storage device is transferred to the comparator, the voltage level can be sensed, for example, using a selection signal, and the voltage level is amplified, for example, by the aforementioned source-coupled amplifier or buffer. . When the ramp signal is below (or is higher for a downhill signal) the voltage level sensed (and amplified) from the charge storage device, the output of the comparator will be at a low level (or in some embodiments high) quasi). As the number of clock cycles increases, the ramp signal can gradually rise (or decrease for down-slope signals).

In block 1650, the comparator is when the ramp signal reaches a voltage level that is equal to or greater than (or is lower for the downslope signal) from the charge storage device (eg, a threshold voltage that differs by one transistor) The output may change state, such as flipping from a low level or switching to a high level.

In block 1660, the count value of the clock counter at the point in time when the output state of the comparator changes (eg, flips or switches) may be stored as a first digit value, such as an n-bit region stored in pixel memory. Among the blocks. The first digit value may correspond to the intensity of the optical signal at the pixel. The first digit value of each pixel of the image sensor can be read line by line to form a digital image frame.

In some embodiments, after the charge storage device is reset at a time point before the end of the exposure period, the second ramp signal generated by the comparator, the clock counter, and the reference signal generator can be utilized to be similar to the optical signal. The associated voltage level is digitized to digitize the voltage level of the charge storage device to a second digit value. The difference between the first digit value and the second digit value can more accurately represent the intensity of the optical signal at the pixel.

Various embodiments of the invention may comprise or be implemented with an artificial reality system. A form of reality that has been adjusted in some way before being presented to the user, and may include, for example, virtual reality (VR), augmented reality (AR), mixed reality (MR or hybrid reality), or Some combinations and/or derivatives. Artificial reality content can contain fully generated content or generated content combined with the captured content (eg, retrieved from the real world). The artificial reality content may include video, audio, tactile feedback, or some combination thereof, and any of them may be presented in a single channel or in multiple channels (eg, a stereoscopic video that produces a three-dimensional effect to the viewer). Moreover, in some embodiments, the artificial reality may be linked to an application, product, accessory, service, or some combination thereof, such as for creating content in a human reality and/or otherwise in an artificial reality. Use (for example, perform activities in it). Artificial reality systems that provide artificial reality content can be implemented on a variety of platforms, including head-mounted displays (HMDs) connected to host systems, stand-alone HMDs, mobile devices or computing systems, or any other capable of providing artificial reality. Content to a hardware platform for one or more viewers.

17 is a simplified block diagram of an example artificial reality system environment 1700 including a near-eye display 1720, in accordance with some embodiments. The artificial reality system environment 1700 shown in FIG. 17 can include a near-eye display 1720, an external imaging device 1750, and an input-output interface 1740, each of which is coupled to a console 1710. Although the example artificial reality system environment 1700 shown in FIG. 17 includes a near-eye display 1720, an external imaging device 1750, and an input-output interface 1740, the artificial reality system environment 1700 can include any number of the above-described elements, or can omit any The above components. For example, one or more external imaging devices 1750 in communication with console 1710 can monitor a plurality of near-eye displays 1720. In an alternative configuration, the artificial reality system environment 1700 can include different or additional components than those described above.

The near-eye display 1720 can be a head-mounted display (HMD) to present content to the user. Examples of media presented by near-eye display 1720 include one or more of images, video, audio, or some combination thereof. In some embodiments, the audio may be presented via an external device (eg, a speaker and/or earphone) that receives audio information from the near-eye display 1720, console 1710, or both, and presents the audio based on the audio information. data. The near-eye display 1720 can include one or more rigid bodies that can be coupled to each other rigidly or non-rigidly. The rigid coupling between the plurality of rigid bodies can make these coupled rigid bodies act as a single rigid body. The non-rigid coupling between the plurality of rigid bodies can enable the rigid bodies to move relative to each other. In various embodiments, the near-eye display 1720 can be implemented in any suitable form, including in the form of a pair of glasses. Some embodiments of the near-eye display 1720 will be further illustrated in Figures 2 and 3 below. Moreover, in various embodiments, the functionality described herein can be used in an earphone that optically or electronically combines environmental images external to the near-eye display 1720 with and from the console 1710 or from any other source to use. The content received by the console. Therefore, the near-eye display 1720 can augment the image of the entity, real-world environment outside the near-eye display 1720 with the generated content (eg, image, video, sound, etc.) to present the augmented reality to the user.

In various embodiments, the near-eye display 1720 can include display electronics 1722, display optics 1724, one or more locators 1726, one or more position sensors 1728, an eye tracking unit 1730, and an inertial measurement unit ( One or more of the inertial measurement unit (IMU) 1732. In various embodiments, the near-eye display 1720 can omit any of these elements or include additional elements. Moreover, in some embodiments, the near-eye display 1720 can include elements that combine the functions of the various elements shown in FIG.

Display electronics 1722 can display images to the user based on the information received from console 1710. In various embodiments, display electronics 1722 can include circuitry for generating an image of a virtual or real object, and/or certain components for driving display optics 1724 (such as an electronic steering mirror as detailed below) (electrically steerable mirror)) circuit. In some embodiments, display electronic component 1722 can include one or more display panels, such as a liquid crystal display (LCD), a liquid crystal overlay (LCOS) display, an organic light emitting diode (OLED) display, a micro LED. (mLED) display, active matrix organic light emitting diode display (AMOLED), transmissive organic light emitting diode display (TOLED), digital micro mirror device (DMD) or some other display. In an embodiment of the near-eye display 1720, the display electronics 1722 can comprise a TOLED panel that can include sub-pixels to emit primary color light, such as red, green, blue, white, or yellow. In some embodiments, display electronics 1722 can display 3D images through the steric effects produced by the two-dimensional panel to create subjective perception of image depth. For example, display electronics 1722 can include left and right displays that are respectively located in front of the user's left and right eyes. The left display and the right display may present image copies that are translated relative to each other to produce a stereoscopic effect.

In some embodiments, display optical component 1724 can optically display image content (eg, using an optical waveguide and coupler), or amplify image light received from display electronic component 1722 to correct optical associated with the image light. The error, combined with the image light from the display electronics 1722 and the environment, presents the corrected and combined image light to the user of the near-eye display 1720. In various embodiments, display optical element 1724 can include one or more optical elements. Examples of optical components may include substrates, optical waveguides, apertures, Fresnel lenses, convex lenses, concave lenses, filters, diffractive optical elements, or may affect from display electronic components. 1722 and any other suitable optical component of the ambient image light. Display optics 1724 can include a combination of different optical elements as well as mechanical coupling to maintain the relative spacing and orientation of the optical elements in the combination. One or more optical elements in display optics 1724 can have an optical coating such as an anti-reflective coating, a reflective coating, a filter coating, or a combination of different optical coatings.

The amplification of the image light performed by display optics 1724 can result in display electronics 1722 having a smaller volume, lighter weight, and less power consumption than larger displays. In addition, the enlargement process can enhance the field of view of the displayed content. In some embodiments, the effective focal length of display optical element 1724 can be greater than the spacing between display optical element 1724 and display electronic component 1722 to amplify the image light projected by display electronic component 1722. The degree of magnification of the image light by display optical element 1724 can be adjusted by adding or removing optical elements from display optical element 1724.

Display optics 1724 can be designed to correct for one or more optical errors, such as two-dimensional optical errors, three-dimensional optical errors, or a combination thereof. The two-dimensional error can include optical aberrations that occur in two dimensions. Example types of two-dimensional errors may include barrel distortion, pincushion distortion, longitudinal chromatic aberration, and transverse chromatic aberration. The three-dimensional error can include optical aberrations that occur in three dimensions. Example types of three-dimensional errors may include spherical aberration, comatic aberration, field curvature, and astigmatism. In some embodiments, the content provided to the display electronic component 1722 for display may be pre-distorted, and the display optical component 1724 may receive image light generated from the pre-distorted content from the display electronic component 1722, The distortion is corrected.

The plurality of locators 1726 can be disposed at a plurality of specific locations on the near-eye display 1720 relative to each other and relative to a reference point on the near-eye display 1720. The console 1710 can identify the location 1726 in the image captured by the external imaging device 1750 to determine the location, pointing, or both of the artificial reality. The locator 1726 can be a light emitting diode (LED), a corner cube reflector, a reflective marker, a light source that contrasts with the environment in which the near-eye display 1720 operates, or a combination of some of these components. In embodiments where the locator 1726 is an active component (eg, an LED or other type of illuminating device), the light emitted by the locator 1726 can be in the visible light band (eg, about 380 nm to 750 nm), infrared (IR). ) in any combination of optical bands (eg, about 750 nm to 17 nm), ultraviolet bands (eg, about 170 nm to about 380 nm), other portions of the electromagnetic spectrum, or multiple partial bands of the electromagnetic spectrum .

The external imaging device 1750 can generate slow correction data based on the correction parameters received from the console 1710. The slow correction data can include one or more images showing the observed position of the location 1726 that can be detected by the external imaging device 1750. External imaging device 1750 can include one or more cameras, one or more cameras, any other device capable of capturing images containing one or more locators 1726, or some combination of the above. Additionally, external imaging device 1750 can include one or more filters (eg, for increasing the signal to noise ratio). External imaging device 1750 can be used to detect light emitted or reflected by position 1726 in the field of view of external imaging device 1750. In embodiments where the locator 1726 includes a passive element such as a retroreflector, the external imaging device 1750 can include a light source that illuminates some or all of the locators 1726, which can reflect light back into the external imaging device 1750. Light source. The slow correction data can be transmitted from the external imaging device 1750 to the console 1710, and the external imaging device 1750 can receive one or more calibration parameters from the console 1710 to adjust one or more imaging parameters (eg, focal length, focus, frame rate, Sensor temperature, shutter speed, aperture, etc.).

Position sensor 1728 can generate one or more measurement signals in response to the action of near-eye display 1720. Examples of position sensor 1728 may include an accelerator, a gyroscope, a magnetometer, other motion detection or error correction sensors, or some combination of the above. For example, in some embodiments, position sensor 1728 can include multiple accelerators to measure translational motion (eg, forward/backward/upward/downward/leftward/rightward) and multiple gyroscopes in volume Rotate motion (such as pitch, yaw, or scroll). In some embodiments, the various position sensors can be oriented orthogonal to one another.

The IMU 1732 can be an electronic device that generates fast correction data based on measurement signals received from one or more position sensors 1728. The position sensor 1728 can be disposed outside of the IMU 1732, within the IMU 1732, or some combination thereof. Based on one or more measurement signals from one or more position sensors 1728, the IMU 1732 can generate fast correction data that represents the estimated position of the near-eye display 1720 relative to its initial position. For example, the IMU 1732 can integrate the measurement signal received from the accelerator over time to estimate the rate vector, and can integrate the rate vector over time to determine an estimated position of the reference point on the near-eye display 1720. In addition, the IMU 1732 can provide sampled measurement signals to the console 1710, which can determine the fast calibration data. While a reference point can generally be defined as a point in space, in various embodiments, the reference point can also be defined as a point within the near-eye display 1720 (eg, the center of the IMU 1732).

The eye tracking unit 1730 can include one or more eye tracking systems. The eye tracking system can include an imaging system to form an image of one or more eyes, and can optionally include an illuminator that can generate light directed at the eye such that light reflected by the eye can be captured by the imaging system. For example, the eye tracking unit 1730 can include a coherent light source (eg, a laser diode) of the emitted light system in the visible or infrared spectrum and a lens that can capture light reflected by the user's eyes. In another example, the eye tracking unit 1730 can capture reflected radio waves, where the radio waves are emitted by the micro radar unit. The eye tracking unit 1730 can use a low power illuminator to illuminate at a very high frequency that does not harm the eye or cause physical discomfort. The eye tracking unit 1730 can be configured to increase the contrast in the eye image captured by the eye tracking unit 1730 while reducing the total power consumed by the eye tracking unit 1730 (eg, reducing the illumination included in the eye tracking unit 1730). And the power consumed by the imaging system). For example, in some embodiments, the eye tracking unit 1730 can consume less than 1700 milliwatts of power.

Eye tracking unit 1730 can be used to estimate the orientation of the user's eyes. The pointing of the eye may correspond to the direction of gaze of the user within the near-eye display 1720. The orientation of the user's eye can be defined as the direction of the foveal axis, which is the axis between the fovea (the area with the highest concentration of photoreceptors on the retina of the eye) and the center of the pupil of the eye. . In general, when the user's eyes are fixed at one point, the central fovea of the user's two eyes will intersect this point. The pupil axis of the eye can be defined as the axis that passes through the center of the pupil and is perpendicular to the plane of the cornea. In general, even if the pupil axis intersects the central socket axis at the center of the pupil, the pupil axis may not be directly aligned with the central socket axis. In some eye tracking embodiments, since the foveal axis is defined by the fovea located at the back of the eye, it may be difficult or impossible to directly measure the fovea. Thus, in some embodiments, the orientation of the pupil axis can be detected, and the central socket axis may be estimated based on the detected pupil axis.

For example, the near-eye display 1720 can utilize the pointing of the eye to determine the inter-pupillary distance (IPD) of the user; determine the direction of the gaze; and introduce a depth cues (eg, blur the image outside the main line of sight of the user); Collecting heuristics on user interactions in artificial reality media (eg, time spent on any particular subject, object, or frame that is a function of exposing stimuli), based at least on the user's Some other features of an eye pointing, or some combination of them. Since the eye tracking unit 1730 can determine the orientation of the user's two eyes, it can determine where the user looks. For example, the determination of the gaze direction of the user may include determining a convergence point based on the determined orientations of the left and right eyes of the user. The point of convergence may be the intersection of the two fovea axes of the user's eyes (or the closest point between the two axes). The direction of the user's gaze may be the direction of the line passing through the convergence point and the intermediate point between the pupils of the user's eyes.

The input and output interface 1740 can be a device for the user to send an action request to the console 1710. Action requirements can be requirements for performing specific actions. For example, an action request can start or end an application or perform a specific action within the application. Input and output interface 1740 can include one or more input devices. An exemplary input device can include a keyboard, mouse, game controller, glove, button, touch screen, or any other suitable device for receiving an action request and transmitting the received action request to the console 1710. The action requirements received by the input and output interface 1740 can be communicated to the console 1710, which can perform actions corresponding to the desired action. In some embodiments, the input and output interface 1740 can provide haptic feedback to the user in accordance with instructions received from the console 1710. For example, the input and output interface 1740 can provide tactile feedback when the action request is accepted, or the console 1710 has performed the required action and transmitted an instruction to the input and output interface 1740.

The console 1710 can provide content for presentation to the user to the near-eye display 1720 based on information received from one or more of the external imaging device 1750, the near-eye display 1720, and the input-output interface 1740. In the example shown in FIG. 17, the console 1710 can include an application store 1712, an earphone tracking module 1714, an artificial reality engine 1716, and an eye tracking module 1718. Some embodiments of console 1710 may include different or additional modules than those depicted in FIG. The functions described further below may be distributed between elements of console 1710 in a manner different than that described herein.

In some embodiments, the console 1710 can include a processor and a non-transitory computer readable storage medium, wherein the non-transitory computer readable storage medium stores a plurality of instructions executable by the processor. The processor can include a plurality of processing units to execute the instructions in parallel. The computer readable storage medium can be any type of memory, such as a hard disk drive, a removable memory or a solid state drive (such as a flash memory or a dynamic random access memory (DRAM)). In various embodiments, the modules of console 1710 described in connection with FIG. 17 can be encoded as instructions in a non-transitory computer readable storage medium that, when executed by a processor, cause The processor performs the functions described further below.

The application store 1712 can store one or more applications for execution by the console 1710. An application can contain a set of instructions that, when executed by the processor, produce content for presentation to the user. The content generated by the application can be responsive to input from the user, through the user's eye movement, received input, or input received from the input and output interface 1740. Examples of applications can include game applications, conferencing applications, video playback applications, or other suitable applications.

The earphone tracking module 1714 can utilize the slow or fast correction data from the external imaging device 1750 to track the movement of the near-eye display 1720. For example, the earphone tracking module 1714 can utilize the observations from the slow correction data and the model of the near-eye display 1720 to determine the position of the reference point of the near-eye display 1720. The headset tracking module 1714 can utilize the location information from the fast calibration data to determine the location of the reference point of the near-eye display 1720. Moreover, in some embodiments, the headset tracking module 1714 can use a portion of the fast correction data, slow correction data, or some combination of the two to predict the future location of the near-eye display 1720. The headset tracking module 1714 can provide an estimated or predicted future location of the near-eye display 1720 to the artificial reality engine 1716.

The headset tracking module 1714 can use one or more correction parameters to correct the artificial reality system environment 1700, and can adjust one or more calibration parameters to reduce the error in determining the position of the near-eye display 1720. For example, the earphone tracking module 1714 can adjust the focus of the external imaging device 1750 to obtain a more accurate position of the observed position on the near-eye display 1720. In addition, the corrections performed by the headset tracking module 1714 can also take into account the information received from the IMU 1732. Additionally, if the tracking of the near-eye display 1720 is lost (eg, the external imaging device 1750 loses at least a threshold number of lines of sight 1726), the earphone tracking module 1714 can recalibrate some or all of the calibration parameters.

The artificial reality engine 1716 can execute an application within the artificial reality system environment 1700 and receive position information of the near-eye display 1720, acceleration information of the near-eye display 1720, speed information of the near-eye display 1720, the near-eye display 1720 from the earphone tracking module 1714. Predict future locations, or some combination of the above information. The artificial reality engine 1716 can also receive estimated eye position and pointing information from the eye tracking module 1718. Based on the received information, the artificial reality engine 1716 can determine the content that is provided to the near-eye display 1720 for presentation to the user. For example, if the received information indicates that the user is looking to the left, the artificial reality engine 1716 can generate content for the near-eye display 1720 that mirrors the user's eye movement in the virtual environment. Moreover, in response to the action request received from the input and output interface 1740, the artificial reality engine 1716 can perform an in-app action, wherein the application is executed on the console 1710 and the artificial reality engine 1716 can provide feedback to User, indicating that the action has been executed. The feedback may be through visual or audible feedback of the near-eye display 1720 or through tactile feedback of the input-output interface 1740.

The eye tracking module 1718 can receive the eye tracking data from the eye tracking unit 1730 and determine the eye position of the user based on the eye tracking data. The position of the eye may include an orientation, a location, or both of the eye relative to the near-eye display 1720 or any of its elements. Since the axis of rotation of the eye changes depending on the location of the eye in the orbit, determining the location of the eye in the eye socket allows the eye tracking module 1718 to more accurately determine the direction of the eye.

In some embodiments, the eye tracking unit 1730 can output eye tracking data including images of the eyes, and the eye tracking module 1718 can determine the position of the eyes based on the images. For example, the eye tracking module 1718 can store the correspondence between the plurality of images captured by the eye tracking unit 1730 and the plurality of eye positions to capture an image from the eye tracking unit 1730. Decide on a reference eye position. Alternatively or additionally, the eye tracking module 1718 can determine the updated eye position relative to the reference eye position by comparing the image that determines the reference eye position with the image from which the eye position is to be updated. The eye tracking module 1718 can utilize measurements from different imaging devices or other sensors to determine eye position. For example, as described above, the eye tracking module 1718 can utilize the measurements from the slow eye tracking system to determine the reference eye position, and then determine from the fast eye tracking system the updated eye position relative to the reference eye position until The next reference eye position is determined based on measurements from the slow eye tracking system.

The above methods, systems and devices are all examples. Various embodiments may omit, substitute, or add various processes or elements as appropriate. For example, in alternative configurations, the methods described may be performed in a different order than that described, and/or various stages may be added, omitted, and/or combined. Moreover, the features described with respect to certain embodiments may be combined in various other embodiments. Different aspects and elements of the embodiments can be combined in a similar manner. In addition, many of the described elements are exemplary, and the scope of the disclosure is not limited to those specific examples.

The specific details in the above are provided for an in-depth explanation of the embodiments. However, the embodiments may be practiced without these specific details. For example, well-known details of circuits, processes, systems, structures, and techniques are not shown in order to avoid obscuring the embodiments. The description is merely illustrative of the embodiments, and is not intended to limit the scope, the In addition, the description of the foregoing embodiments will provide those of ordinary skill in the art the implementation of the embodiments. Various changes may be made in the function and arrangement of the elements without departing from the spirit and scope of the disclosure.

Moreover, some embodiments are described as processes that are depicted as flowcharts or block diagrams. Although each embodiment describes multiple operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of these operations can be rearranged. The process can have additional steps not included in the figure. Additionally, embodiments of the methods may be implemented by hardware, software, firmware, mediation software, microcode, hardware description language, or any combination thereof. When implemented in software, firmware, mediation software or microcode, the code or code segments used to perform the relevant tasks can be stored in a computer readable medium such as a storage medium. The processor can perform the associated tasks.

It will be apparent to those skilled in the art that substantial changes can be made depending on the particular requirements. For example, hardware that is specific or specific for use may be used, and/or specific components may be implemented by hardware, software (including portable software, such as a small application, etc.) or both. Or both. In addition, connections to other computing devices such as network input and output devices may be employed.

Referring to the drawings, elements that can include memory can include non-transitory machine readable media. The terms "machine readable medium" and "computer readable medium" as used herein mean any storage medium that participates in providing material to cause the machine to operate in a particular manner. In the embodiments provided above, various machine readable media may be involved in providing instructions/code to a processing unit and/or other device for execution. Additionally or alternatively, machine readable media may be used to store and/or load instructions/code. In many embodiments, the computer can read media entities and/or tangible storage media. Such media can take many forms, including but not limited to non-volatile media, volatile media, and transmission media. For example, common forms of computer readable media include magnetic and/or optical media, such as compact disk (CD) or digital versatile disk (DVD), punch card. , paper tape, any other physical medium with a hole pattern, random access memory (RAM), programmable read-only memory (PROM), erasable programmable only Erasable programmable read-only memory (EPROM), flash erasable programmable memory (FLASH-EPROM), any other memory chip or card gate, carrier wave as described later, or computer Any other medium in which instructions and/or code are read. The computer program product may contain code and/or machine executable instructions representing a process, a function, a subroutine, a program, a routine, an application, a subroutine, a module, a package, a category. (class), or any combination of multiple instructions, data structures, or program descriptions.

Those skilled in the art will appreciate that any of a variety of different technologies and techniques can be used to represent information and signals for communicating the messages described herein. For example, the data, instructions, commands, information, signals, bits, symbols, and wafers that may be referenced throughout the above description may be by voltage, current, electromagnetic waves, magnetic or magnetic particles, light fields, or light particles, or Any combination of them is indicated.

The terms "and" and "or" as used herein may include a plurality of meanings that are also intended to depend, at least in part, on the context in which the terms are used. In general, if "or" is used to unite a list, such as A, B or C, it means A, B and C. This is used in an inclusive sense, also means A, B. Or C, here is used in a sense of exclusion. In addition, the term "one or more" may be used to describe any feature, structure, or characteristic in the singular, or may be used to describe a certain combination of a plurality of I features, structures, or characteristics. However, it should be noted that this is merely an illustrative example, and the claimed subject matter is not limited to this example. In addition, if the term "at least one" is used in conjunction with a list, such as A, B, or C, it can be interpreted to mean any combination of A, B, and/or C, such as A, AB, AC, BC, AA, ABC. , AAB, AABBCCC, etc.

Further, while certain embodiments have been described using specific combinations of hardware and software, it should be understood that other combinations of hardware and software are also possible. Certain embodiments may be practiced only in hardware, only in software, or using a combination thereof. In one example, the software may be implemented by a computer program product comprising computer code or instructions executable by one or more processors for performing any or all of the steps, operations or operations described herein. The process in which the computer program can be stored on a non-transitory computer readable medium. The various processes described herein can be implemented in any combination on the same processor or on different processors.

Where an apparatus, system, component or module is described as being used to perform certain operations or functions, such as by designing an electronic circuit to perform an operation, by programming a programmable electronic circuit (such as a microprocessor) Run computer instructions or code to perform operations, or program the processor or core to run code or instructions stored on non-transitory media, or any combination of the above. Multiple trips can be communicated using a variety of techniques, including but not limited to conventional techniques for inter-process communication, and different trip pairs can use different techniques, or the same pair of trips can be used at different times. technology.

Accordingly, the specification and drawings are to be regarded as However, it will be apparent that additions, reductions, deletions, and other modifications and changes can be made without departing from the broader spirit and scope. Accordingly, although specific embodiments are described herein, these are not intended to be limiting. Various modifications and equivalents are intended to be included within the scope of the appended claims.

100,200‧‧‧ near-eye display

105, 205‧‧‧ framework

110, 210‧‧‧ display

120a～120d, 250a, 250b‧‧‧ image sensor

130, 240a ~ 240f‧‧‧ illuminators

A~D‧‧‧ direction

235‧‧‧ eyeballs

300‧‧‧Image Sensor

310‧‧‧pixel array

320‧‧‧ analog digital conversion interface

330‧‧‧Digital Analog Conversion and Support Circuit

340‧‧‧Control circuit

400‧‧‧Active pixels

410‧‧‧Fixed light diode

412‧‧‧ Parasitic electricity

420‧‧‧Transition gate

430‧‧‧Floating diffusion capacitor

440‧‧‧Reset brake

450‧‧‧Source Parallel Amplifying Transistor

460‧‧‧Selection gate

480‧‧‧ lines

510‧‧‧Image Sensor

520, 530, 540‧‧‧

600‧‧‧Digital Pixel Image Sensor

610‧‧‧Digital Pixel Array

620‧‧‧ column driver and global signal driver circuit

630‧‧‧Global Counter

640‧‧‧Counter buffer

650‧‧‧Ramp generation and buffer circuit

660‧‧‧Sense Amplifier

670‧‧‧Amplification bias circuit

680‧‧‧ line memory

690‧‧‧Digital Blocks

695‧‧‧Power adjustment circuit

698‧‧‧Action Industry Processor Interface Circuit

700‧‧‧Digital pixels

702‧‧‧Light diode

703‧‧‧Integral Capacitance

704‧‧‧Transition gate

706‧‧‧Measure capacitance

708‧‧‧Measurement control signals

710‧‧‧Optical buffer

712‧‧‧ analog output node

718‧‧‧Reset switch

720‧‧‧Reset signal

722‧‧‧charge trough

724‧‧‧Exposure enable signal

726‧‧‧Shutter switch

750‧‧‧pixel digitizer

752‧‧‧Reference signal

754‧‧‧ Comparator

760‧‧‧Digital Output Generator

762‧‧‧clock counter

764‧‧‧pixel memory

766‧‧‧Counting output

770‧‧‧Reference signal generator

772‧‧‧Digital Analog Converter

780‧‧‧clock signal

790‧‧‧Pixel Output Bus

810‧‧‧ analog voltage signal in pixel 1

820‧‧‧ analog voltage signal in pixel 2

830‧‧‧ reference signal

840‧‧‧ Comparator output in pixel 1

850‧‧‧ Comparator output in pixel 2

860‧‧‧clock signal

870‧‧‧ clock count

D1, D2‧‧‧ digit value

T0, t1, t2‧‧‧ time points

900‧‧‧Digital pixels

910‧‧‧Fixed light diode

920‧‧‧Transition gate

930‧‧‧ Floating Diffusion Node

940‧‧‧Reset brake

950‧‧‧Selection gate

942, 952‧‧‧Optoelectronics

960‧‧‧ analog correlation double sampling circuit

970‧‧‧ comparator

980‧‧‧pixel memory

RST, SEL, TX‧‧‧ control signals

1000‧‧‧ digit pixels

1010‧‧‧Fixed light diode

1020‧‧‧Transition gate

1030‧‧‧Floating diffusion capacitor

1040‧‧‧Remove the brake

1050‧‧‧ comparator

1060‧‧‧m bit memory block

1070‧‧‧n-bit memory block

1100‧‧‧ digit pixels

1105‧‧‧Light sensor

1110‧‧‧Fixed light diode

1120‧‧‧Transition gate

1130‧‧‧Floating diffusion capacitor

1140‧‧‧Remove the brake

1150‧‧‧ comparator

1160‧‧‧ and gate

1170‧‧‧ and gate

1180‧‧‧m grid

1190‧‧‧n cells

RAMP‧‧‧Slope Signal

ENABLE_RST‧‧‧Reset enable signal

ENABLE_SIG‧‧‧Read enable signal

WR‧‧‧ write enable input

1200‧‧ ‧ time period

1210‧‧ ‧Shutter period

1212‧‧ points period

1214‧‧‧Reset period

1216‧‧‧Reset level conversion period

1218‧‧‧ Charge Transfer Period

1220‧‧‧Signal level conversion period

1222‧‧‧data export period

T_Frame‧‧‧ frame image cycle

1250‧‧‧Reset signal

1252‧‧‧First reset pulse

1254‧‧‧Second reset pulse

1260‧‧‧Transfer Control Signal

1262‧‧‧First transfer control pulse

1264‧‧‧Second transfer control pulse

1270‧‧‧Slope signal

1272, 1274‧‧ ‧ slope

1280‧‧‧ clock count

1282‧‧‧Reset count value

1284‧‧‧Signal count

1290‧‧‧Mobile Industry Processor Interface

1292‧‧‧Output data

1300‧‧‧ comparator

1310, 1320‧‧‧P channel transistor

1325‧‧‧ nodes

1330, 1340, 1350‧‧‧N-channel transistors

1360‧‧‧ reverser

VFD‧‧‧ voltage signal

VRAMP‧‧‧Slope Signal

1400‧‧‧ comparator

1410, 1430 ~ 1460‧‧ ‧ transistor

1420‧‧‧Precharged transistor

COMP‧‧‧ node

Write‧‧‧ write signal

Writeb‧‧‧Write b signal

1500‧‧‧ Timing diagram

1510‧‧‧Precharge signal

1520‧‧‧Slope signal

1530‧‧‧Voltage signal

1540‧‧‧ Voltage signal of node COMP

1550‧‧‧Write signal

1560‧‧‧Write b signal

Vtp‧‧‧ threshold voltage

Time t1‧‧‧

1700‧‧‧Artificial Reality System Environment

1710‧‧‧ console

1712‧‧‧ App Store

1714‧‧‧ Headphone Tracking Module

1716‧‧‧Artificial engine

1718‧‧‧Eye tracking module

1720‧‧‧ near-eye display

1722‧‧‧Display electronic components

1724‧‧‧Display optical components

1726‧‧‧ Locator

1728‧‧‧ position sensor

1730‧‧‧Eye tracking unit

1732‧‧‧Inertial measurement unit

1740‧‧‧Input and output interface

1750‧‧‧External imaging device

The detailed description of the following illustrative embodiments refers to the following drawings: FIG. 1A is a perspective view of a near-eye display including a simplified example of various sensors, in accordance with certain embodiments. 1B is a cross-sectional view of a near-eye display including a simplified example of a plurality of sensors, in accordance with some embodiments. 2A is a front elevational view of a near-eye display including a simplified example of multiple sensors, in accordance with certain embodiments. 2B is a cross-sectional view of a near-eye display including a simplified example of multiple sensors, in accordance with certain embodiments. 3 is a simplified block diagram of an image sensor with an example of analog pixels. 4 illustrates an active pixel of a three-electrode of an example of an active pixel sensor (APS). FIG. 5A illustrates an example state of pixels of different columns in an image sensor using a rolling shutter at a first time. FIG. 5B illustrates an example state of pixels of different columns in an image sensor using a rolling shutter at a second time. 6 is a simplified block diagram of an exemplary global shutter digital pixel image sensor in accordance with some embodiments. 7 is a simplified block diagram of an example digital pixel of an exemplary global shutter digital pixel image sensor in accordance with some embodiments. 8 illustrates example operations of example digital pixels of an exemplary global shutter digital pixel image sensor, in accordance with some embodiments. 9 is a simplified block diagram of an exemplary digital pixel including an analog correlation double sampling (CDS) circuit, in accordance with some embodiments. 10 is a simplified block diagram of an exemplary digital pixel including a digital CDS circuit, in accordance with some embodiments. Figure 11 illustrates example digital pixels including a digital CDS circuit in accordance with some embodiments. 12A illustrates an example timing cycle of an example time frame of a global shutter digital pixel sensor having a digital CDS circuit in accordance with some embodiments. 12B is a timing diagram of the operation of a digital pixel having a digital CDS circuit in a global shutter digital pixel image sensor, in accordance with some embodiments. Figure 13 depicts a comparator including a DC bias circuit. Figure 14 illustrates an example comparator including a pre-charge circuit in accordance with some embodiments. 15 is a timing diagram of the operation of the example comparator of FIG. 14 in accordance with some embodiments. 16 is a flow chart of an exemplary digital imaging method in accordance with some embodiments. 17 is a simplified block diagram of an artificial reality system environment including an example of a near-eye display that may implement some of the examples disclosed herein.

Claims (20)

A digital pixel image sensor comprising: a plurality of pixels, each of the pixels comprising: a photodiode for generating a plurality of charges in response to an optical signal; and a charge storage device for storing the photodiode The generated charges, the stored charges generate a voltage signal on the charge storage device; a pixel memory; and a digitizer comprising: a comparator for receiving a ramp signal and the voltage signal The voltage level of the ramp signal rises or falls after each cycle of the clock signal, and is used to change the voltage level after the voltage level of the ramp signal reaches a voltage level of the voltage signal. An output state of the device; and a digital output generating circuit for receiving a first number of time points corresponding to the start of the ramp signal and the comparator at a time point of the output state change of the comparator a total number of cycles of the clock signal between the time points at which the output state changes, and storing the first amount to the pixel memory, the first number corresponding to the voltage A digital value of the voltage level of the signal. The digital pixel image sensor of claim 1, further comprising a clock counter, configured to: after the start of the ramp signal, calculate a total number of the cycles of the received clock signal; and send the corresponding A counter value of the total number of cycles to the pixels. The digital pixel image sensor of claim 2, further comprising a reference signal generator, configured to: generate the ramp signal based on the counter value; and send the ramp signal to the pixels. The digital pixel image sensor of claim 1, further comprising: a transfer gate coupled to the photodiode and the charge storage device; wherein the photodiode is used to accumulate the charge during an exposure period; Wherein the transfer gate is used to: disconnect the photodiode from the charge storage device during the exposure period; after the exposure period, connect the photodiode to the charge storage device to accumulate the Charge is transferred from the photodiode to the charge storage device; and after the accumulated charge is transferred, the photodiode is disconnected from the charge storage device. The digital pixel image sensor of claim 1, further comprising: a reset circuit for resetting the charge storage device to a reset voltage level. The digital pixel image sensor of claim 5, wherein after the charge storage device is reset, the comparator is further configured to: receive a second ramp signal, wherein after each cycle of the clock signal, A voltage level of the second ramp signal rises or falls; and after the voltage level of the second ramp signal reaches the reset voltage level, the output state of the comparator is changed; wherein the second slope is When the voltage level of the signal reaches the reset voltage level and the output state of the comparator changes, the digital output generating circuit is further configured to: receive a second quantity corresponding to the second slope signal to start And a total number of cycles of the clock signal between the time point and the time point of the output state change of the comparator; and storing the second quantity to the pixel memory, the second quantity corresponding to the weight And a digit value of the voltage level; wherein the difference between the first quantity and the second quantity corresponds to an intensity of the optical signal. The digital pixel image sensor of claim 6, wherein the pixel memory comprises: an M-bit memory block for storing the second quantity; and an N-bit memory block for storing the first Quantity, where N is greater than M. The digital pixel image sensor of claim 6, further comprising: a transfer gate coupled to the photodiode and the charge storage device, wherein the transfer gate is used to: the photodiode during an exposure period Disconnecting from the charge storage device; after the exposure period, connecting the photodiode to the charge storage device to transfer the charge from the photodiode to the charge storage device; and after transferring the charges, The photodiode is disconnected from the charge storage device. The digital pixel image sensor of claim 8, wherein the reset circuit is configured to reset the charge storage device before the exposure period ends and the transfer gate connects the photodiode to the charge storage device. The digital pixel image sensor of claim 1, wherein the comparator comprises a precharge circuit coupled to an output node of the comparator, the precharge circuit for charging the output node of the comparator Until the flow level. The digital pixel image sensor of claim 10, wherein the comparator further comprises a P-channel MOS transistor, a gate of the PMOS transistor being coupled to the charge storage device, the PMOS A source of the transistor is configured to receive the ramp signal, and the output node of the comparator is coupled to a drain of the PMOS transistor. The digital pixel image sensor of claim 1, wherein each of the pixels further comprises a shutter, and the shutter of each of the pixels is controlled by the same exposure enable signal. A digital pixel of an image sensor, the digital pixel comprising: a photodiode for generating a plurality of charges in response to an optical signal; a charge storage device for storing the electric charges generated by the photodiode The stored charge generates a voltage signal on the charge storage device; a pixel memory; and a digitizer comprising: a comparator for receiving a ramp signal and the voltage signal, wherein the signal is in a clock signal After each cycle, a voltage level of the ramp signal rises or falls, and is used to change an output state of the comparator after the voltage level of the ramp signal reaches a voltage level of the voltage signal; a digital output generating circuit for receiving a first number of time points corresponding to the start of the ramp signal and the time when the output state of the comparator changes at a time point of the output state change of the comparator a total number of cycles of the clock signal between the points, and storing the first quantity to the pixel memory, wherein the first quantity corresponds to the voltage of the voltage signal A quasi-digit value. The digital pixel of claim 13, further comprising: a reset gate for resetting the charge storage device to a reset voltage level, wherein the comparator is further used after the charge storage device is reset: Receiving a second ramp signal, wherein after each period of the clock signal, a voltage level of the second ramp signal rises or falls; and the voltage level of the second ramp signal reaches the reset voltage After the leveling, the output state of the comparator is changed; wherein the digital output generating circuit is further configured to change the output state of the comparator after the voltage level of the second ramp signal reaches the reset voltage level At a time point, receiving a second number corresponding to a total number of cycles of the clock signal between a time point at which the second ramp signal begins and the time point at which the output state of the comparator changes; and storing The second quantity is to the pixel memory, and the second quantity corresponds to a digit value of the reset voltage level; wherein a difference between the first quantity and the second quantity corresponds to one of the optical signals strength. A digital imaging method, the method comprising: receiving an optical signal by a photodiode of a pixel in an image sensor during an exposure period; and converting the optical signal to the optical signal by the pixel a voltage signal on a charge storage device of the pixel; a clock counter is activated to calculate a number of clock cycles of a clock signal; and a voltage comparator and a ramp signal are compared by a comparator of the pixel, wherein the slope signal is A voltage level linearly increases or decreases with the number of clock cycles; the comparator changes an output state of the comparator when the voltage level of the ramp signal reaches a voltage level of the voltage signal And storing, at a time point of the output state change of the comparator, a first quantity corresponding to the number of clock cycles at this time to a pixel memory of the pixel as a first of the voltage signal Digital value. The method of claim 15, further comprising: disconnecting the charge storage device from the photodiode; resetting the charge storage device to a DC voltage level; activating the clock counter to calculate the clock signal The number of the clock cycles is compared by the comparator, wherein the DC voltage level and a second ramp signal are compared, wherein a voltage level of the second ramp signal linearly increases or decreases with the number of clock cycles; And storing, at a time point when the voltage level of the second ramp signal reaches the DC voltage level, a second quantity corresponding to the number of the clock cycles to the pixel memory as a second digit value. The method of claim 16, wherein resetting the charge storage device to the DC voltage level comprises: resetting the charge storage device to the DC voltage level at the end of the exposure period. The method of claim 15, wherein converting the optical signal to the voltage signal on the charge storage device comprises: disconnecting the charge storage device and the photodiode; by the photodiode, A plurality of charges are generated during the exposure period as a response to receive the optical signal; during the exposure period, the charges are accumulated in the photodiode; after the exposure period, the charge storage device is connected to the photodiode to And transferring the accumulated charges to the charge storage device, wherein the accumulated charges generate the voltage signal on the charge storage device; and disconnecting the charge storage device and the photodiode. The method of claim 15, further comprising: resetting the photodiode and the charge storage device prior to the exposure period. The method of claim 15, before comparing the voltage signal and the ramp signal, further comprising: connecting an output node of the comparator to a DC voltage source to pre-charge the output node of the comparator; The output node of the comparator is disconnected from the DC voltage source.