KR102611704B1 - Image sensor and image capturing method - Google Patents

Abstract

An image sensor according to an embodiment of the disclosed invention includes a pixel array including a plurality of pixels that output a pixel signal according to a pixel state that changes depending on the intensity of incident light; and a frame memory corresponding to the plurality of pixels and including a plurality of memory cells configured to store transmission time data of the pixel signal, wherein the pixel array is provided to respectively correspond to the plurality of pixels, , a pixel flag unit configured to set the token setting state of each pixel; and a pixel circuit configured to control the pixel flag unit according to the pixel state, wherein the pixel flag unit: sends a token signal corresponding to the pixel signal to each pixel according to the pixel state and the token setting state. The token setting state may be configured to change as the token signal is transmitted to the memory cell.

Description

Image sensor and image acquisition method {IMAGE SENSOR AND IMAGE CAPTURING METHOD}

The present invention relates to an image sensor and an image acquisition method that can reduce power consumption by reducing the number of bits each pixel transmits to a memory cell.

CMOS image sensors are used in various recognition devices. Image sensors used in mobile phones, surveillance cameras, etc. are focused on ultra-high-performance imaging and consume hundreds of mW of power. Since this causes the battery to quickly discharge when performing always-on monitoring, the need for a low-power image sensor that can reduce power consumption is emerging.

In general, the pixels and frame memory of an image sensor are manufactured into different chips through an optimized process and are connected to each other through chip-to-chip interconnection. Accordingly, when digital data is transmitted from the pixel to the frame memory, a large parasitic capacitance is driven at high speed, resulting in large power consumption.

This power consumption increases as the amount of data transmitted increases and the required transmission speed increases, and is determined by the number of bits transmitted per pixel and the number of pixels. Therefore, technology is needed to reduce the power consumed during data transmission by reducing the number of bits transmitted by each pixel.

The present invention is intended to provide an image sensor and an image acquisition method that can reduce power consumption compared to existing image sensors and image acquisition methods by minimizing the amount of data transmitted from each pixel to a frame memory.

Additionally, the present invention is intended to provide an image sensor and an image acquisition method that minimize power consumption by allowing a pixel signal to be transmitted from a pixel to a memory cell only once during one frame period.

Additionally, the present invention is intended to provide an image sensor and image acquisition method that can achieve excellent low-power performance indicators and maximize power efficiency.

In addition, the present invention is intended to provide an image sensor and image acquisition method that can achieve a lower level of random noise compared to existing image sensors and image acquisition methods despite low power consumption.

An image sensor according to one aspect of the disclosed invention includes a pixel array including a plurality of pixels that output a pixel signal according to a pixel state that changes depending on the intensity of incident light; and a frame memory corresponding to the plurality of pixels and including a plurality of memory cells configured to store transmission time data of the pixel signal, wherein the pixel array is provided to respectively correspond to the plurality of pixels, , a pixel flag unit configured to set the token setting state of each pixel; and a pixel circuit configured to control the pixel flag unit according to the pixel state, wherein the pixel flag unit: sends a token signal corresponding to the pixel signal to each pixel according to the pixel state and the token setting state. The token setting state may be configured to change as the token signal is transmitted to the memory cell.

In addition, the pixel flag unit: when the token signal is transmitted to the memory cell during a predetermined frame period, the token setting state is reset, and as the token setting state is reset, it transmits the token signal to the memory cell until the end of the frame period. It can be configured not to retransmit.

Additionally, the pixel circuit includes: a photodiode configured to output a photo-sensing voltage according to the intensity of incident light; a comparator configured to compare the photo-sensing voltage with a reference voltage and output a pixel state signal indicating the pixel state; and a switch unit configured to control the pixel flag unit according to the pixel status signal.

In addition, the pixel flag unit may include a token capacitor connected to the switch unit and charged with a token charge related to the token setting state in a charge charging terminal.

Additionally, the switch unit may include a token transfer transistor connected between the token capacitor and a column line connected to the memory cell, and the pixel state signal may be input to a gate terminal of the token transfer transistor.

Additionally, the pixel flag unit may be configured to discharge the token charge while the token transfer transistor is turned on by the pixel status signal and transmit the token signal to the memory cell through the column line.

In addition, the pixel circuit includes: a first pixel transistor connected between the charge charging terminal and the power supply terminal of the token capacitor, operating complementary to the token transfer transistor, and receiving the pixel state signal as input to a gate terminal; and a second pixel transistor connected between the gate terminal of the first pixel transistor and ground, and whose gate terminal is connected to the charge charging terminal and the drain terminal or source terminal of the first pixel transistor.

Additionally, a sense amplifier installed in the column line to amplify the token signal and output the token signal to the memory cell; and a token removal switch unit provided between the column line and ground and configured to be turned on and off complementary to the switch unit to remove a residual token signal of the column line.

Additionally, the pixel flag unit includes: a token storage unit that stores the token setting state; an AND logic gate configured to receive the token setting state, the pixel state, and the pixel selection signal and perform an AND operation to output a token transmission signal; and a switch unit provided between the token storage unit and the memory cell and configured to transmit the token signal from the token storage unit to the memory cell according to the token transmission signal.

A method of acquiring an image by the image sensor according to one aspect of the disclosed invention includes setting a token setting state of each pixel of the image sensor by the pixel flag unit; detecting, by a pixel circuit, a pixel state of each pixel of the image sensor that changes depending on the intensity of incident light, and controlling the pixel flag unit according to the pixel state; transmitting, by the pixel flag unit, a token signal corresponding to a pixel signal to a memory cell corresponding to each pixel according to the pixel state and the token setting state; and resetting the token setting state of each pixel by transmitting the token signal to the memory cell by the pixel flag unit.

In addition, after the token setting state is reset during a set frame period, blocking retransmission of the token signal to the memory cell until the end of the frame period may be further included.

In addition, the step of controlling the pixel flag unit includes: outputting a photo-sensing voltage according to the intensity of incident light by a photo diode; Comparing the photo-sensing voltage with a reference voltage using a comparator to output a pixel state signal indicating the pixel state; and controlling the pixel flag unit according to the pixel state signal by a switch unit.

Additionally, the step of setting the token setting state may include: charging a token charge to the charge terminal of a token capacitor connected to the switch unit.

Additionally, amplifying the token signal transmitted to the column line by a sense amplifier and outputting the token signal to the memory cell; and removing the residual token signal of the column line by a token removal switch unit that is turned on and off complementary to the switch unit.

In addition, performing an AND operation on the token setting state, the pixel state, and the pixel selection signal stored in the token storage portion of the pixel flag unit by an AND logic gate to output a token transmission signal; and transmitting, by a switch unit, the token signal from the token storage unit to the memory cell according to the token transmission signal.

According to one aspect of the disclosed invention, the amount of data transmitted from each pixel to the frame memory can be minimized to 1 bit per frame, thereby reducing power consumption compared to existing image sensors and image acquisition methods.

Additionally, according to an embodiment of the present invention, power consumption can be minimized by transmitting a pixel signal from a pixel to a memory cell only once during one frame period.

Additionally, according to an embodiment of the present invention, excellent low-power performance indicators can be achieved and power efficiency when acquiring images by an image sensor can be maximized.

In addition, according to the embodiment of the present invention, despite low power consumption, a lower level of random noise can be achieved compared to existing image sensors and image acquisition methods.

1 is a configuration diagram of an image sensor according to an embodiment.
Figure 2 is another configuration diagram of an image sensor according to an embodiment.
FIG. 3 is a diagram illustrating a process for storing transmission time data of a pixel signal in a memory cell in one embodiment.
FIG. 4 is a diagram illustrating a photodiode according to an embodiment.
FIG. 5 is a diagram illustrating obtaining a pixel signal as a point in time until a constant voltage changes in each pixel, according to an embodiment.
Figure 6 is a diagram illustrating a pixel circuit according to one embodiment.
FIG. 7 is a diagram showing how pixels and memory cells are sequentially connected during one frame period, according to one embodiment.
FIG. 8 is a diagram illustrating how pixels and memory cells are sequentially connected during a first scan cycle period, according to one embodiment.
FIG. 9 is a diagram showing that pixels and memory cells are sequentially connected during a second scan cycle period, according to one embodiment.
FIG. 10 is a diagram showing that pixels and memory cells are sequentially connected during a third scan cycle period, according to one embodiment.
FIG. 11 is a diagram illustrating a circuit showing the connection relationship between a switch unit, a token capacitor, and a sense amplifier according to an embodiment.
FIG. 12 is a diagram illustrating a circuit showing the connection relationship between transistors according to an embodiment.
Figure 13 is a diagram for explaining the operation of a circuit according to one embodiment.
Figure 14 is another diagram for explaining the operation of a circuit according to an embodiment.
FIG. 15 is a diagram illustrating a circuit including an AND logic gate according to an embodiment.
Figure 16 is a flowchart of an image acquisition method according to one embodiment.
Figure 17 is a graph and table showing the performance of an image sensor according to an embodiment.
Figure 18 is a table showing the degree to which an image sensor according to an embodiment is improved over a conventional sensor.

Like reference numerals refer to like elements throughout the specification. This specification does not describe all elements of the embodiments, and general content or overlapping content between the embodiments in the technical field to which the disclosed invention pertains is omitted. The term '~unit' used in the specification may be implemented as software or hardware, and depending on the embodiments, multiple '~units' may be implemented as one component, or one '~unit' may be implemented as a plurality of components. It is also possible to include elements.

Throughout the specification, when a part is said to be “connected” to another part, this includes cases where it is electrically connected, and includes cases where it is connected not only directly, but also indirectly.

Additionally, when a part "includes" a certain component, this means that it may further include other components rather than excluding other components, unless specifically stated to the contrary.

Terms such as first and second are used to distinguish one component from another component, and the components are not limited by the above-mentioned terms.

Singular expressions include plural expressions unless the context clearly makes an exception.

The identification code for each step is used for convenience of explanation. The identification code does not explain the order of each step, and each step may be performed differently from the specified order unless a specific order is clearly stated in the context. there is.

Hereinafter, the operating principle and embodiments of the disclosed invention will be described with reference to the attached drawings.

FIG. 1 is a configuration diagram of an image sensor according to an embodiment, and FIG. 2 is another configuration diagram of an image sensor according to an embodiment.

1 and 2, the image sensor 1 according to an embodiment of the present invention includes a pixel array 100 in which a plurality of pixels 110 are arranged, and a frame memory in which a plurality of memory cells 210 are arranged. (200), and may include a column line (300).

The image sensor 1 can be divided into a charge coupled device (CCD) and a CMOS image sensor (Complementary Metal-Oxide Semiconductor Image Sensor; CIS). The image sensor 1 according to one embodiment may be a CMOS image sensor, but the image sensor 1 of the present invention is not limited thereto.

CMOS image sensors can be used in various recognition devices. Since always-on monitoring operation is generally required in recognition devices, battery consumption becomes the standard for performance, and low-power operation of the sensor is essential. Generally, a large amount of power is consumed to transmit a large amount of data acquired from an image sensor to a frame memory 200 at high speed. This power consumption is proportional to the amount of data transmitted, which is determined by the number of bits and pixels transmitted per pixel 110. Therefore, if the number of bits transmitted per pixel 110 can be reduced, power consumption can also be reduced.

The pixel array 100 may include a plurality of pixels 110. That is, the pixel array 100 may include a plurality of pixels 110 arranged in a matrix or the like according to certain rules.

Each of the plurality of pixels 110 may output a pixel signal according to a pixel state that changes depending on the intensity of incident light.

The pixel state may be a state of each pixel 110 that changes over time when each pixel 110 receives incident light. In an embodiment, the pixel state may be changed by the output voltage or output current of the photodiode of each pixel 110, which converts incident light into an electrical signal and outputs it. A pixel signal may be a signal whose pixel state changes over time and is output when certain conditions are met.

The frame memory 200 may include a plurality of memory cells 210. Each of the plurality of memory cells 210 corresponds to a plurality of pixels 110 and may be configured to store transmission time data of the pixel signal. At this time, the memory cell 210 may be connected to the corresponding pixel 110 through a column line 300.

In an embodiment, the plurality of pixels 110 and the plurality of memory cells 210 may be connected in a one-to-one correspondence, but two or more pixels among the plurality of pixels 110 may be connected to one memory cell 210, or Some pixels among the plurality of pixels 110 may be connected to the plurality of memory cells 210.

Each pixel 110 of the pixel array 100 may include a pixel flag unit 130 and a pixel circuit 120.

The pixel flag unit 130 is provided to correspond to each of the plurality of pixels 110, and may be configured to set the token setting state of each corresponding pixel 110. At this time, the pixel flag unit 130 may be included as a component of the corresponding pixel 110, but is not necessarily included in the corresponding pixel 110.

A token may be a concept provided for each pixel 110. Specifically, the token of each pixel 110 sends a signal to the memory cell 210 corresponding to the pixel 110 only once during the frame cycle when incident light is input to the corresponding pixel 110 in one frame cycle. It could be a concept for The token setting state may be information indicating whether the aforementioned token is set for each pixel 110.

The pixel flag unit 130 may transmit a token signal to the memory cell 210 corresponding to each pixel 110 according to the pixel state and token setting state.

The token signal is a signal corresponding to a pixel signal and may be a signal transmitted to indicate when the state of the pixel 110, which changes depending on the intensity of incident light, satisfies a condition.

That is, when a token signal is transmitted from the pixel 110 to the memory cell 210, the memory cell 210 can determine when the token signal was transmitted by a timer, and based on this transmission time The memory cell 210 can determine the intensity of light incident on the corresponding pixel 110.

As such, the image sensor 1 according to one embodiment has a pixel 110 corresponding to each memory cell 210, and each pixel 110 generates a voltage when transmitting the received light intensity information to the memory cell 210. Rather than transmitting information about the size, a simple 1-bit signal can be transmitted. The memory cell 210 can determine the intensity of light incident on the corresponding pixel 110 based on the time when this 1-bit signal is received. As a result, the pixel 110 of the present invention can transmit information about the intensity of light to the memory cell as a signal of only 1 bit, and thus the amount of power consumed by each pixel 110 can be reduced.

Meanwhile, even if incident light is incident on the pixel 110 once, repeatedly transmitting the token signal to the memory cell 210 throughout one frame period of acquiring an image may result in unnecessary energy consumption.

Specifically, in the present invention, one frame period consists of a plurality of scan cycle periods. A method of acquiring an image during one frame repeats the scan cycle, determines when the state of the pixel 110 changed if there is a pixel 110 among the plurality of pixels 110, and determines when the state of the pixel 110 changed. Based on the viewpoint, the intensity of light received by the corresponding pixel 110 in the corresponding frame is determined. Therefore, once the token signal has been transmitted to the memory cell 210, the intensity of light received by the corresponding pixel 110 in the corresponding frame can already be known, and it is no longer necessary to transmit a specific signal to the memory cell 210. You can. That is, if the token signal is transmitted to the memory cell 210 during one scan cycle period, in terms of energy saving, the token signal is not transmitted to the memory cell 210 in the next scan cycle cycles until the end of the frame cycle. desirable.

The pixel flag unit 130 may be configured to change the token setting state as the token signal is transmitted to the memory cell 210.

The pixel circuit 120 may be configured to control the pixel flag unit 130 according to the pixel state. At this time, the pixel circuit 120 may be included as a component of the corresponding pixel 110, but is not necessarily included in the corresponding pixel 110.

FIG. 3 is a diagram illustrating a process for storing transmission time data of a pixel signal in a memory cell in one embodiment.

Referring to FIG. 3, when a token signal is transmitted to the memory cell 210 during a set frame period, the pixel flag unit 130 resets the token setting state, and as the token setting state is reset, the memory cell 130 is reset until the end of the frame period. It may be configured not to retransmit the token signal to 210.

That is, if the token signal has already been transmitted in a certain frame period, the pixel flag unit 130 can change the token setting state so that the token signal is not transmitted again during that frame period. In other words, at this time, it is determined that the token that has already been set has been delivered to the memory cell 210, and it can be said that there are no more tokens to be delivered.

Meanwhile, if no incident light is input to the pixel 110 in a frame period, a signal may not be sent to the memory cell 210 corresponding to the pixel 110 during the frame period. In this case, the token may continue to be set during the frame period.

In Figure 3, the state of each pixel can be divided into a state before inversion (BT; before threshold) and a state after inversion (AT; after threshold).

For example, the state (sm) of a pixel may be inverted at any time tm. The time point tm can be detected by a repeated scan cycle.

If the pixel 110 is in the pre-inversion state (BT) during one scan cycle period, nothing may occur in the memory cell 210. This situation may be maintained until the nth scan cycle (Cycle n). In the nth scan cycle, the state (sm) of the pixel is inverted and changed to the state (AT), and the memory cell 210 sets a timer value when the corresponding pixel 110 is selected in the scan cycle cycle. Data can be recorded by latching.

Here, while sm is maintained at AT in the scan cycle after the nth scan cycle (Cycle n), latch should not occur again in the memory cell. Therefore, once a pixel has latched a memory cell, it must not send a signal to the memory cell again during that frame period.

It is desirable for each memory cell 210 to latch the timer value only when the state (sm) of the pixel changes in one frame period. When a new frame cycle starts, that is, in the reset phase, the pixel 110 can reset the pixel state (sm) to the state before inversion (BT) and set a token in the pixel flag unit 130. there is.

The token remains set while the pixel's state is in the before-inversion state (BT), and when the pixel's state (sm) changes to the after-inversion state (AT) in the nth scan cycle (Cycle n), the token remains set. A token (tkn) may be transmitted to latch the memory cell 210 with a timer value.

In this next scan cycle, because the token (tkn) has already been used, the memory cell 210 may not be latched again until the next frame cycle begins and the state (sm) of the pixel is reset.

In conclusion, if there is light incident on a specific pixel 110 during one frame period, the pixel 110 can transmit a small signal corresponding to 1 bit to the memory cell 210 only once in that frame, and the This can have the effect of saving energy rather than transmitting signals through .

The pixel circuit 120, the pixel flag unit 130, and the memory cell 210 may include one processor among a plurality of processors included in the image sensor 1. Additionally, the image acquisition method according to the embodiments of the present invention described so far and the embodiments to be described in the future may be implemented in the form of a program that can be driven by a processor.

Here, the program may include program instructions, data files, and data structures, etc., singly or in combination. Programs may be designed and produced using machine code or high-level language code. The program may be specially designed to implement the above-described method for modifying the code, or may be implemented using various functions or definitions known and available to those skilled in the art in the field of computer software. A program for implementing the above-described information display method may be recorded on a recording medium readable by a processor.

The memory can store programs that perform the operations described above and the operations described later, and the memory can execute the stored programs. In the case where there are multiple processors and memories, it is possible for them to be integrated into one chip or to be provided in physically separate locations. The memory may include volatile memory such as SRAM (Static Random Access Memory, S-RAM) and D-Lab (Dynamic Random Access Memory) for temporarily storing data. In addition, memory is non-volatile memory such as Read Only Memory (ROM), Erasable Programmable Read Only Memory (EPROM), and Electrically Erasable Programmable Read Only Memory (EEPROM) for long-term storage of control programs and control data. may include.

The processor may include various logic circuits and operation circuits, process data according to a program provided from memory, and generate control signals according to the processing results.

FIG. 4 is a diagram illustrating a photodiode according to an embodiment, and FIG. 5 is a diagram illustrating obtaining a pixel signal until a certain voltage changes in each pixel 110 according to an embodiment. am.

The pixel circuit 120 may include a photodiode 121 configured to output a photo-sensing voltage according to the intensity of incident light.

4 and 5, the photodiode (PD: photodiode) 121 of the pixel 110 converts the incident light into photocurrent, which is connected to the junction capacitor of the photodiode 121. ) can appear as a voltage drop. Here, light intensity information can be expressed as the amount of voltage change versus time, that is, the slope of the voltage change.

To measure this slope, conventional imaging systems fix the measurement time and measure the voltage change during that time. This method based on voltage change exposes the photodiode 121 to light and outputs a signal to the column line 300 by a pixel selection signal after a certain period of time.

Meanwhile, in order to know the slope of the voltage change of the photodiode 121, the amount of voltage change can be fixed and the time taken to reach the change amount can be measured.

The existing method of reading the change in voltage acquires the voltage for each pixel 110 and obtains it as an N-bit digital signal, but the method of measuring time is to obtain the voltage of the junction capacitor at each pixel 110 at a constant voltage ( The time until it changes to VTH) can be obtained as a 1-bit digital signal.

The pixel flag unit 130 according to one embodiment may transmit information of the time until a certain voltage changes in each pixel 110 described above to the memory cell. As a result, in one frame, when the voltage of the junction capacitor becomes a constant voltage (VTH), a 1-bit signal is transmitted to the memory cell 210, thereby transmitting the intensity information of light incident to the corresponding pixel 110 to the memory cell. It can be delivered to (210).

Figure 6 is a diagram illustrating a pixel circuit according to one embodiment.

The intensity of light can be determined by performing time modulation to measure the time it takes to change a certain voltage and finding the point of inversion of the output through a timer.

Referring to (a) of FIG. 6, the pixel circuit 120 may include a diode and a comparator 122.

The comparator 122 may compare the photo-sensing voltage (VPD) with the reference voltage (VTH) and output a pixel state signal indicating the pixel state.

Referring to (b) of FIG. 6, the photo-sensing voltage (VPD) decreases in size with a constant voltage change, and when the reference voltage with a constant photo-sensing voltage (VPD) becomes the same value as the reference voltage (VTH), the pixel (110) ) can be inverted. By measuring the time (T _p ) from the point of reset until the pixel output is inverted, the intensity of light received by the corresponding pixel 110 in the corresponding frame can be measured. At this time, various methods such as Pulse Width Modulation (PWM) and Pulse Frequency Modulation (PFM) can be used.

FIG. 7 is a diagram showing how pixels and memory cells are sequentially connected during one frame period, according to one embodiment.

Referring to FIG. 7, one frame period may consist of a plurality of scan cycle periods.

The state sm of the pixels 110 arranged in the mth row in the pixel array 100 may be inverted at any time tm. The time point tm can be detected by a repeated scan cycle.

That is, in the cycle of each scan cycle, the pixel 110 and the memory cell 210 can be selected sequentially from the first row to the last row, and the pixel 110 arranged in the mth row can also be selected in one scan cycle cycle. It can be selected once during a period.

If the pixel 110 in the mth row is in the pre-inversion state (BT) during the first scan cycle period, nothing may occur in the memory cell 210. This situation may be maintained until the nth scan cycle (Cycle n). In the nth scan cycle, the state (sm) of the pixel is inverted and then changed to the state (AT), and the memory cell 210 starts a timer (timer) when the pixels 110 in the mth row are selected in the scan cycle cycle. ) You can record data by latching the value.

FIG. 8 is a diagram showing that pixels and memory cells are sequentially connected during a first scan cycle period according to an embodiment, and FIG. 9 is a diagram showing pixels and memory cells are sequentially connected during a second scan cycle period according to an embodiment. 10 is a diagram showing that pixels and memory cells are sequentially connected during the third scan cycle period according to one embodiment.

Referring to FIG. 8, during the first scan cycle period of one frame period, the pixel 110 and the memory cell 210 may be synchronized and sequentially selected once.

After one scan cycle, all pixels 110 may be mapped once to the memory cell 210 corresponding to each pixel 110.

If light is not incident on the pixel 110 in the scan cycle, nothing may occur even if the pixel 110 is mapped to the memory cell 210.

As shown in FIG. 8, even if light is incident on the pixel 110 in the corresponding scan cycle, the pixel signal Sp may not be output when the light detection voltage VPD is greater than a certain reference voltage VTH. Also, the token signal may not be transmitted to the memory cell 210.

Referring to FIG. 9, during the second scan cycle period of one frame period, the pixel 110 and the memory cell 210 may be synchronized and sequentially selected once.

When the light detection voltage (VPD) decreases and becomes equal to the constant reference voltage (VTH), the pixel signal (Sp) may be output by the comparator 122.

If the pixel 110 is mapped to the memory cell 210 while the pixel signal Sp is output, the token signal may be transmitted to the memory cell 210. That is, the memory cell 210 can determine, through a timer, which scan cycle is the scan cycle in which the photo-sensing voltage (VPD) and the reference voltage (VTH) become equal.

Referring to FIG. 10, during the third scan cycle of one frame period, the pixel 110 and the memory cell 210 may be synchronized and sequentially selected once.

At this time, although the pixel 110 and the corresponding memory cell 210 are mapped while the pixel signal Sp is output, the token signal is already transmitted to the memory cell ( 210), the token setting state is reset and the token signal may not be retransmitted to the memory cell 210 until the end of the corresponding frame period.

When one frame period ends, the token setting state is reset, and the pixel 110 and the memory cell 210 are synchronized and the process of sequentially selecting once during a plurality of cycle periods included in the next frame period may be repeated.

FIG. 11 is a diagram illustrating a circuit showing the connection relationship between a switch unit, a token capacitor, and a sense amplifier according to an embodiment.

Referring to FIG. 11, the pixel circuit 120 may further include a switch unit 123.

The switch unit 123 may be configured to control the pixel flag unit 130 according to the pixel status signal. Specifically, the switch unit 123 may be configured to electrically connect the pixel flag unit 130 and the column line 300 when receiving a pixel state signal from the comparator 122.

The pixel flag unit 130 may include a token capacitor 131. The token capacitor 131 is connected to the switch unit 123, and a token charge related to the token setting state may be charged to the charge charging terminal. Even if the token capacitor 131 is discharged once, it can be recharged when a new frame cycle begins.

When the switch unit 123 receives the pixel status signal, it electrically connects the token capacitor 131 and the column line 300 to discharge the token charge and transmits the token signal to the memory cell 210 through the column line 300. It can be done as much as possible.

The token capacitor 131 included in a specific pixel 110 may be connected to the column line 300 by closing the selection switch selm when the corresponding pixel 110 is selected in one scan cycle.

Meanwhile, since the charge amount (CTKN) of the token capacitor 131 is generally much smaller than the parasitic capacitance of the column line 300, the voltage of the column line 300 changes very small to a few mV. To make this into a binary signal, a sense amplifier 310 can be connected between the column line 300 connected to the pixel array 100 and the column line 300 connected to the frame memory 200.

The sense amplifier 310 may be installed in the column line 300 to amplify the token signal and output it to the memory cell 210.

The output WR of the sense amplifier 310 is a signal applied to the selection switch selm, and the selected memory cell 210 may be latched with a timer value.

The token removal switch unit 320 may be provided between the column line 300 and ground.

The token removal switch unit 320 may be configured to operate complementary to the switch unit 123 to remove the residual token signal of the column line 300.

Specifically, when the signal applied to the selection switch (selm) is turned off, the token removal switch unit 320 is closed by the CLR signal, thereby removing the remaining charges on the column line 300, and the influence of the previous token is removed from the column line 300. You can make sure it doesn't get left behind.

In this way, the image sensor 1 according to one embodiment can further reduce power consumption by minimally moving the column line 300 to transmit a 1-bit signal.

FIG. 12 is a diagram showing a circuit showing the connection relationship of transistors according to an embodiment, FIG. 13 is a diagram for explaining the operation of the circuit according to an embodiment, and FIG. 14 is a diagram showing the operation of the circuit according to an embodiment. This is another drawing to explain.

The switch unit 123 may include a token transfer transistor 124.

Referring to FIG. 12 , the token transfer transistor 124 may be connected between the token capacitor 131 and the column line 300 connected to the memory cell 210.

The pixel status signal may be input to the gate terminal of the token transfer transistor 124.

The pixel flag unit 130 may be configured to discharge the token charge while the token transfer transistor 124 is turned on by the pixel status signal and transmit the token signal to the memory cell 210 through the column line 300. .

12, 13, and 14, the pixel circuit 120 may include a first pixel transistor 125 and a second pixel transistor 126.

The first pixel transistor 125 is connected between the charge terminal and the power supply terminal of the token capacitor 131, and a pixel state signal can be input to the gate terminal. The first pixel transistor 125 may operate complementary to the token transfer transistor 124.

The second pixel transistor 126 may be connected between the gate terminal of the first pixel transistor 125 and ground, and the gate terminal may be connected to a charge charging terminal and a drain terminal or source terminal of the first pixel transistor 125.

The light detection voltage (VPD) can be input to a 2-stage comparator. The two-stage comparator may include an M1 transistor and an M2 transistor. At this time, the threshold voltage of the M1 transistor may be VTH1.

The output of a two-stage comparator may be input to the left side of the pixel flag unit 130.

In the reset phase of each frame cycle, token capacitor 131 is charged to VDD. At this time, the reference voltage (VREF) of the two-stage comparator is set to VDD to reset the photodiode 121.

After the reset phase of each frame period, the reference voltage (VREF) of the second-stage comparator can be set to an arbitrary voltage lower than VDD.

While the photo-sensing voltage (VPD) is lowered, the state of the second-stage comparator can be maintained until the M1 transistor is turned on.

When the photo-sensing voltage (VPD) of the M1 transistor decreases by VTH, that is, the value of the photo-sensing voltage (VPD) becomes VREF-VTH1, and the corresponding pixel 110 is selected and the pixel selection signal (sel) is input, the token All charges charged in the capacitor 131 are transmitted to the column line 300. When the charge transfer is completed, the pixel selection signal (sel) is turned off and the token removal switch unit 320 is closed to completely discharge the column line 300.

FIG. 15 is a diagram illustrating a circuit including an AND logic gate according to an embodiment.

Referring to FIG. 15, the pixel flag unit 130 may include a token storage unit 132, an AND logic gate 133, and a switch unit 123.

The token storage unit 132 may store the token setting state. The token storage unit 132 may charge the token capacitor of the token storage unit 132 by receiving a setting signal (eg, token charge charging voltage) at the beginning of each frame cycle.

The AND logic gate 133 may receive the token setting state (tkn), pixel state (Sp), and pixel selection signal (sel), perform AND operation, and output a token transmission signal.

In other words, the token setting state of the AND logic gate 133 is not logic '0' because the token has already been transmitted in one frame cycle, and the pixel state (Sp) changes after a certain period of time after light is incident on the pixel 110. It becomes logic '1', and when the signal (sel) (logic '1') for selecting the corresponding pixel 110 in the current cycle period is input, the token signal stored in the token storage unit 132 is output. A token transmission signal corresponding to ' can be output.

The AND logic gate 133 detects that the token has already been transmitted in one frame cycle and the token setting state corresponds to logic '0', the pixel state (Sp) corresponds to logic '0', or the corresponding pixel (110) in the cycle cycle. ), the token transmission signal (token non-transmission signal) corresponding to logic '0' is used so that no signal is output from the pixel 110 to the memory cell. Can be printed.

The switch unit 123 is provided between the token storage unit 132 and the memory cell 210, and may be configured to transmit a token signal from the token storage unit 132 to the memory cell 210 according to the token transmission signal. .

At least one component may be added or deleted in response to the performance of the components described above. Additionally, it will be easily understood by those skilled in the art that the mutual positions of the components may be changed in response to the performance or structure of the system.

Figure 16 is a flowchart of an image acquisition method according to one embodiment. This is only a preferred embodiment for achieving the purpose of the present invention, and of course, some components may be added or deleted as needed.

Referring to FIG. 16, the pixel flag unit 130 can set the token setting state of each pixel 110 of the image sensor 1 (1001).

The pixel circuit 120 may detect the pixel state of each pixel 110 of the image sensor 1 that changes depending on the intensity of incident light and control the pixel flag unit 130 according to the pixel state.

Specifically, the photodiode 121 may output a photo-sensing voltage according to the intensity of incident light (1002).

The comparator 122 may compare the photo-sensing voltage with a reference voltage and output a pixel state signal indicating the pixel state (1003).

The switch unit 123 may control the pixel flag unit 130 according to the pixel status signal (1004). Specifically, the switch unit 123 may be configured to electrically connect the pixel flag unit 130 and the column line 300 when receiving a pixel state signal from the comparator 122.

The pixel flag unit 130 may transmit a token signal corresponding to the pixel signal to the memory cell 210 corresponding to each pixel 110 according to the pixel state and token setting state (1005).

The pixel flag unit 130 may reset the token setting state of each pixel 110 by transmitting the token signal to the memory cell 210 (1006).

After the token setting state is reset during a set frame period, the pixel flag unit 130 may block retransmission of the token signal to the memory cell 210 until the end of the frame period (1007).

In order to verify the performance of the image sensor 1 according to an embodiment of the present invention, a study was conducted using a prototype image sensor 1.

At this time, a 120x120 pixel array 100 and a 120x120x8-bit frame memory 200 were configured. The pixel array 100 and the frame memory 200 can generally be implemented as separate chips using different processes, but in the image sensor 1 of this prototype, they are implemented on the same chip, and are implemented using metal lines. The columns were connected to each other through metal lines.

FIG. 17 is a graph and table showing the performance of an image sensor according to an embodiment, and FIG. 18 is a table showing the degree to which the image sensor according to an embodiment is improved over a conventional sensor.

Referring to FIG. 17, it can be seen that the power consumption required for the prototype image sensor 1 to transmit data and store it in memory is 17.2 μW at a frame of 40 fps.

In addition, it can be confirmed that the prototype image sensor 1 shows a figure-of-merit (FoM) value of 29.8pJ/pixel-frame at a frame of 40fps.

Referring to FIG. 18, it can be seen that the prototype image sensor 1 of the present invention used in the experiment is superior to other image sensors in terms of figure-of-merit (FoM) value. In addition, it was confirmed that the prototype image sensor 1 of the present invention is effective in achieving a relatively low level of random noise compared to other image sensors, despite low power consumption.

The image sensor and image acquisition method according to the embodiment of the present invention as described above can be used in devices that require imaging, especially Internet of Things (IoT) devices that require low power characteristics or the use of portable batteries, and wearable devices. It can be used in a variety of application fields, including wearable devices, face recognition door locks, eye tracking virtual reality headsets, artificial intelligence robots, etc. The attached drawings are as shown above. The disclosed embodiments have been described with reference to the following. A person skilled in the art to which the present invention pertains will understand that the present invention can be practiced in forms different from the disclosed embodiments without changing the technical idea or essential features of the present invention. The disclosed embodiments are illustrative and should not be construed as limiting.

1: Image sensor
100: pixel array
110: pixel
120: pixel circuit
121: photodiode
122: comparator
123: switch part
124: Token transfer transistor
125: first pixel transistor
126: second pixel transistor
130: Pixel flag section
131: token capacitor
132: Token storage unit
133: AND logic gate
200: frame memory
210: memory cell
300: Column line
310: sense amplifier
320: Token removal switch unit

Claims (15)

a pixel array including a plurality of pixels that output pixel signals according to pixel states that change depending on the intensity of incident light; and
A frame memory including a plurality of memory cells corresponding to the plurality of pixels and configured to store transmission time data of the pixel signal,
The pixel array is,
a pixel flag unit provided to correspond to each of the plurality of pixels and configured to set a token setting state of each pixel; and
A pixel circuit configured to control the pixel flag unit according to the pixel state,
The pixel circuit is:
a photodiode configured to output a photo-sensing voltage according to the intensity of incident light;
a comparator configured to compare the photo-sensing voltage with a reference voltage and output a pixel state signal indicating the pixel state; and
A switch unit configured to control the pixel flag unit according to the pixel status signal,
The pixel flag section:
Transmitting a token signal corresponding to the pixel signal to a memory cell corresponding to each pixel according to the pixel state and the token setting state, and configured to change the token setting state as the token signal is transmitted to the memory cell,
It is connected to the switch unit and includes a token capacitor in which a token charge related to the token setting state is charged to the charge charging terminal,
The switch unit includes a token transfer transistor connected between the token capacitor and a column line connected to the memory cell,
configured to discharge the token charge while the token transfer transistor is turned on by the pixel state signal and transmit the token signal to the memory cell through the column line,
The image sensor wherein the pixel status signal is input to the gate terminal of the token transfer transistor. According to paragraph 1,
The pixel flag section:
When the token signal is transmitted to the memory cell during a set frame period, the token setting state is reset, and as the token setting state is reset, the image sensor is configured not to retransmit the token signal to the memory cell until the end of the frame period. . delete delete delete delete a pixel array including a plurality of pixels that output pixel signals according to pixel states that change depending on the intensity of incident light; and
A frame memory including a plurality of memory cells corresponding to the plurality of pixels and configured to store transmission time data of the pixel signal,
The pixel array is,
a pixel flag unit provided to correspond to each of the plurality of pixels and configured to set a token setting state of each pixel; and
A pixel circuit configured to control the pixel flag unit according to the pixel state,
The pixel circuit is:
a photodiode configured to output a photo-sensing voltage according to the intensity of incident light;
a comparator configured to compare the photo-sensing voltage with a reference voltage and output a pixel state signal indicating the pixel state; and
A switch unit configured to control the pixel flag unit according to the pixel status signal,
The pixel flag section:
Transmitting a token signal corresponding to the pixel signal to a memory cell corresponding to each pixel according to the pixel state and the token setting state, and configured to change the token setting state as the token signal is transmitted to the memory cell,
It is connected to the switch unit and includes a token capacitor in which a token charge related to the token setting state is charged to the charge charging terminal,
The switch unit includes a token transfer transistor connected between the token capacitor and a column line connected to the memory cell,
The pixel state signal is input to the gate terminal of the token transfer transistor,
The pixel circuit is:
a first pixel transistor connected between the charge charging terminal and the power supply terminal of the token capacitor, operating complementary to the token transfer transistor, and inputting the pixel state signal to a gate terminal; and
An image sensor further comprising a second pixel transistor connected between a gate terminal of the first pixel transistor and ground, and a gate terminal connected to the charge charging terminal and a drain terminal or source terminal of the first pixel transistor. a pixel array including a plurality of pixels that output pixel signals according to pixel states that change depending on the intensity of incident light; and
A frame memory including a plurality of memory cells corresponding to the plurality of pixels and configured to store transmission time data of the pixel signal,
The pixel array is,
a pixel flag unit provided to correspond to each of the plurality of pixels and configured to set a token setting state of each pixel; and
A pixel circuit configured to control the pixel flag unit according to the pixel state,
The pixel circuit is:
a photodiode configured to output a photo-sensing voltage according to the intensity of incident light;
a comparator configured to compare the photo-sensing voltage with a reference voltage and output a pixel state signal indicating the pixel state; and
A switch unit configured to control the pixel flag unit according to the pixel status signal,
The pixel flag section:
Transmitting a token signal corresponding to the pixel signal to a memory cell corresponding to each pixel according to the pixel state and the token setting state, and configured to change the token setting state as the token signal is transmitted to the memory cell,
It is connected to the switch unit and includes a token capacitor in which a token charge related to the token setting state is charged to the charge charging terminal,
The switch unit includes a token transfer transistor connected between the token capacitor and a column line connected to the memory cell,
The pixel state signal is input to the gate terminal of the token transfer transistor,
a sense amplifier installed in the column line and configured to amplify the token signal and output it to the memory cell; and
An image sensor further comprising a token removal switch unit provided between the column line and ground and configured to be turned on and off in a complementary manner to the switch unit to remove a residual token signal of the column line. a pixel array including a plurality of pixels that output pixel signals according to pixel states that change depending on the intensity of incident light; and
A frame memory including a plurality of memory cells corresponding to the plurality of pixels and configured to store transmission time data of the pixel signal,
The pixel array is,
a pixel flag unit provided to correspond to each of the plurality of pixels and configured to set a token setting state of each pixel; and
A pixel circuit configured to control the pixel flag unit according to the pixel state,
The pixel flag section:
Transmitting a token signal corresponding to the pixel signal to a memory cell corresponding to each pixel according to the pixel state and the token setting state, and configured to change the token setting state as the token signal is transmitted to the memory cell,
a token storage unit that stores the token setting state;
an AND logic gate configured to receive the token setting state, the pixel state, and the pixel selection signal and perform an AND operation to output a token transmission signal; and
An image sensor including; a switch unit provided between the token storage unit and the memory cell and configured to transmit the token signal from the token storage unit to the memory cell according to the token transmission signal. A method of acquiring an image by the image sensor of any one of claims 1, 2, and 7 to 9, comprising:
setting a token setting state of each pixel of the image sensor by the pixel flag unit;
detecting, by a pixel circuit, a pixel state of each pixel of the image sensor that changes depending on the intensity of incident light, and controlling the pixel flag unit according to the pixel state;
transmitting, by the pixel flag unit, a token signal corresponding to a pixel signal to a memory cell corresponding to each pixel according to the pixel state and the token setting state; and
An image acquisition method comprising: resetting the token setting state of each pixel by transmitting the token signal to the memory cell by the pixel flag unit. According to clause 10,
After the token setting state is reset during a set frame period, blocking retransmission of the token signal to the memory cell until the end of the frame period. According to clause 10,
The steps for controlling the pixel flag unit are:
outputting a photo-sensing voltage according to the intensity of incident light by a photodiode;
Comparing the photo-sensing voltage with a reference voltage using a comparator to output a pixel state signal indicating the pixel state; and
An image acquisition method comprising: controlling the pixel flag unit according to the pixel state signal by a switch unit. According to clause 10,
The steps for setting the token setting state are:
An image acquisition method comprising: charging a token charge to the charge charging terminal of a token capacitor connected to the switch unit. According to clause 13,
Amplifying the token signal transmitted through a column line by a sense amplifier and outputting the token signal to the memory cell; and
An image acquisition method further comprising: removing the residual token signal of the column line by a token removal switch unit that is turned on and off complementary to the switch unit. According to clause 10,
outputting a token transmission signal by performing an AND operation on the token setting state, the pixel state, and the pixel selection signal stored in the token storage portion of the pixel flag unit by an AND logic gate; and
Transmitting, by a switch unit, the token signal from the token storage unit to the memory cell according to the token transmission signal.

Images