KR20210107192A - Image Mask Processing Circuit Using Switch-Capacitor Integrator and Method Therof - Google Patents

Abstract

Disclosed are an image mask processing circuit using a switch-capacitor integrator which performs image mask processing on hardware using capacitor characteristics and a method thereof. Pixel values are not converted into digital codes within an image sensor, and a voltage can be divided by a desired value using the switch-capacitor structure in the analog preprocessing process, so that convolution operation is possible on hardware. Accordingly, power consumption can be reduced and processing speed can be improved. In addition, since only desired data can be extracted from an image using the capacitor characteristics, mask processing can be easily implemented on hardware.

Description

Image Mask Processing Circuit Using Switch-Capacitor Integrator and Method Therof

The present invention relates to an image mask processing circuit and method using a switch-capacitor integrator, and more particularly, to an image mask processing circuit and method using a switch-capacitor integrator for performing image mask processing on hardware using capacitor characteristics. will be.

Convolutional Neural Networks (CNNs) are a type of deep learning that are mainly used to recognize images. In CNN, a convolution operation, that is, mask processing, is required for images or data, which is generally processed through software.

1 is a diagram illustrating a conventional image mask process.

Referring to Figure 1, conventional image mask processing is generally processed through software. That is, the process of putting a mask on the input image, which is the mask processing process, and then making the sum of values obtained by multiplying each pixel and the mask into an output image is processed through software. Accordingly, since the software processing process is complicated and the power consumption is large, the signal processing speed is lowered and the area is increased.

Korean Patent Laid-Open Patent 10-2008-0069887

An object of the present invention is to provide an image mask processing circuit and method using a switch-capacitor integrator capable of performing a convolution operation through voltage distribution within an image sensor itself without using an Image Signal Processor (ISP) is doing

The switch-capacitor integrator of the present invention for solving the above problems includes a charging unit connected between an input terminal and a first node and storing a reset signal or a signal signal input through the input terminal, the first It is connected between a first node and an output terminal, and an offset unit configured to function as a virtual ground of the first node, and is connected between a second node connected to the first node and the output terminal, and is inputted through the input terminal. and a voltage divider for dividing a difference between the reset signal and the signal signal.

The charging unit may include a first switch connected to the input terminal, a first capacitor connected between the first switch and the first node, and a second switch connected between the first node and a reference voltage input terminal.

The offset unit may include a second capacitor connected to the first node, an inverter connected between the second capacitor and a third node, a third switch connected between an input terminal and an output terminal of the inverter, and between the third node and the voltage divider. It may include a fourth switch connected to.

The voltage divider may include a third capacitor connected to the second node, a fifth switch connected between the third capacitor and a fourth node, a fourth capacitor connected to the second node, and a third capacitor connected between the fourth capacitor and a fourth node. It may include a sixth switch connected and a seventh switch connected between the fourth capacitor and the sixth switch.

The third capacitor may store a signal corresponding to a difference between the reset signal and the signal signal by using the first node as a virtual ground.

When the fifth switch and the sixth switch are turned on, the signals stored in the third capacitor may be distributed to each other by the fourth capacitor.

The fourth capacitor may be reset by turning on the seventh switch.

An eighth switch connected between the third node and the output terminal may be further included.

When the eighth switch is turned on, the signal divided by the voltage divider may be output through the output terminal.

A method of operating an image mask processing circuit using a switch-capacitor integrator of the present invention for solving the above problems includes a reset step of storing a reset signal input through an input terminal, the reset signal and a signal signal input through the input terminal and a recording step of storing the difference of ? in a capacitor, a distribution step of distributing the signal stored in the capacitor, and an output step of outputting the divided signal through an output terminal.

The resetting may further include resetting the capacitor through a virtual ground.

The distribution step may be distributed through a distribution capacitor connected to the capacitor.

The distribution capacitor may be reset in the output stage.

The distribution step and the output step may be repeatedly performed until a desired output signal is output.

According to the present invention, a convolution operation is possible in hardware because a voltage can be divided by a desired value using a switch-capacitor structure in an analog preprocessing process without converting a pixel value into a digital code in the image sensor. Accordingly, power consumption can be reduced and processing speed can be improved.

In addition, since only desired data can be extracted from an image using the capacitor characteristics, mask processing can be easily implemented in hardware.

The technical effects of the present invention are not limited to those mentioned above, and other technical effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a diagram illustrating a conventional image mask process.
2 is a view showing an image sensor according to the present invention.
3 is a diagram illustrating an embodiment of a mask for explaining image mask processing according to the present invention.
4 is a circuit diagram showing an image mask processing circuit of the present invention.
5 to 8 are circuit diagrams for explaining an operation method of the image mask processing circuit of the present invention.

Since the present invention can apply various transformations and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. In describing the present invention, if it is determined that a detailed description of a related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. do it with

Example

2 is a view showing an image sensor according to the present invention.

3 is a diagram illustrating an embodiment of a mask for explaining image mask processing according to the present invention.

2 and 3 , the image sensor according to the present invention has a column parallel structure using a row driver. Accordingly, pixel data is output one row at a time, and two data, a reset signal (Vrst) and a signal signal (Vsig), are output from one pixel using a 4-tr pixel. With these reset signal (Vrst) and signal signal (Vsig) data, the difference value (ΔPixel) between the two data can be detected by using the active switch-capacitor structure of the integrator, and through this, an unwanted offset generated in the pixel can be removed.

For example, as shown in FIG. 3 , it may have a 2×2 mask structure. That is, it may have an N-th row and an N+1-th row in the vertical direction and C1 and C2 th pixel structures in the horizontal direction.

In this mask structure, data for a pixel may be output one row at a time, and data for a reset signal Vrst and a signal signal Vsig may be outputted to each pixel, respectively. In addition, such data is converted into a masked value according to the capacitance value of the capacitor in the image mask processing circuit, and a convolution operation is simply performed.

4 is a circuit diagram showing an image mask processing circuit of the present invention.

Referring to FIG. 4 , the image mask processing circuit according to the present invention includes a charging unit 100 , an offset unit 200 , and a voltage dividing unit 300 .

The charging unit 100 is connected between the input terminal IN and the first node N1 and stores the reset signal Vrst or the signal signal Vsig input through the input terminal IN.

The charging unit 100 may include a first switch SW1 , a first capacitor Cs, and a second switch SW2 .

The first switch SW1 may be connected to the input terminal IN, and the first capacitor Cs may be connected between the first switch SW1 and the first node N1 . Accordingly, the first capacitor Cs may store the reset signal Vrst or the signal signal Vsig input through the input terminal IN by the switching operation of the first switch SW1 .

Also, the second switch SW2 may be connected between the first node N1 and the reference voltage input terminal Vref. That is, the reference voltage signal Vref may be applied to the first capacitor Cs through the first node N1 by the switching operation of the second switch SW2 .

Accordingly, the first capacitor Cs of the charging unit 100 has an input signal input from the input terminal IN by the switching operation of the first switch SW1 and the second switch SW2 and the reference voltage input terminal Vref. ) may be stored in the reference voltage signal Vref.

For example, the first node N1 may function as a virtual ground by an offset unit 200 to be described later. Accordingly, when the first switch SW1 and the second switch SW2 are turned on, the reference voltage signal Vref input from the reference voltage input terminal Vref and the input terminal IN are applied to the first capacitor Cs. A signal corresponding to the difference between the inputted input signals may be stored.

The offset unit 200 is connected between the first node N1 and the output terminal OUT, and serves as a virtual ground for the first node N1.

That is, the offset unit 200 may include a second capacitor Caz, an inverter INV, a third switch SW3, and a fourth switch SW4.

The second capacitor Caz may be connected to the first node N1 . In addition, an inverter INV may be used as an integrator to form a feedback loop. Here, the inverter INV may be connected between the second capacitor Caz and the third node N3 . The third switch SW3 may be connected between the input terminal and the output terminal of the inverter INV, and the fourth switch SW4 may be connected between the third node N3 and the fourth node N4 .

Charge corresponding to the offset voltage may be stored in the second capacitor Caz. Accordingly, the second capacitor Caz of the offset unit 200 may function as an offset capacitor for removing the offset voltage generated in the inverter INV.

The switch-capacitor integrator according to the present invention by the above-described charging unit 100 and offset unit 200 operates as an integrator after a sampling operation in response to an input signal.

During the sampling operation, the input terminal IN and the reference voltage input terminal Vref are connected to the first node N1 , and the first capacitor Cs is sampled by the input signal and the reference voltage signal Vref. In this case, the first node N1 functions as a virtual ground by the operation of the offset unit 200 . That is, during the integration operation, the first node N1 is connected to the voltage divider 300 and forms a feedback loop by the inverter INV. Accordingly, the signal charged in the first node N1 during the sampling operation is transferred to the voltage divider 300 during the integration operation. Accordingly, the voltage divider 300 stores the signal stored in the first capacitor Cs, and the integrator performs an integration operation.

The voltage divider 300 is connected between the second node N2 connected to the first node N1 and the output terminal OUT, and distributes an input signal input through the input terminal IN. For example, the voltage divider 300 may store and distribute the signal Vrst-Vsig corresponding to the difference between the reset signal Vrst input through the input terminal IN and the signal signal Vsig.

The voltage divider 300 may include a third capacitor Ch, a fifth switch SW5, a fourth capacitor Cd, a sixth switch SW6, and a seventh switch SW7.

The third capacitor Ch may be connected to the second node N2 , and the fifth switch SW5 may be connected between the third capacitor Ch and the fourth node N4 . Also, the fourth capacitor Cd may be connected to the second node N2 , and the sixth switch SW6 may be connected between the fourth capacitor Cd and the fourth node N4 . Here, the signal stored in the first capacitor Cs through the first node N1 during the integration operation may be stored in the third capacitor Ch. For example, a signal Vrst-Vsig corresponding to a difference between the reset signal Vrst input through the input terminal IN and the signal signal Vsig may be stored in the third capacitor Ch.

로 나누어준 신호(V_h1)가 저장될 수 있다. 이는 제3 커패시터(Ch)의 종래 저장된 값(V_h0)을

로 곱해준 것과 동일한 결과를 갖는다. 이러한 분배 과정은 출력 단자(OUT)에 원하는 출력 신호가 출력될 때까지 반복적으로 수행될 수 있으며, 반복적으로 수행될 때마다 제3 커패시터(Ch)에

를 곱해주게 된다. 즉, 본 발명의 전압 분배부(300)에 따라 제3 커패시터(Ch)가 제4 커패시터(Cd)에 의해 분배되는 과정은 마스크를 이용하여 입력 이미지의 각 픽셀에 마스크 값을 곱하는 과정과 동일한 결과 값을 얻을 수 있다.In addition, the signal stored in the third capacitor Ch may be distributed through the fourth capacitor Cd connected to the second node N2 and the fourth node N4 . That is, when the fifth switch SW5 and the sixth switch SW6 are turned on, the signal stored in the third capacitor Ch may be distributed to each other by the fourth capacitor Cd serving as the distribution capacitor. Here, when the signal stored in the third capacitor Ch is distributed by the fourth capacitor Cd after the integration operation, the value V _{h0 conventionally stored by the fourth capacitor Cd is applied to the third capacitor Ch.}

A signal (V _h1 ) divided by can be stored. This is the conventionally stored value (V _h0 ) of the third capacitor (Ch)

It has the same result as multiplying by . This distribution process may be repeatedly performed until a desired output signal is output to the output terminal OUT, and each time it is repeatedly performed, it is applied to the third capacitor Ch.

will be multiplied by That is, the process in which the third capacitor Ch is divided by the fourth capacitor Cd according to the voltage divider 300 of the present invention is the same as the process of multiplying each pixel of the input image by a mask value using a mask. value can be obtained.

As described above, the image mask processing circuit using the switch-capacitor integrator according to the present invention does not convert pixel values into digital codes in the image sensor, but uses the switch-capacitor structure in the image sensor itself in the analog preprocessing process. Convolution operation is possible through voltage division. Accordingly, power consumption can be reduced and processing speed can be improved.

Continuingly, referring to FIG. 4 , the seventh switch SW7 of the voltage divider 300 is connected between the fourth capacitor Cd and the sixth switch SW6 , and is connected to the reference voltage input terminal Vref. can The third capacitor Ch or the fourth capacitor Cd may be reset by a switching operation of the seventh switch SW7 . For example, if the operation of dividing the third capacitor Ch by the fourth capacitor Cd is repeated, the seventh switch SW7 is turned on so that the third capacitor Ch is multiplied by the same value each time it is repeated. The fourth capacitor Cd is reset.

When a desired output signal is obtained after the division process is repeated by the voltage divider 300 , the final output signal V _ref +V _h is generated by the switching operation of the fourth switch SW4 and the eighth switch SW8 . is output

operation mode

5 to 8 are circuit diagrams for explaining an operation method of the image mask processing circuit of the present invention.

An image mask processing method of the present invention will be described in detail below with reference to FIGS. 5 to 8 .

The operation mode of the image mask processing circuit using the switch-capacitor integrator of the present invention is divided into a total of four modes: a reset mode, a write mode, a divide mode, and an output mode. According to the mode of the switch and the type of the applied signal is defined.

First, referring to FIG. 5 , in the reset mode, the reset signal Vrst input through the input terminal IN is stored. In the reset mode operation, the reset signal Vrst is applied from the pixel through the input terminal IN. In addition, the first switch SW1 and the second switch SW2 of the charging unit 100 , the third switch SW3 of the offset unit 200 , the fifth switch SW5 of the voltage dividing unit 300 , and the sixth The switch SW6 and the seventh switch SW7 are turned on, respectively.

That is, the Vref-Vrst signal is stored in the first capacitor Cs when the first switch SW1 and the second switch SW2 are turned on. Here, the offset unit 200 performs a feedback operation by turning on the third switch SW3 , and a signal corresponding to the offset signal of the inverter INV is stored in the second capacitor Caz. In this case, the first node N1 functions as a virtual ground. The amount of internal charge of the third capacitor Ch and the fourth capacitor Cd is reset to zero by turning on the fifth switch SW5 , the sixth switch SW6 , and the seventh switch SW7 .

Referring to FIG. 6 , a signal corresponding to a difference between a reset signal Vrst and a signal signal Vsig is stored in the write mode. In the write mode operation, the signal signal Vsig is applied from the pixel through the input terminal IN. In addition, the first switch SW1 of the charging unit 100 , the fourth switch SW4 of the offset unit 200 , and the fifth switch SW5 of the voltage dividing unit 300 are turned on.

신호가 저장될 수 있다. 여기서, Vrst-Vsig 값은 픽셀에서 입력되는 리셋 신호(Vrst)와 시그널 신호(Vsig)의 차이값(ΔPixel)을 나타내기 때문에 제3 커패시터(Ch)에 저장되는 신호는 수학식 1과 같이 나타낼 수 있다.That is, since the signal signal Vsig is applied through the input terminal IN, the Vref-Vsig signal is stored in the first capacitor Cs. In addition, as the sixth switch SW6 and the seventh switch SW7 are turned off, the difference between the signal Vref-Vrst stored in the reset mode in the first capacitor Cs and the signal Vref-Vsig newly stored A signal of is stored in the third capacitor Ch through the second node N2 with the first node N1 as a virtual ground. The third capacitor Ch has a value corresponding to the difference between the reset signal Vrst and the signal signal Vsig.

A signal may be stored. Here, since the Vrst-Vsig value represents the difference value ΔPixel between the reset signal Vrst inputted from the pixel and the signal signal Vsig, the signal stored in the third capacitor Ch can be expressed as Equation 1 have.

A correlated double sampling (CDS) image signal may be obtained from a pixel through the integrator operations in the reset mode and the write mode described above, and the image signal may be stored through a capacitor.

Referring to FIG. 7 , in the distribution mode, the stored signal is distributed through the reset mode and the write mode. The operation in the distribution mode includes the second switch SW2 of the charging unit 100 , the third switch SW3 of the offset unit 200 , and the fifth switch SW5 and the sixth switch SW6 of the voltage dividing unit 300 . ) is turned on.

As the fifth switch SW5 and the sixth switch SW6 of the voltage divider 300 are turned on and the seventh switch SW7 is turned off, the signal V _{h0 stored in the third capacitor Ch in the write mode} ) may be distributed by the fourth capacitor Cd. A signal distributed and stored in the third capacitor Ch by distribution using the fourth capacitor Cd may be expressed as Equation (2).

로 나누어준 신호가 저장된다. 이는 제3 커패시터(Ch)의 종래 저장된 값을

로 곱해준 것과 동일한 결과를 갖는다. 따라서, 제4 커패시터(Cd)는 제3 커패시터(Ch)에 저장된 신호를 분배하는 분배 커패시터로서 기능한다.That is, the third capacitor Ch is set to a value V _h0 conventionally stored by the fourth capacitor Cd.

The signal divided by This is the conventionally stored value of the third capacitor Ch.

It has the same result as multiplying by . Accordingly, the fourth capacitor Cd functions as a distribution capacitor that distributes the signal stored in the third capacitor Ch.

Referring to FIG. 8 , in the output mode, a signal V _h distributed through the distribution mode may be output. In the output mode operation, the fourth switch SW4 of the offset unit 200 , the fifth switch SW5 and the seventh switch SW7 of the voltage divider 300 , and the eighth switch connected to the output terminal OUT (SW8) is turned on. The signal of the third capacitor Ch distributed and stored by the turn-on of the fourth switch SW4 and the eighth switch SW8 may be output through the output terminal OUT. That is, _{an output signal of V ref} +V _h may be output through the output terminal OUT.

Also, as the sixth switch SW6 of the voltage divider 300 is turned off and the seventh switch SW7 is turned on, the signal stored in the fourth capacitor Cd may be reset.

값을 곱해주게 된다. 즉,

를 α, 분배되는 횟수를 n이라고 가정하면, n번의 분배를 수행했을 때 제3 커패시터(Ch)에 저장되는 값은 수학식 3과 같이 나타낼 수 있다.The distribution mode and the output mode may be repeatedly performed until a desired output signal is output, and whenever the distribution mode is repeatedly performed, the third capacitor Ch is repeatedly applied by the fourth capacitor Cd.

multiplies the value. in other words,

Assuming that is α and the number of times of distribution is n, the value stored in the third capacitor Ch when the distribution is performed n times can be expressed as Equation (3).

According to Equation 3, it is possible to implement a circuit capable of storing the correlated double-sampled value from the pixel in the third capacitor Ch and dividing the value as desired by using the fourth capacitor Cd as the distribution capacitor. Accordingly, the process in which the third capacitor Ch is divided by the fourth capacitor Cd according to the voltage divider 300 of the present invention is the same as the process of multiplying each pixel of the input image by a mask value using a mask. value can be obtained.

As described above, the image mask processing circuit using the switch-capacitor integrator according to the present invention does not convert pixel values into digital codes in the image sensor, and uses the switch-capacitor structure in the analog preprocessing process in the image sensor itself. You can divide the voltage by the desired value. That is, convolution operation is possible on hardware. Accordingly, power consumption can be reduced and processing speed can be improved. In addition, since only desired data can be extracted from the image using the capacitor characteristics, mask processing can be easily implemented in hardware.

On the other hand, the embodiments of the present invention disclosed in the present specification and drawings are merely presented as specific examples to aid understanding, and are not intended to limit the scope of the present invention. It will be apparent to those of ordinary skill in the art to which the present invention pertains that other modifications based on the technical spirit of the present invention can be implemented in addition to the embodiments disclosed herein.

100: charging part 200: offset part
300: voltage divider Cs: first capacitor
Caz: second capacitor Ch: third capacitor
Cd: fourth capacitor INV: inverter
SW1: first switch SW2: second switch
SW3: third switch SW4: fourth switch
SW5: fifth switch SW6: sixth switch
SW7: seventh switch SW8: eighth switch

Claims (14)

a charging unit connected between the input terminal and the first node and configured to store a reset signal or a signal signal input through the input terminal;
an offset unit connected between the first node and an output terminal and configured to function as a virtual ground of the first node; and
A switch-capacitor integrator connected between a second node connected to the first node and the output terminal and comprising a voltage divider for dividing a difference between the reset signal and a signal signal input through the input terminal - image mask processing using a capacitor integrator Circuit. According to claim 1, wherein the charging unit,
a first switch connected to the input terminal;
a first capacitor coupled between the first switch and the first node; and
and a second switch connected between the first node and a reference voltage input terminal. An image mask processing circuit using a capacitor integrator. According to claim 1, wherein the offset portion,
a second capacitor coupled to the first node;
an inverter connected between the second capacitor and a third node;
a third switch connected between an input terminal and an output terminal of the inverter; and
and a fourth switch connected between the third node and the voltage divider. An image mask processing circuit using a capacitor integrator. According to claim 1, wherein the voltage divider,
a third capacitor coupled to the second node;
a fifth switch connected between the third capacitor and a fourth node;
a fourth capacitor coupled to the second node;
a sixth switch connected between the fourth capacitor and a fourth node; and
A switch-capacitor integrator-using image mask processing circuit including a seventh switch connected between the fourth capacitor and the sixth switch. 5. The method of claim 4,
The third capacitor uses the first node as a virtual ground to store a signal corresponding to a difference between the reset signal and the signal signal. 5. The method of claim 4,
When the fifth switch and the sixth switch are turned on, the signal stored in the third capacitor is distributed to each other by the fourth capacitor - an image mask processing circuit using a capacitor integrator. 5. The method of claim 4,
and the fourth capacitor is reset by turning on the seventh switch. An image mask processing circuit using a switch-capacitor integrator. According to claim 1,
The image mask processing circuit using a switch-capacitor integrator further comprising an eighth switch connected between the third node and the output terminal. According to claim 1,
and when the eighth switch is turned on, the signal divided by the voltage divider is output through the output terminal. a reset step of storing a reset signal input through an input terminal;
a writing step of storing a difference between the reset signal and the signal signal input through the input terminal in a capacitor;
a distribution step of distributing the signal stored in the capacitor; and
and an output step of outputting the divided signal through an output terminal. The method of claim 10, wherein the resetting step comprises:
and resetting the capacitor through virtual ground. The method of claim 10, wherein the dispensing step comprises:
A method of operating an image mask processing circuit using a switch-capacitor integrator that is distributed through a distribution capacitor connected to the capacitor. 13. The method of claim 12,
and the distribution capacitor is reset in the output stage. 11. The method of claim 10,
and the distributing step and the outputting step are repeatedly performed until a desired output signal is output.

Images