CN115988350A - A CMOS Image Sensing, Memory and Computing Integrated Circuit Integrated with Sampling and Computing - Google Patents

Abstract

The invention discloses a CMOS image sensing, storage and calculation integrated circuit integrating sampling and calculation, which belongs to the technical field of integrated circuits, and includes a photoelectric storage and calculation module, an operation module, and a sampling module; the photoelectric storage and calculation module converts light intensity signals into Converting it into an electrical signal and multiplying it with the stored weight value, and outputting it as a current; the operation module, coupled to the output terminal of the photoelectric storage and calculation module, can realize voltage output, and is used for summing the input; The sampling module is coupled to the output terminal of the operation module, and is used for performing fixed-period sampling and subtraction operations on the voltage waveform to obtain weighted and summed light intensity information. The invention provides a weight calculation circuit with high-precision optical sensing and automatic sampling processing, which can complete sensing, storage and calculation functions on a single chip. The circuit structure can be applied to fields such as image recognition and image processing, and has the characteristics of small size, high speed and high integration.

Description

A CMOS Image Sensing, Memory and Computing Integrated Circuit Integrated with Sampling and Computing

technical field

The invention belongs to the technical field of integrated circuits, and in particular relates to a CMOS image sensing, storage and calculation integrated circuit integrating sampling and calculation.

Background technique

In recent years, with the rapid development of artificial neural network technology, image recognition based on neural network has gradually become a mature technology. Image recognition has a very wide range of applications today, such as face recognition, satellite remote sensing, and even military reconnaissance. The essence of the neural network is a series of structured trainable parameters, and the final result is output by performing operations such as weighted summation on the input tensor, thereby realizing the processing of the input tensor, or extracting data features for classification, recognition etc. In a broad sense, image recognition by computer can be divided into three steps: the sensor converts the light intensity signal into an electrical signal to realize image acquisition; the computer software stores and processes the collected image information; the artificial neural network converts the processed Forward calculation of the data, and output some feature values of the image information to realize the final recognition of the image.

Since the use of general-purpose computers for neural network calculations requires a large amount of operating resources, and the mismatch between CPU and memory speeds in computers with traditional architectures also greatly limits the speed of operation, one solution is to build a dedicated system-on-chip. Efficiently complete certain types of neural network computations. The sensor-memory-computing integrated circuit refers to a type of circuit that closely combines image acquisition, weight storage and neural network calculation in structure.

At present, there is no circuit that can use mature technology to combine the integrated circuit of sense-memory-computing and post-data sampling and sorting. The reading of sense-memory-calculation results often relies on complex off-chip readout circuits.

Contents of the invention

In order to solve the problems in the prior art, the embodiment of the present invention provides a CMOS sensor-storage-computing integrated circuit structure integrating sampling and computing, which can automatically realize sensing, storage, computing and data sampling and sorting on the chip.

Technical scheme of the present invention is:

A CMOS image sensing, storage and calculation integrated circuit integrating sampling and calculation, including a photoelectric storage and calculation module, an operation module and a sampling module; the photoelectric storage and calculation module is a matrix composed of n*n photoelectric storage units, each The photoelectric storage unit is connected to two column lines and two row lines, wherein the two column lines provide enable signals and reset signals for the storage weights of each column of photoelectric storage units, and the two row lines are used for each row of photoelectric storage units. The sensing unit writes 2 weights; the arithmetic module includes n arithmetic units, and the input of each arithmetic unit is the positive current collection point and the negative current collection point output by all photoelectric storage and calculation modules of each column; The sampling module includes n sampling units, and the input of each column of sampling units is the output of each column of computing units;

The photoelectric storage unit includes a first MOS transistor, a second MOS transistor, a third MOS transistor, a fourth MOS transistor, a photodiode, a first SRAM, and a second SRAM; wherein, the gate of the first MOS transistor is connected to provide reset The column line of the signal, its drain is connected to the power supply, its source is connected to the cathode of the photodiode and the grid of the second MOS tube; the anode of the photodiode is grounded; the drain of the second MOS tube is connected to the power supply, and its source is connected to the third The drain of the MOS transistor and the drain of the fourth MOS transistor; the gate of the third MOS transistor is connected to the output of the first SRAM, and the source of the third MOS transistor outputs a forward current; the gate of the fourth MOS transistor is connected to the second The output of the SRAM, the source of the fourth MOS transistor outputs a negative current; the first SRAM and the second SRAM are respectively written into the weight by two row lines under the control of the enable signal;

The operation unit is used to convert the input positive current and negative current into voltage, and finally perform the subtraction operation;

The sampling unit is used to sample the input voltage and amplify its driving capability when the input terminal is turned on, and to keep the charge on the capacitor plate when the input terminal is turned off, and output a constant voltage.

Further, the operation unit includes a first operational amplifier, a second operational amplifier, a third operational amplifier, a first resistor, a second resistor, a third resistor, a fourth resistor, a fifth resistor, and a sixth resistor, wherein the first The negative input terminal of an operational amplifier is connected to the positive current sink point Ips, its positive input terminal is grounded, and its output terminal is connected to its negative input terminal through the first resistor; the negative input terminal of the second operational amplifier is connected to the reverse current Converging point Ins, its positive input terminal is grounded, its output terminal is connected to its negative input terminal through the second resistor; the positive input terminal of the third operational amplifier is connected to the output terminal of the first operational amplifier after the third resistor, and the third operational amplifier The negative input terminal of the operational amplifier is connected to the output terminal of the second operational amplifier after passing through the fourth resistor, the positive input terminal of the third operational amplifier is grounded after passing through the fifth resistor, and the output terminal of the third operational amplifier is connected after passing through the sixth resistor. connected to its negative input; the output of the third operational amplifier is the output of the operational unit.

Further, the sampling unit includes a first switch, a second switch, a third switch, a fourth operational amplifier, a fifth operational amplifier, a sixth operational amplifier, a seventh operational amplifier, a first capacitor, a second capacitor, a third capacitor, the seventh resistor, the eighth resistor, the ninth resistor, and the tenth resistor, wherein one end of the first switch is connected to the output end of the arithmetic unit, and the other end is connected to one end of the first capacitor and the positive input end of the fourth operational amplifier , the other end of the first capacitor is grounded; the negative input end of the fourth operational amplifier is connected to its output end; one end of the second switch is connected to the output end of the fourth operational amplifier, the other end of the second switch is connected to one end of the second capacitor and The positive input terminal of the fifth operational amplifier, the other end of the second capacitor is grounded; the negative input terminal of the fifth operational amplifier is connected to its output terminal; one terminal of the third switch is connected to the output terminal of the computing unit, and the other terminal is connected to the sixth operational amplifier The positive input terminal of the amplifier and one end of the third capacitor, the other end of the third capacitor is grounded; the negative input terminal of the sixth operational amplifier is connected to its output terminal; the positive input terminal of the seventh operational amplifier is connected after the seventh resistor The output terminal of the fifth operational amplifier, the negative input terminal of the seventh operational amplifier is connected to the output terminal of the sixth operational amplifier after passing through the eighth resistor, the positive input terminal of the seventh operational amplifier is grounded after passing through the ninth resistor, and the seventh The negative input terminal of the operational amplifier is also connected to its output terminal through the tenth resistor; the output terminal of the seventh operational amplifier is the output terminal of the sampling unit.

The beneficial effects of the present invention are: the present invention provides a weight calculation circuit with high-precision optical sensing and automatic sampling processing, which can complete sensing, storage and calculation functions on a single chip. The circuit structure can be applied to fields such as image recognition and image processing, and has the characteristics of small size, high speed and high integration.

Description of drawings

Fig. 1 is a schematic diagram of the logical structure of the present invention.

FIG. 2 is a schematic diagram of the relationship between the light intensity of the photodiode and the magnitude of the reverse current.

Fig. 3 is a schematic diagram of the structure of a CMOS sensing memory computing unit.

Fig. 4 is a schematic diagram of node waveforms in each working mode of the CMOS sensing memory computing unit.

FIG. 5 is a schematic diagram of the waveforms of the output current of the CMOS sensing, memory and computing unit under different light intensities.

FIG. 6 is a schematic diagram of a CMOS image sensing, storage and calculation integrated circuit integrating sampling and calculation.

FIG. 7 is a schematic structural diagram of the computing module in FIG. 5 .

FIG. 8 is a schematic structural diagram of the sampling module in FIG. 5 .

Fig. 9 is a schematic diagram of the voltage waveform of the clock signal in the sampling module.

Fig. 10 is a schematic diagram of the input and output voltage waveforms of the sampling module.

Detailed ways

Below in conjunction with accompanying drawing, describe technical scheme of the present invention in detail:

As shown in Figure 1, it is a schematic diagram of the overall structure of the present invention, including a photoelectric storage and calculation module, an operation module and a sampling module. The photoelectric storage and calculation module is a matrix composed of n*n photoelectric storage units. The storage unit is connected to two column lines and two row lines, wherein the two column lines provide enable signals and reset signals for the storage weights of each column of photoelectric storage units, and the two row lines are used for each row of photoelectric storage units. The unit writes 2-bit weights; the arithmetic module includes n arithmetic units, and the input of each arithmetic unit is the signal of two column lines; the sampling module includes n sampling units, and the input of each column of sampling units is the signal of each column of arithmetic units output.

As shown in Figure 2, it is a schematic diagram of the relationship between the light intensity of the photodiode and the size of the reverse current. When the photodiode is in reverse bias, the incident photons enter the PN junction and excite the electron-hole pairs, and the generated carriers drift under the influence of the reverse voltage to form a photocurrent, and the magnitude of the reverse current is affected by the incident light Intensity control. In the case of a certain reverse bias voltage, the increase of the intensity of the incident light leads to the rise of the diode current, and the two have a linear relationship within a certain range of the intensity of the incident light. Accordingly, the photodiode can be used as a sensing element that linearly converts the light intensity signal into a current signal.

As shown in FIG. 3 , it is a schematic diagram of the structure of a CMOS sensing memory computing unit. When the reset signal Vrst is at a high level, the reset MOS transistor 1 is turned on and charges the parasitic capacitance in the photodiode 7 until it is close to the power supply voltage Vdd. After the charging is completed, the reset tube 1 is turned off by setting Vrst low. At this time, the sensor-memory-computing unit is exposed to a certain light intensity, so that the current flowing through the photodiode 7 is a photocurrent, which is proportional to the incident light intensity. And discharge the internal parasitic capacitance of the photodiode, causing the voltage across the diode to drop. Due to the function of the follower tube 2, the voltage change at its source terminal follows the voltage at both ends of the diode, represented by the signal V1(t). 5 and 6 in the figure are SRAM cells, in which a digital value is written and stored in advance through the external word line and bit line. When the stored bit is 10, it means the weight value is 1, and 01 means the weight value is -1, 00. It means that the weight is 0, and at the same time, the output voltage is continuously output in the form of high/low level. The values B5 and B6 stored in units 5 and 6 respectively control the on-off of computing tubes 3 and 4. Taking SRAM unit 5 and computing tube 3 as an example, if the value B5 stored in 5 is 0, then unit 5 outputs a low level , so that 3 is turned off, and no current flows; if B5 is 1, that is, the output power supply voltage, then 3 is in the deep linear region, which can be equivalent to a resistance Ron, because the sources of 3 and 4 are clamped to the ground by other modules Flat, the current flowing through 3 is expressed as V1(t)/Ron. Based on the analysis of the two situations, the current Ip1(t) flowing through 3 is the product of the analog quantity v1(t)/Ron and the digital quantity B5, similarly, the current In1(t) flowing through 4 is the analog quantity V1(t) The product of /Ron and digital quantity B6. Therefore, the difference between the currents of the two tubes I1(t)=Ip1(t)-In1(t)=(B5-B6) V1(t)/Ron, which is equivalent to taking V1(t)/Ron as the input data, (B5- B6) is the multiplication operation of the weight. The realization of the subtraction here will be completed by the subsequent operation module.

As shown in FIG. 4 , it is a schematic diagram of the waveforms of each node of the CMOS sensing memory computing unit in the weight storage and sensing memory computing work. In the weight storage operation, the target weight on BL is input into SRAM by setting WL high, and then WL is set low so that the weight is locked in SRAM without being affected by BL. The output end continues to output, and the grid of the calculation tube is controlled to realize the multiplication function. In the sense-memory operation, the conduction current of the calculation tube controlled by the SRAM with a weight value of 1 changes with a certain duty cycle with Vrst, and the conduction current of the calculation tube controlled by an SRAM with a weight value of 0 is always 0. The figure shows the change of Ips and Ins when B5=1 and B6=0, where Ips follows the change of the voltage across the photodiode, and Ins=0.

As shown in FIG. 5 , it is a schematic diagram of the waveform of the output current of the CMOS sensing memory and computing unit under different light intensities. Here B5-B6=1, so the current is positive, and the working cycle of the unit is t0, which is divided into a charging reset phase with a length of t1 and a discharging working phase of (t0-t1). According to the relationship in Figure 1, the average speed of discharge can be used to represent the light intensity in this period, that is, I(t1+n·t0)-I(t0+n·t0)/t0=Cpd·Iin(n), where Cpd is a constant determined by the photodiode process, Iin(n) is the light intensity detected in the nth cycle, and this step of calculation will be realized by the subsequent sampling module. Figure 4 shows the output current curve when the light intensity gradually increases with time. It can be seen that the average slope of the curve is larger than that of the previous cycle in the discharge phase of each cycle. The change of the weight value B5-B6 will make the output current curve scale proportionally along the vertical axis. If B5-B6=0, the curve will always be equal to 0; if B5-B6=-1, the current at each point on the curve will be Taking a negative value will result in the opposite result after being calculated by the subsequent sampling module. The output terminal connection of a plurality of CMOS sensing storage and calculation units can further expand the range of the output current, which is used to represent the summation result of the multiplication of a plurality of digital weights and the analog current converted from the light intensity signal.

As shown in FIG. 6 , it is a schematic diagram of a CMOS image sensing, storage and calculation integrated circuit integrating sampling and calculation. In an n*n-scale circuit, all sense-memory-calculation units in the same column are controlled by the same reset signal, and all Ip and In are summed by the arithmetic module, and then the two are subtracted to complete the weighted summation operation, and finally calculate the weighted and summed light intensity signal through the sampling module. In addition to the power line, there are n column lines WL in the circuit of n*n scale to control the enablement of the storage weights of each column, and 2n row lines BL are used to write 2-bit weights for all sense memory calculation units, and The n column lines Vrst provide reset signals. The positive summing current Ips of n column lines and the negative summing current Ins of n lines are followed by n computing modules and n sampling modules to output n mutually independent weighted and summed light intensity signals. When the frequency of light intensity changes is less than the operating frequency of the sensor-memory-computation unit, the CMOS image sensor-memory-computation integrated circuit can accurately receive the light intensity signal and perform a column-wise weighted summation operation.

As shown in FIG. 7 , the operational circuit module is composed of operational amplifiers 8 , 9 , 10 , load resistors 11 , 12 , input resistors 13 , 14 , 16 , and feedback resistor 15 . Its function is to clamp the column line, convert the column line current into a voltage, and finally perform the subtraction operation. Wherein R11=R12, R13=R14=R15=R16. All operational amplifiers have a very high gain to achieve "virtual short" and "virtual break", so that the current input terminals Ips and Ins are clamped to GND, so that the operational tube in the pre-sensing storage unit can work normally. In the operational circuit module, all amplifiers are powered by dual power supplies, so that the column line clamping function can be normally realized and negative voltage can be output. Operational amplifier 8 and load resistor 11 convert Ips to Vps=Ips R11, similarly operational amplifier 9 and load resistor 12 convert Ins to Vns=Ins R12, composed of operational amplifier 10 and resistors R13, R14, R15, R16 The voltage subtractor outputs Vo=Vps-Vns to complete the operation of weighted summation.

As shown in Figure 8, the sampling module is composed of operational amplifiers 20, 21, 22, 23, sampling switches 17, 18, 19, sampling capacitors 24, 25, 26, input resistors 27, 28, 29, and feedback resistors 30. where R27=R28=R29=R30. The circuit formed by the combination of sampling capacitor and operational amplifier samples the input voltage and amplifies its driving ability when the input terminal is turned on, and the charge is kept on the capacitor plate when the input terminal is disconnected, and the output voltage is constant, realizing sample-hold-buffering The function of the sampling capacitor here is small, so that the capacitor can be charged quickly during the conduction of the sampling switch, and the sampling voltage can be accurately recorded. At the beginning of the discharge phase of all SRAM units in the column, a pulse briefly opens the sampling switch 17, so that the sampling capacitor 24 records Vo at this time. At the end of the discharge phase, another pulse makes the sampling switches 18 and 19 open simultaneously, and the voltage on the sampling capacitor 24 is copied to the sampling capacitor 26 through the buffer circuit formed by the operational amplifier 20. Since the duty cycle is constant, the capacitor 26 and The voltage difference between 25 is proportional to the light intensity of the weighted summation, and the final image recognition result Vout of the column is output through the buffer amplifier and the voltage subtractor.

As shown in FIG. 9 , it is a schematic diagram of the voltage waveform of the clock signal in the sampling module. Each analog switch is composed of a pair of NMOS and PMOS, and needs to be controlled by two clock signals. The sampling module has two independent switches, and needs four clock signals in total. In the figure, Vrst is the reset signal of the control sensor, which is used to compare the timing logic of each signal. Vn1 and Vp1 in the figure are the clock signals controlling the switch 17 in Figure 7, and pulses need to be emitted on the falling edge of Vrst to make the sampling switch record the output voltage at the beginning of discharge; Vn2 and Vp2 are the clocks controlling the switches 18 and 19 in Figure 7 signal, it is necessary to emit a pulse before the next rising edge of Vrst to enable the sampling switch to record the output voltage at the end of discharge, and at the same time conduct the initial discharge voltage to the input terminal of the voltage subtractor to calculate the sampling result. These clock signals are all determined by the reset signal Vrst, and the pulse length is very short, so Vo basically does not change during the sampling time of the capacitor.

As shown in FIG. 10 , it is a schematic diagram of voltage waveforms of input Vo and output Vout of the sampling module. Due to the function of the transmission gates 18 and 19, the output Vout of the sampling module only changes the voltage value at the end of the working cycle of the sensor-memory-calculation array, and outputs the sensor-memory-calculation result of the light intensity in this period. This working mode shows in the waveform that in each cycle, Vout always outputs the result of the sense-memory-calculation of the last cycle, that is, the readout time window is equal to the duty cycle of the sense-memory-calculation. Figure 10 illustrates two complete duty cycles and a portion of a third cycle in which the weighted light intensity in the second duty cycle is higher than in the first cycle, resulting in an output voltage of Vout in the third cycle that is higher than output voltage in the second cycle.

Claims (3)

1. A CMOS image sense storage and calculation integrated circuit integrating sampling and calculation is characterized in that it includes a photoelectric storage and calculation module, an operation module and a sampling module; the photoelectric storage and calculation module is composed of n*n photoelectric storage units Each photoelectric storage unit is connected to two column lines and two row lines, and the two column lines respectively provide the enable signal and reset signal for the storage weight of each column photoelectric storage unit, and the two row lines It is used to write a 2-bit weight value for each row of photoelectric storage units; the operation module includes n operation units, and the input of each operation unit is the positive current sink point and negative current output of all photoelectric storage and calculation modules in each column. To the current collection point; the sampling module includes n sampling units, and the input of each column of sampling units is the output of each column of computing units; The photoelectric storage unit includes a first MOS transistor, a second MOS transistor, a third MOS transistor, a fourth MOS transistor, a photodiode, a first SRAM, and a second SRAM; wherein, the gate of the first MOS transistor is connected to provide reset The column line of the signal, its drain is connected to the power supply, its source is connected to the cathode of the photodiode and the grid of the second MOS tube; the anode of the photodiode is grounded; the drain of the second MOS tube is connected to the power supply, and its source is connected to the third The drain of the MOS transistor and the drain of the fourth MOS transistor; the gate of the third MOS transistor is connected to the output of the first SRAM, and the source of the third MOS transistor outputs a forward current; the gate of the fourth MOS transistor is connected to the second The output of the SRAM, the source of the fourth MOS transistor outputs a negative current; the first SRAM and the second SRAM are respectively written into the weight by two row lines under the control of the enable signal; The operation unit is used to convert the input positive current and negative current into voltage, and finally perform the subtraction operation; The sampling unit is used to sample the input voltage and amplify its driving capability when the input terminal is turned on, and to keep the charge on the capacitor plate when the input terminal is turned off, and output a constant voltage. 2. A kind of integrated sampling and computing integrated circuit of CMOS image sensing, storage and computing according to claim 1, characterized in that, said computing unit comprises a first operational amplifier, a second operational amplifier, a third operational amplifier, a first resistor , the second resistor, the third resistor, the fourth resistor, the fifth resistor, and the sixth resistor, wherein the negative input terminal of the first operational amplifier is connected to the positive current sink point Ips, its positive input terminal is grounded, and its output terminal Connect its negative input terminal through the first resistor; the negative input terminal of the second operational amplifier is connected to the reverse current sink point Ins, its positive input terminal is grounded, and its output terminal is connected to its negative input terminal through the second resistor; The positive input terminal of the three operational amplifiers is connected to the output terminal of the first operational amplifier after the third resistor, and the negative input terminal of the third operational amplifier is connected to the output terminal of the second operational amplifier after the fourth resistor, and the output terminal of the third operational amplifier The positive input terminal is grounded after passing through the fifth resistor, and the output terminal of the third operational amplifier is connected with the negative input terminal after passing through the sixth resistor; the output terminal of the third operational amplifier is the output terminal of the operation unit. 3. A kind of integrated sampling and computing integrated circuit of CMOS image sensing, storage and computing according to claim 2, characterized in that the sampling unit comprises a first switch, a second switch, a third switch, a fourth operational amplifier, a first Five operational amplifiers, sixth operational amplifiers, seventh operational amplifiers, first capacitors, second capacitors, third capacitors, seventh resistors, eighth resistors, ninth resistors, and tenth resistors, wherein one end of the first switch is connected to The output end of the operational unit, the other end is connected to one end of the first capacitor and the positive input end of the fourth operational amplifier, and the other end of the first capacitor is grounded; the negative input end of the fourth operational amplifier is connected to its output end; the second switch One end of the second switch is connected to the output end of the fourth operational amplifier, the other end of the second switch is connected to one end of the second capacitor and the positive input end of the fifth operational amplifier, and the other end of the second capacitor is grounded; the negative input of the fifth operational amplifier The end is connected to its output end; one end of the third switch is connected to the output end of the computing unit, the other end is connected to the positive input end of the sixth operational amplifier and one end of the third capacitor, and the other end of the third capacitor is grounded; the sixth operational amplifier The negative input terminal is connected to its output terminal; the positive input terminal of the seventh operational amplifier is connected to the output terminal of the fifth operational amplifier through the seventh resistor, and the negative input terminal of the seventh operational amplifier is connected to the sixth operational amplifier through the eighth resistor The output terminal of the amplifier, the positive input terminal of the seventh operational amplifier is also grounded after the ninth resistor, and the negative input terminal of the seventh operational amplifier is also connected to its output terminal after the tenth resistor; the output terminal of the seventh operational amplifier is Sampling unit output.

Images