CN111343398A - CMOS (complementary Metal oxide semiconductor) sensing and calculating integrated circuit structure based on dynamic visual sensing technology - Google Patents

CMOS (complementary Metal oxide semiconductor) sensing and calculating integrated circuit structure based on dynamic visual sensing technology Download PDF

Info

Publication number
CN111343398A
CN111343398A CN202010272251.0A CN202010272251A CN111343398A CN 111343398 A CN111343398 A CN 111343398A CN 202010272251 A CN202010272251 A CN 202010272251A CN 111343398 A CN111343398 A CN 111343398A
Authority
CN
China
Prior art keywords
circuit
sensing
light intensity
data
sampling
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010272251.0A
Other languages
Chinese (zh)
Other versions
CN111343398B (en
Inventor
胡绍刚
周桐
邓杨杰
于奇
刘洋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Electronic Science and Technology of China
Original Assignee
University of Electronic Science and Technology of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Electronic Science and Technology of China filed Critical University of Electronic Science and Technology of China
Priority to CN202010272251.0A priority Critical patent/CN111343398B/en
Publication of CN111343398A publication Critical patent/CN111343398A/en
Application granted granted Critical
Publication of CN111343398B publication Critical patent/CN111343398B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70SSIS architectures; Circuits associated therewith
    • H04N25/76Addressed sensors, e.g. MOS or CMOS sensors
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70SSIS architectures; Circuits associated therewith
    • H04N25/71Charge-coupled device [CCD] sensors; Charge-transfer registers specially adapted for CCD sensors
    • H04N25/75Circuitry for providing, modifying or processing image signals from the pixel array

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)

Abstract

The invention discloses a CMOS (complementary metal oxide semiconductor) sensing and calculating integrated circuit structure based on a dynamic visual sensing technology. The invention mainly comprises a sensing circuit module based on an AER mode and a storage and calculation integrated circuit module, wherein the sensing circuit module based on the AER mode comprises: the dynamic visual sensing active phase element circuit is used for sensing an external light intensity signal and converting the external light intensity signal into a corresponding electric signal; the correlated double sampling circuit is used for carrying out double sampling on the electric signal; the differential comparator is used for making difference on the subsampled data and comparing the subsampled data with a reference voltage; the light intensity detection circuit is used for sensing whether the dynamic light intensity changes or not; the logic decision circuit is used for combining the data of the differential comparator and the light intensity detection circuit to carry out validity decision and outputting a decision result. The purposes of saving storage area, reducing calculation power consumption and improving calculation speed are achieved.

Description

CMOS (complementary Metal oxide semiconductor) sensing and calculating integrated circuit structure based on dynamic visual sensing technology
Technical Field
The invention belongs to the technical field of image sensing technology and integrated circuits, and particularly relates to a CMOS (complementary metal oxide semiconductor) sensing and storage integrated circuit structure based on a dynamic visual sensing technology.
Background
The development of big data technology and the rise of deep learning technology wave taking a neural network as a core are taken as representatives, and higher requirements are put forward on the calculation power of the traditional mainstream hardware platform. Since the deep learning algorithm needs to process streaming data in the calculation, when a hardware platform based on the von neumann computing architecture is used for processing related tasks, a large amount of data can flow between the computing unit and the storage unit. The latter read-write speed is much slower than the former computing speed, the operation process of accessing the memory occupies most of the overall energy consumption and delay, and the processing speed of the data is limited, which is called as von neumann bottleneck or memory bottleneck. The memory bottleneck causes the computing system to have the defects of high power consumption, low speed and the like. In a calculation task centered on a large data volume, the problem of memory separation is more prominent.
Under the background, a calculation and storage integrated framework similar to a cranial nerve structure is gradually developed and is used as a model similar to a human brain, a data storage unit and a calculation unit are fused into a whole, so that the data transportation is reduced, and the calculation parallelism and the energy efficiency are greatly improved. It is certain that the integrated computing and storing chip and its specific application will be quickly dropped to the ground driven by the gradual mature technology and the simultaneous application requirement.
The current research on the storage and computation integrated circuit focuses only on two aspects of storage and computation, and the research on the combination of the storage and computation integrated circuit and other applications is little and less, particularly on a sensing and computation integrated circuit combined with a sensing circuit.
Disclosure of Invention
In order to solve the problems in the prior art, research on the existing CMOS active-source pixel circuit finds that the CMOS active-source pixels exist in the form of a large-scale pixel array, that is, the pixel array, the storage unit and the operation unit are independent circuit modules. The von Neumann structure inevitably causes the requirement of separate data buses, address buses and control buses among different circuit modules, corresponding decoding circuits and control circuits, and the working mode has low operation speed and large generated power consumption. Therefore, the invention adopts the idea of integration of sensing and storage, and can realize the acquisition, storage and linear operation of dynamic visual sensing data in one unit by combining the CMOS dynamic visual sensing active pixel circuit and the storage and calculation integrated circuit. The purposes of saving storage area, reducing calculation power consumption and improving calculation speed are achieved.
The technical scheme adopted by the invention is as follows:
the CMOS sensing and storage integrated circuit structure based on the dynamic visual sensing technology is characterized by comprising a sensing circuit module based on an AER mode and a storage integrated circuit module;
the sensing circuit module based on the AER mode is used for sensing dynamic image data and specifically comprises a dynamic visual sensing active pixel circuit, a related secondary sampling circuit, a differential comparator circuit, a light intensity change detection circuit and a logic judgment circuit; wherein the content of the first and second substances,
the dynamic visual sensing active pixel circuit is used for sensing an input optical signal and converting the input optical signal into an electric signal;
the correlated secondary sampling circuit is coupled to the output end of the dynamic visual sensing active-phase pixel circuit and is used for sampling and holding the voltage signal which is converted and output in a reset period and an integration period respectively;
the differential comparator circuit is coupled to the output end of the related secondary sampling circuit and is used for carrying out differential operation on the secondary sampling result and comparing the operation result with the reference voltage;
the light intensity change detection circuit is used for sensing whether the dynamic light intensity changes or not;
the logic judgment circuit is respectively coupled to the differential comparator circuit and the light intensity change detection circuit and is used for carrying out validity judgment by combining data of the differential comparator and the light intensity detection circuit and outputting a judgment result; the method specifically comprises the following steps: only when the differential comparator judges that the light intensity is changed and the light intensity detection circuit judges that the dynamic light intensity is changed, effective data '1' is output and stored in the SRAM, otherwise, the effective data '1' is stored in the SRAM;
the storage and calculation integrated circuit module is used for storing and calculating dynamic image data and specifically comprises an SRAM unit, a weight write-in circuit, a digital logic unit, an analog accumulator and a linear amplifier; wherein the content of the first and second substances,
the SRAM unit is coupled to the output end of the logic judgment circuit and is used for storing external weight data and sensing data;
the weight value writing circuit is coupled to the input end of the SRAM unit and is used for writing external weight value data into the SRAM unit;
the digital logic unit is coupled to the output end of the SRAM unit, and the circuit structure of the digital logic unit comprises a multiplier for multiplying an external weight value and sensing data, a counter for identifying a multiplication result, and a pulse generating unit for generating a pulse signal;
the analog accumulator is coupled to the output end of the pulse generating unit and used for accumulating the number of the pulse signals in a capacitance charging mode and converting the pulse signals into analog voltage signals;
the linear amplifier is coupled to the output end of the analog accumulator and is used for further linearly amplifying the analog voltage signal.
Furthermore, the dynamic visual sensing active-phase pixel circuit mainly comprises a photodiode, a reset diode, a source follower and a row selection switch tube; wherein the content of the first and second substances,
the photodiode is used for sensing the external illumination intensity and converting the illumination intensity into an induced current;
the reset diode periodically works in a reset period and an integration period under the control of a reset signal; in a reset period, a reset diode is turned on to charge a negative terminal node of the photodiode; during an integration period, the reset diode is turned off, and the charge on the parasitic capacitance of the negative terminal node of the photodiode is linearly discharged by the induced current;
the source follower is used as a buffer, so that the charge of the negative terminal node of the photodiode can be prevented from leaking in the process of reading signals;
and the row selection switch tube is used for controlling the output of signals.
Furthermore, the related secondary sampling circuit consists of an MOS switching tube controlled by a clock signal, a sampling capacitor and a sampling signal reading circuit;
the MOS switching tube controlled by the clock signal is used for controlling the switching tube to be opened at the tail ends of a reset period and an integration period respectively and sampling voltage signals at the two moments;
the sampling capacitor is used for holding sampling data;
the sampling signal reading circuit is formed by connecting three P-type MOS tubes in series and is used as a buffer for reading a sampling voltage result and lifting the level.
Further, the differential comparator circuit is composed of a differential operation circuit for realizing a differential function;
the differential operation circuit for realizing the differential function consists of an operational amplifier and four corresponding equal resistors, can perform equal-proportion difference operation on two sampling voltages and output a differential operation result, and the differential operation result represents light intensity information.
Furthermore, the light intensity change detection circuit can judge whether the dynamic light intensity changes, namely, whether the variation of the light intensity is equal in different periods or not, namely whether the representative image is dynamic change or not, and only dynamically changed light intensity data is sampled.
Furthermore, the SRAM cell stores two portions of data, one of which is external weight data, and the other is dynamic image sensing data.
Further, the weight writing circuit is used for writing external weight data into the SRAM unit, and comprises 4 writing transistors and 2 parasitic capacitors. Because the SRAM requires the data lines to be pre-charged and pre-discharged before writing data, the accuracy and stability of the written data can be ensured.
Furthermore, the digital logic unit circuit structure comprises a multiplier for multiplying an external weight value and sensing data, a counter for identifying a multiplication result, and a pulse generating unit for generating a pulse signal;
the specific algorithm of the multiplier is that the sum operation is carried out on each bit of the external input weight and the sensing data;
the counter is used for counting the number of '1' in the data of the multiplication result;
the pulse generating unit emits an equal number of short-time pulse signals according to the counting result of the counter.
Furthermore, the analog accumulator is composed of a charging transistor, a discharging transistor, a summing capacitor and a buffer, and the working principle is as follows: when a pulse is received once, the charging transistor is turned on, the summing capacitor is charged once by small current, and the voltage on the capacitor is accumulated in equal proportion, so that a digital signal result can be converted into an analog voltage signal; and when the capacitor is fully charged, the charge is fully discharged by turning on the discharge transistor.
Furthermore, the linear amplifier is composed of an operational amplifier and a corresponding resistor, and the input of the operational amplifier is coupled with the output end of the analog accumulator, so that the output result of the analog accumulator can be subjected to linear operation and further amplified.
The invention has the beneficial effects that: the invention combines the existing CMOS memory-computation integrated chip with a dynamic visual sensing circuit adopting Address-Event triggering (AER) technology, thereby having the characteristics of high sampling rate, high speed, high precision and low time delay.
Drawings
FIG. 1 is a schematic diagram of an overall structure of a CMOS sensing and computing integrated circuit based on dynamic visual sensing technology;
FIG. 2 is a schematic diagram of the specific circuit configuration of FIG. 1;
FIG. 3 is a schematic diagram of a dynamic visual sensing active pixel circuit configuration;
FIG. 4 is a schematic diagram of the dynamic visual sense active pixel circuit output voltage;
FIG. 5 is a schematic diagram of a circuit configuration of a correlated double sampling circuit;
FIG. 6 is a schematic diagram of the operation of the correlated double sampling circuit;
FIG. 7 is a schematic circuit diagram of the differential comparator circuit;
FIG. 8 is a schematic diagram of a circuit configuration of the light intensity change detection circuit;
FIG. 9 is a schematic circuit configuration diagram of a logic decision circuit incorporating a differential comparator circuit and a light intensity change detection circuit;
FIG. 10 is a schematic diagram showing the output waveforms of the key signals of the logic decision circuit for the same light intensity transition during two consecutive periods;
FIG. 11 is a schematic diagram showing the output waveforms of the key signals of the logic decision circuit when the light intensity changes are different in two consecutive periods;
FIG. 12 is a circuit diagram of a weight write circuit;
FIG. 13 is a schematic circuit diagram of an SRAM cell;
FIG. 14 is a schematic diagram of the overall circuit structure of the weight write circuit and the SRAM cell;
FIG. 15 is a schematic diagram of a digital logic circuit;
FIG. 16 is a schematic diagram of a circuit configuration of an analog accumulation circuit;
FIG. 17 is a schematic circuit diagram of a linear amplifier;
FIG. 18 is a diagram illustrating the output waveforms of the analog accumulator and the linear amplifier for the pulse signal corresponding to the operation result;
FIG. 19 is a schematic diagram of another digital circuit structure for performing multiplication of 4-bit weight and 1-bit sensing data.
Detailed Description
The invention provides a CMOS sensing and calculating integrated circuit structure based on a dynamic visual sensing technology on the basis of the prior art, and the circuit structure can realize the functions of dynamic visual sensing, storage and calculation on the same circuit. The pixel array constituted by such circuits can realize not only sensing and storage of moving image data of a larger scale but also more complicated parallel arithmetic processing.
In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail by referring to specific embodiments in the accompanying drawings, it being understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.
As shown in fig. 1, a CMOS sensing and storage integrated circuit overall architecture based on dynamic visual sensing technology is composed of a sensing circuit module 1 based on an AER method and a storage integrated circuit module 2; the sensing circuit module 1 based on the AER mode comprises a dynamic visual sensing active pixel unit 3, a related secondary sampling circuit 5, a differential comparator circuit 7, a light intensity change detection circuit 9 and a logic judgment circuit 12; the storage and calculation integrated circuit module 2 comprises a weight value writing circuit 15, an SRAM unit 17, a digital logic unit 19, an analog accumulator 21 and a linear amplifier circuit 23; the signal flow when the circuit works is as follows: firstly, an external light intensity signal 14 is sensed by the DVS active pixel unit 3 and generates a sensing voltage signal 4; the related secondary sampling circuit 5 respectively samples the induction voltage signals 4 with two different periods and outputs a sampling result 6; the differential comparator circuit 7 performs differential operation on the sampling result 6 to obtain differential operation results 8 and 10; the light intensity detection circuit 9 obtains light intensity change information 11 according to the multiplication result 8; the logic decision circuit 12 comprehensively processes the difference operation result 10 and the light intensity change information 11 to obtain final sensing data 13; the SRAM cell 17 stores the external input weight 23 written by the weight write circuit 15 and the sensing data 13 at the same time, and transmits the storage data 18 to the digital logic unit 19; processing the external input weight 23 and the sensing data 13 in the digital logic unit 19 to obtain a multiplication result and sending pulse signal data 20 to the analog accumulation circuit 21; the analog accumulation circuit 21 transfers the accumulated result 22 to the linear amplification circuit 23, and obtains a final operation result 25.
Fig. 2 is a schematic diagram of a specific circuit structure corresponding to fig. 1.
Fig. 3 is a schematic structural diagram of the dynamic visual sensing active pixel circuit 3. This dynamic visual sensing active pixel cell is referred to as a 4-tube structure, and includes a reset tube 26, a source follower tube 27 and a row select tube 28, as well as a transistor 30 providing bias, in addition to a photodiode 31 in the pixel cell. The working principle of the pixel is as follows: the photodiode 31 goes through two cycles per operation under the control of the reset signal 21: a charging period and an integration period. In the charging period, the reset signal 21 keeps high level, the reset tube 26 is opened and the upper parasitic capacitance of the node 32 is charged to VDD-Vth, wherein Vth is the threshold voltage of the reset tube 26; during the integration period, the reset tube 26 is turned off and the photodiode 31 is illuminated with a certain intensity, and the charge stored in the parasitic capacitor on the node 32 is discharged by the induced current generated by the photodiode, so that the voltage on the node 32 is linearly decreased. The two working periods are performed circularly, so that the external light intensity change information can be sensed in real time through the photodiode 31, and the voltage signal on the node 32 can be read out without loss through the source follower 27 and the row selection switch 28, namely, the voltage signal is the output voltage signal 29.
Fig. 4 is a schematic diagram of the output voltage of the dynamic visual sense active pixel circuit 3. Each period T is divided into a charging period and a sub-period; during the charging period 0-t 1, node 32 is charged to VDD-Vth resulting in output node 29 being charged to Vr, which is about VDD-2 Vth; during the integration period t 1-t 2, the voltage at node 32 drops approximately linearly due to the presence of the induced current, and thus the voltage at output node 39 drops approximately linearly to vs 1; the situation of the second period T can be analyzed similarly.
As shown in fig. 5, the correlated double sampling circuit 5 has two identical sampling paths, and the circuit is composed of sampling tubes 33 and 40, sampling capacitors 35 and 42, source followers 38 and 45, switching tubes 37 and 44, and bias tubes 36 and 43. The working principle of the circuit is as follows: at the end of the charging period, the sampling tube 33 is opened through the control signal 34, the output voltage 29 is sampled and stored in the sampling capacitor 35, the sampling voltage is read out through the source follower 38, and the charging period sampling voltage 39 is obtained at the source node of the switching tube 37; the integration period operates as described above to obtain an integration period sample voltage 46.
Fig. 6 is a schematic diagram illustrating the operation principle of the correlated double sampling circuit 5. According to the output voltage change of the dynamic visual sensing active pixel circuit 3, at the end of the charging period, when the voltage of the node 29 is charged to VR, the control signal 34 of the sampling tube 33 generates a pulse signal to sample the voltage VR of the node 29 into the sampling capacitor 35, since the charging period will make the voltage of the node 29 rise to VR each time, so that VR remains unchanged thereafter, and at the same time, the sampled voltage is read out to the node VR without loss through the follower 38 and the switching tube 37 and remains unchanged, wherein VR is about VR + Vth; at the end of the integration period the voltage at node 29 decreases to a minimum value for one period, at which time the control signal 41 of the sampling tube 40 generates a pulse signal that samples the voltage VS1 at node 29 into the sampling capacitor 42, and the sampled voltage is read through the follower 45 and the switching tube 44 lossless to node VS, which is approximately VS1+ Vth until the next sampling. The situation of the second period T is obtained by the same analysis.
As shown in fig. 7, a schematic diagram of the differential comparator circuit 7 is shown, and the differential comparator mainly includes a differential circuit and a comparator circuit. The differential circuit consists of three equal resistors 40, 42 and 43 (all R), a feedback resistor 42 (Rf) and a high-precision operational amplifier 44, and if the resistor R is selected to be Rf, the output of the operational amplifier can be derived from the principles of "virtual short" and "virtual break" as follows: therefore, by comparing VD with the reference voltage Vrefh, it can be determined whether or not a sufficient light intensity is sensed during one period T. If VD is large, it means that the voltage change of VS is large during the integration period, i.e. enough light intensity information is sensed, so that the comparator 45 outputs a high level, otherwise it outputs a low level.
As shown in FIG. 8, it is a schematic diagram of the light intensity change detection circuit 9, which is also composed of a differential comparison circuit and a dual comparator circuit, wherein the differential comparison circuit is composed of three equal resistors 46, 47, 48 (all resistance values are R) and a feedback resistor 49 (resistance value is Rf), and a high-precision operational amplifier 50, if the resistance value R is selected, it can be obtained from the above analysis that △ V is VD2-VD1 ', wherein VD2 represents the voltage VD of the node 54 calculated by the differential comparison circuit in the period T, VD1 ' represents the voltage VD of the node 54 calculated by the differential comparison circuit in the last period T, wherein VD1 ' can be realized by a buffer 53, so that the change of VD voltage in two adjacent periods T can be detected, then the obtained △ V is compared with two reference voltages + -efl at the same time, if the change of △ V is higher than or lower than Vrefl, the circuit outputs a high level indicating that the light intensity change of the light intensity in the last period T can be detected, otherwise, the change of the light intensity in the last period T can be detected, and if the light intensity change of the light intensity in the last period T is detected, the dynamic level of the light intensity change is not detected, the same as the last period T.
As shown in fig. 9, is a schematic circuit configuration diagram of the logic decision circuit 12 in combination with the differential comparator circuit 7 and the light intensity change detection circuit 9. The circuit mainly comprises a comparator 44 in a differential comparator circuit 7, comparators 51 and 52 in a light intensity conversion detection circuit 9, a logic OR gate 55 and a logic AND gate 56, and the working principle is as follows: only when the differential comparator circuit 7 senses the illumination intensity information of sufficient intensity and the light intensity detection circuit 9 detects that the light intensity change of the period T is inconsistent with the light intensity change of the previous period T, it is determined that the dynamic light intensity information is detected as a whole and a high level is output, otherwise a low level is output.
Fig. 10 is a schematic diagram showing output waveforms of the key signals VC, VCD, and VA of the logic decision circuit 12 when the light intensity transitions are the same in two consecutive periods; it can be seen from the change in Vph in the graph that sufficient light intensity is sensed during two consecutive periods T, resulting in the same amount of voltage change. Therefore, although the differential comparator circuit 7 outputs a high level indicating that the intensity of light is sensed, the light intensity change detection circuit 9 does not sense the dynamically changing light intensity information, and therefore outputs a low level. While the logic decision circuit also outputs a low level.
Fig. 11 is a schematic diagram showing output waveforms of the key signals VC, VCD, and VA of the logic determination circuit 12 when the light intensity conversion conditions are different in two consecutive periods; it can be seen from the change of Vph in the graph that in two consecutive periods T, sufficient light intensity is sensed, but the amount of voltage change generated in the two periods is not the same. Therefore, the differential comparator circuit 7 outputs a high level indicating that the intensity of the light is sensed, and the light intensity change detection circuit 9 also senses the dynamically changing light intensity information, thereby outputting a high level. At the same time the logic decision circuit also outputs a high level, representing that dynamic sense data is recognized, and stores the data into the SRAM cell 17.
As shown in fig. 12, it is a schematic structural diagram of the weight write circuit 15, which mainly includes write tubes 59 and 60 and bias tubes 57 and 58, where 61 and 62 are parasitic capacitances, and the operation principle is as follows: when the corresponding weight data W needs to be written, the signal 63 and the inverted signal 64 thereof need to be simultaneously input, assuming that data "1" needs to be written into the SRAM cell 17, the input signal 63 is at a low level, the inverted signal 64 thereof is at a high level, the node 65(BL) is pulled down to GND, the node 66(BLB) is charged to VDD, and the written data "0" can be analyzed in the same manner, and then the weight data can be stably written into the SRAM cell 17 by cooperating with the corresponding control signal of the SRAM cell 17.
As shown in fig. 13, which is a schematic diagram of a single memory cell structure of the SRAM cell 17, a CMOS inverter formed by a PMOS transistor 67 and an NMOS transistor 71 is connected end to end with a CMOS inverter formed by a PMOS transistor 68 and an NMOS transistor 72, which is a classic SRAM structure and can be used to store 1-bit data. To store a data "1", it is necessary that signal 65(BL) remain high and signal 66(BLB) remain low; to store a data "0," it is desirable that signal 65(BL) remain low and signal 66(BLB) remain high. When data needs to be stored into the SRAM cell 17, preprocessing by the weight value writing circuit 15 is required, that is, when the weight value writing circuit 15 precharges the data node 65(BL) and the node 66(BLB), and after the written data is ready, the write control signal 73 keeps a high level to allow data writing, and at this time, writing of the weight value data is completed; the sensing data is also stored in the SRAM cell 17, and the writing principle is the same as described above.
Fig. 14 is a schematic diagram of the overall circuit structure of the weight value writing circuit and the SRAM cell.
As shown in fig. 15, it is a schematic diagram of the architecture of the digital logic unit 19, and the digital logic unit 19 is implemented by using a digital circuit, which mainly includes a multiplier 76, a counter 77, and a pulse generating unit 78. Assuming that the external weight data and the sensing data are both 1bit data, the specific algorithm of the multiplier 76 is to perform an and operation on the external input weight and the sensing data; the counter is used for counting the number of '1' appearing in the data of the multiplication result; and the pulse generating unit sends out an equal number of short-time pulse signals according to the counting result of the counter.
As shown in fig. 16, it is a schematic diagram of a circuit structure of the analog accumulator 21, which mainly comprises a charging tube 80, a discharging tube 81, a summing capacitor 84 and a buffer 86, and the working principle is as follows: the hold signal 83 is kept low, so long as when the pulse generating unit 78 delivers a pulse, the charging tube 80 charges the summing capacitor 84 for a short period of time with a small current, and the charging current forms a summing voltage on the summing capacitor 84, which is read out by the buffer 86 to a subsequent circuit, and the summing voltage can be cleared through the discharging tube 81.
As shown in fig. 17, the linear amplifier 23 is a schematic diagram, and is composed of a high-precision operational amplifier 90, an input resistor 87, a matching resistor 88, and a feedback resistor 89, and it can be deduced from the principles of "virtual short" and "virtual short" of the operational amplifier that VOUT is-R2/R1 × Va, that is, the output result of the analog accumulator can be linearly operated and further amplified to obtain the final output VOUT of the system.
As shown in fig. 18, it is a schematic diagram of the output voltage when the analog accumulator 21 and the linear amplifier 23 are operated, and it can be seen from the diagram that, after the digital logic unit 19 obtains the result of the multiplication operation, the corresponding pulse signal is issued to the analog accumulator 21 by the pulse generating unit 78, the analog accumulator 21 is charged, and every time a pulse signal is generated, the voltage VQ of the analog accumulator 21 is increased by △ v, and can be read by the linear amplifier 23 through the voltage follower 86, and can be amplified by-R2/R1.
Fig. 19 is a schematic diagram of another digital circuit structure for realizing multiplication of 4-bit weight and 1-bit sensing data. The weight writing circuit 15, the logic decision circuit 12, and the structure and the operation principle of the SRAM cell are not changed, and the external input weight 23 stores 4-bit weight data into the SRAM cell 17. The weight data of 4 bits and the sensing data of 1bit are operated, and the specific rule of multiplication operation is changed into that: performing AND operation on the weight data of 4 bits and the sensing data of 1 bit; that is, every time the logic decision circuit 12 stores a "1" into the SRAM cell 17, 4-bit weight data is accumulated once into the 8-bit adder 100; if the data stored by the logic decision circuit 12 into the SRAM cell 17 is "0", the data in the previous 8-bit adder 100 is saved until the 8-bit adder 100 overflows, and the 8-bit adder 100 is cleared at this time. The initial value of the 8-bit adder 100 is set to 8' b 00000000. The digital logic unit 98 may also be implemented by digital circuitry.

Claims (10)

1. The CMOS sensing and storage integrated circuit structure based on the dynamic visual sensing technology is characterized by comprising a sensing circuit module based on an AER mode and a storage integrated circuit module;
the sensing circuit module based on the AER mode is used for sensing dynamic image data and specifically comprises a dynamic visual sensing active pixel circuit, a related secondary sampling circuit, a differential comparator circuit, a light intensity change detection circuit and a logic judgment circuit; wherein the content of the first and second substances,
the dynamic visual sensing active pixel circuit is used for sensing an input optical signal and converting the input optical signal into an electric signal;
the correlated secondary sampling circuit is coupled to the output end of the dynamic visual sensing active-phase pixel circuit and is used for sampling and holding the voltage signal which is converted and output in a reset period and an integration period respectively;
the differential comparator circuit is coupled to the output end of the related secondary sampling circuit and is used for carrying out differential operation on the secondary sampling result and comparing the operation result with the reference voltage;
the light intensity change detection circuit is used for sensing whether the dynamic light intensity changes or not;
the logic judgment circuit is respectively coupled to the differential comparator circuit and the light intensity change detection circuit and is used for carrying out validity judgment by combining data of the differential comparator and the light intensity detection circuit and outputting a judgment result; the method specifically comprises the following steps: only when the differential comparator judges that the light intensity is changed and the light intensity detection circuit judges that the dynamic light intensity is changed, effective data '1' is output and stored in the SRAM, otherwise, the effective data '1' is stored in the SRAM;
the storage and calculation integrated circuit module is used for storing and calculating dynamic image data and specifically comprises an SRAM unit, a weight write-in circuit, a digital logic unit, an analog accumulator and a linear amplifier; wherein the content of the first and second substances,
the SRAM unit is coupled to the output end of the logic judgment circuit and is used for storing external weight data and sensing data;
the weight value writing circuit is coupled to the input end of the SRAM unit and is used for writing external weight value data into the SRAM unit;
the digital logic unit is coupled to the output end of the SRAM unit, and the circuit structure of the digital logic unit comprises a multiplier for multiplying an external weight value and sensing data, a counter for identifying a multiplication result, and a pulse generating unit for generating a pulse signal;
the analog accumulator is coupled to the output end of the pulse generating unit and used for accumulating the number of the pulse signals in a capacitance charging mode and converting the pulse signals into analog voltage signals;
the linear amplifier is coupled to the output end of the analog accumulator and is used for further linearly amplifying the analog voltage signal.
2. The CMOS sensing and computing integrated circuit structure based on the dynamic visual sensing technology according to claim 1, wherein the dynamic visual sensing active pixel circuit mainly comprises a photodiode, a reset diode, a source follower and a row selection switch tube; wherein the content of the first and second substances,
the photodiode is used for sensing the external illumination intensity and converting the illumination intensity into an induced current;
the reset diode periodically works in a reset period and an integration period under the control of a reset signal; in a reset period, a reset diode is turned on to charge a negative terminal node of the photodiode; during an integration period, the reset diode is turned off, and the charge on the parasitic capacitance of the negative terminal node of the photodiode is linearly discharged by the induced current;
the source follower is used as a buffer, so that the charge of the negative terminal node of the photodiode can be prevented from leaking in the process of reading signals;
and the row selection switch tube is used for controlling the output of signals.
3. The CMOS sensing and computing integrated circuit structure based on the dynamic visual sensing technology according to claim 2, wherein the correlated double sampling circuit is composed of a MOS switch tube controlled by a clock signal, a sampling capacitor and a sampling signal reading circuit;
the MOS switching tube controlled by the clock signal is used for controlling the switching tube to be opened at the tail ends of a reset period and an integration period respectively and sampling voltage signals at the two moments;
the sampling capacitor is used for holding sampling data;
the sampling signal reading circuit is formed by connecting three P-type MOS tubes in series and is used as a buffer for reading a sampling voltage result and lifting the level.
4. The CMOS sensing and computing integrated circuit structure based on dynamic visual sensing technology according to claim 3, wherein the differential comparator circuit is composed of a differential operation circuit for implementing a differential function;
the differential operation circuit for realizing the differential function consists of an operational amplifier and four corresponding equal resistors, can perform equal-proportion difference operation on two sampling voltages and output a differential operation result, and the differential operation result represents light intensity information.
5. The CMOS sensor-integrator circuit structure of claim 4, wherein the light intensity variation detection circuit is capable of determining whether the dynamic light intensity is changed, that is, by determining whether the variation of the light intensity, which represents whether the image is dynamically varied, is equal in different periods, only dynamically varied light intensity data is sampled.
6. The CMOS sensor-memory integrated circuit structure based on dynamic visual sensing technology as claimed in claim 5, wherein said SRAM cells store two parts of data respectively, one part being external weight data and the other part being dynamic image sensing data.
7. The CMOS sensing integrated circuit structure based on dynamic visual sensing technology of claim 6, wherein the weight write circuit is used to write external weight data to the SRAM cell, which comprises 4 write transistors and 2 parasitic capacitances.
8. The CMOS sensing and computing integrated circuit structure based on dynamic visual sensing technology according to claim 7, wherein the digital logic unit circuit structure comprises a multiplier for multiplying an external weight value and sensing data, a counter for identifying the multiplication result, and a pulse generating unit for generating a pulse signal;
the specific algorithm of the multiplier is that the sum operation is carried out on each bit of the external input weight and the sensing data;
the counter is used for counting the number of '1' in the data of the multiplication result;
the pulse generating unit emits an equal number of short-time pulse signals according to the counting result of the counter.
9. The CMOS integrated circuit structure for sensing and calculating based on dynamic visual sensing technology according to claim 8, wherein the analog accumulator is composed of a charge transistor, a discharge transistor, a summing capacitor and a buffer, and its operation principle is as follows: when a pulse is received once, the charging transistor is turned on, the summing capacitor is charged once by small current, and the voltage on the capacitor is accumulated in equal proportion, so that a digital signal result can be converted into an analog voltage signal; and when the capacitor is fully charged, the charge is fully discharged by turning on the discharge transistor.
10. The CMOS integrated circuit structure for sensing and computing based on dynamic visual sensing technology according to claim 9, wherein the linear amplifier is composed of an operational amplifier and a corresponding resistor, and an input of the operational amplifier is coupled to an output terminal of the analog accumulator, so as to perform linear operation on an output result of the analog accumulator for further amplification.
CN202010272251.0A 2020-04-09 2020-04-09 CMOS (complementary Metal oxide semiconductor) sensing and calculating integrated circuit structure based on dynamic visual sensing technology Active CN111343398B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010272251.0A CN111343398B (en) 2020-04-09 2020-04-09 CMOS (complementary Metal oxide semiconductor) sensing and calculating integrated circuit structure based on dynamic visual sensing technology

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010272251.0A CN111343398B (en) 2020-04-09 2020-04-09 CMOS (complementary Metal oxide semiconductor) sensing and calculating integrated circuit structure based on dynamic visual sensing technology

Publications (2)

Publication Number Publication Date
CN111343398A true CN111343398A (en) 2020-06-26
CN111343398B CN111343398B (en) 2021-10-26

Family

ID=71187529

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010272251.0A Active CN111343398B (en) 2020-04-09 2020-04-09 CMOS (complementary Metal oxide semiconductor) sensing and calculating integrated circuit structure based on dynamic visual sensing technology

Country Status (1)

Country Link
CN (1) CN111343398B (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112601037A (en) * 2020-12-14 2021-04-02 电子科技大学 Floating gate device-based image sensing and storage integrated pixel unit and pixel array
CN113052024A (en) * 2021-03-12 2021-06-29 电子科技大学 Sense and memory integrated circuit structure based on memristor and sensor
CN113596361A (en) * 2021-08-02 2021-11-02 电子科技大学 Sense-memory-computation integrated circuit structure for realizing positive and negative weight calculation in pixel
CN115756388A (en) * 2023-01-06 2023-03-07 上海后摩智能科技有限公司 Multi-mode storage and calculation integrated circuit, chip and calculation device
CN115906735A (en) * 2023-01-06 2023-04-04 上海后摩智能科技有限公司 Multi-bit-number storage and calculation integrated circuit based on analog signals, chip and calculation device
CN115988350A (en) * 2022-10-20 2023-04-18 电子科技大学 CMOS image sensing, storing and calculating integrated circuit integrating sampling calculation
CN117479027A (en) * 2023-11-07 2024-01-30 上海宇勘科技有限公司 SRAM-based method for simulating dynamic vision sensor array

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101309360A (en) * 2007-05-17 2008-11-19 索尼株式会社 Video input processor, imaging signal-processing circuit, and method for reducing noises in imaging signals
CN102710906A (en) * 2012-04-16 2012-10-03 天津大学 CMOS (complementary metal-oxide-semiconductor transistor) image sensor for realizing two-dimensional discrete cosine transformation
WO2018237110A1 (en) * 2017-06-22 2018-12-27 Newvoicemedia Us Inc. Using conversational similarity for dimension reduction in deep analytics
US20190050875A1 (en) * 2017-06-22 2019-02-14 NewVoiceMedia Ltd. Customer interaction and experience system using emotional-semantic computing
CN110870297A (en) * 2017-07-19 2020-03-06 高通股份有限公司 Motion detection on CMOS image sensor die using inter-pixel grid relationships
CN110891152A (en) * 2018-09-07 2020-03-17 三星电子株式会社 Image sensor including CMOS image sensor pixels and dynamic vision sensor pixels

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101309360A (en) * 2007-05-17 2008-11-19 索尼株式会社 Video input processor, imaging signal-processing circuit, and method for reducing noises in imaging signals
CN102710906A (en) * 2012-04-16 2012-10-03 天津大学 CMOS (complementary metal-oxide-semiconductor transistor) image sensor for realizing two-dimensional discrete cosine transformation
WO2018237110A1 (en) * 2017-06-22 2018-12-27 Newvoicemedia Us Inc. Using conversational similarity for dimension reduction in deep analytics
US20190050875A1 (en) * 2017-06-22 2019-02-14 NewVoiceMedia Ltd. Customer interaction and experience system using emotional-semantic computing
CN110870297A (en) * 2017-07-19 2020-03-06 高通股份有限公司 Motion detection on CMOS image sensor die using inter-pixel grid relationships
CN110891152A (en) * 2018-09-07 2020-03-17 三星电子株式会社 Image sensor including CMOS image sensor pixels and dynamic vision sensor pixels

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112601037A (en) * 2020-12-14 2021-04-02 电子科技大学 Floating gate device-based image sensing and storage integrated pixel unit and pixel array
CN112601037B (en) * 2020-12-14 2021-10-26 电子科技大学 Floating gate device-based image sensing and storage integrated pixel unit and pixel array
CN113052024A (en) * 2021-03-12 2021-06-29 电子科技大学 Sense and memory integrated circuit structure based on memristor and sensor
CN113596361A (en) * 2021-08-02 2021-11-02 电子科技大学 Sense-memory-computation integrated circuit structure for realizing positive and negative weight calculation in pixel
CN113596361B (en) * 2021-08-02 2022-07-12 电子科技大学 Sense-memory-computation integrated circuit structure for realizing positive and negative weight calculation in pixel
CN115988350A (en) * 2022-10-20 2023-04-18 电子科技大学 CMOS image sensing, storing and calculating integrated circuit integrating sampling calculation
CN115756388A (en) * 2023-01-06 2023-03-07 上海后摩智能科技有限公司 Multi-mode storage and calculation integrated circuit, chip and calculation device
CN115906735A (en) * 2023-01-06 2023-04-04 上海后摩智能科技有限公司 Multi-bit-number storage and calculation integrated circuit based on analog signals, chip and calculation device
CN115906735B (en) * 2023-01-06 2023-05-05 上海后摩智能科技有限公司 Multi-bit number storage and calculation integrated circuit, chip and calculation device based on analog signals
CN117479027A (en) * 2023-11-07 2024-01-30 上海宇勘科技有限公司 SRAM-based method for simulating dynamic vision sensor array
CN117479027B (en) * 2023-11-07 2024-05-24 上海宇勘科技有限公司 SRAM-based method for simulating dynamic vision sensor array

Also Published As

Publication number Publication date
CN111343398B (en) 2021-10-26

Similar Documents

Publication Publication Date Title
CN111343398B (en) CMOS (complementary Metal oxide semiconductor) sensing and calculating integrated circuit structure based on dynamic visual sensing technology
Biswas et al. Conv-RAM: An energy-efficient SRAM with embedded convolution computation for low-power CNN-based machine learning applications
JP2020503752A (en) Dynamic vision sensor architecture
CN112151092B (en) Storage unit, storage array and in-memory computing device based on 4-pipe storage
CN113596361B (en) Sense-memory-computation integrated circuit structure for realizing positive and negative weight calculation in pixel
CN111309670A (en) CMOS (complementary Metal oxide semiconductor) inductive storage and calculation integrated circuit structure
CN114546335B (en) Memory computing device for multi-bit input and multi-bit weight multiplication accumulation
US20230196079A1 (en) Enhanced dynamic random access memory (edram)-based computing-in-memory (cim) convolutional neural network (cnn) accelerator
CN114038492B (en) Multiphase sampling memory internal computing circuit
CN115390789A (en) Magnetic tunnel junction calculation unit-based analog domain full-precision memory calculation circuit and method
CN112116937A (en) SRAM circuit structure for realizing multiplication and or logic operation in memory
CN113936717B (en) Storage and calculation integrated circuit for multiplexing weight
CN116092553A (en) Memory with multiplication and addition functions
US11626153B2 (en) Low power static random-access memory
CN112700810B (en) CMOS sense-memory integrated circuit structure integrating memristors
KR100306876B1 (en) High speed image sensor having feedback circuit
CN1229994A (en) Semiconductor memory device
CN117130978A (en) Charge domain in-memory computing circuit based on sparse tracking ADC and computing method thereof
CN115954029A (en) Multi-bit operation module and in-memory calculation circuit structure using the same
CN115988350A (en) CMOS image sensing, storing and calculating integrated circuit integrating sampling calculation
CN112737590A (en) Current type reading circuit applied to high-speed large-area array ADC (analog to digital converter)
CN112468747A (en) Current type reading circuit applied to large-area array image sensor and control method thereof
JP4628586B2 (en) Photodetector
CN201242868Y (en) Sensitive amplifier for EEPROM and read circuit comprised by the same
WO2024082377A1 (en) D6t in-memory computing accelerator capable of always linearly discharging and reducing numeric steps

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant