CN115295544B - Infrared sensing and calculation integrated chip and working method thereof - Google Patents
Infrared sensing and calculation integrated chip and working method thereof Download PDFInfo
- Publication number
- CN115295544B CN115295544B CN202211194541.3A CN202211194541A CN115295544B CN 115295544 B CN115295544 B CN 115295544B CN 202211194541 A CN202211194541 A CN 202211194541A CN 115295544 B CN115295544 B CN 115295544B
- Authority
- CN
- China
- Prior art keywords
- infrared
- infrared detector
- detector array
- sensing
- bias voltage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004364 calculation method Methods 0.000 title claims abstract description 29
- 238000000034 method Methods 0.000 title claims abstract description 17
- 230000005855 radiation Effects 0.000 claims abstract description 43
- 230000009471 action Effects 0.000 claims abstract description 12
- 238000006243 chemical reaction Methods 0.000 claims description 33
- 230000003321 amplification Effects 0.000 claims description 30
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 30
- 239000003990 capacitor Substances 0.000 claims description 11
- 239000000126 substance Substances 0.000 claims description 7
- 238000005070 sampling Methods 0.000 claims description 5
- 238000011017 operating method Methods 0.000 claims description 4
- 230000001953 sensory effect Effects 0.000 claims 3
- 238000013473 artificial intelligence Methods 0.000 abstract description 10
- 230000005540 biological transmission Effects 0.000 abstract description 10
- 230000010354 integration Effects 0.000 abstract description 6
- 230000006698 induction Effects 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 239000004065 semiconductor Substances 0.000 description 3
- 210000000225 synapse Anatomy 0.000 description 3
- 210000004556 brain Anatomy 0.000 description 2
- 229910044991 metal oxide Inorganic materials 0.000 description 2
- 150000004706 metal oxides Chemical class 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000013139 quantization Methods 0.000 description 2
- 101100446506 Mus musculus Fgf3 gene Proteins 0.000 description 1
- 101000767160 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) Intracellular protein transport protein USO1 Proteins 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 239000008358 core component Substances 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 230000005669 field effect Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000003278 mimic effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
- H01L25/00—Assemblies consisting of a plurality of individual semiconductor or other solid state devices ; Multistep manufacturing processes thereof
- H01L25/18—Assemblies consisting of a plurality of individual semiconductor or other solid state devices ; Multistep manufacturing processes thereof the devices being of types provided for in two or more different subgroups of the same main group of groups H01L27/00 - H01L33/00, or in a single subclass of H10K, H10N
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
- G01J5/10—Radiation pyrometry, e.g. infrared or optical thermometry using electric radiation detectors
- G01J5/20—Radiation pyrometry, e.g. infrared or optical thermometry using electric radiation detectors using resistors, thermistors or semiconductors sensitive to radiation, e.g. photoconductive devices
- G01J5/22—Electrical features thereof
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
- G01J5/10—Radiation pyrometry, e.g. infrared or optical thermometry using electric radiation detectors
- G01J5/20—Radiation pyrometry, e.g. infrared or optical thermometry using electric radiation detectors using resistors, thermistors or semiconductors sensitive to radiation, e.g. photoconductive devices
- G01J5/22—Electrical features thereof
- G01J5/24—Use of specially adapted circuits, e.g. bridge circuits
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
- G01J5/48—Thermography; Techniques using wholly visual means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/15—Correlation function computation including computation of convolution operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
- H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
- H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
- H01L27/144—Devices controlled by radiation
- H01L27/146—Imager structures
- H01L27/14643—Photodiode arrays; MOS imagers
- H01L27/14649—Infrared imagers
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03F—AMPLIFIERS
- H03F1/00—Details of amplifiers with only discharge tubes, only semiconductor devices or only unspecified devices as amplifying elements
- H03F1/38—Positive-feedback circuit arrangements without negative feedback
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M1/00—Analogue/digital conversion; Digital/analogue conversion
- H03M1/12—Analogue/digital converters
- H03M1/124—Sampling or signal conditioning arrangements specially adapted for A/D converters
- H03M1/1245—Details of sampling arrangements or methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Physics (AREA)
- Power Engineering (AREA)
- Pure & Applied Mathematics (AREA)
- Computational Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Microelectronics & Electronic Packaging (AREA)
- Computer Hardware Design (AREA)
- Algebra (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Condensed Matter Physics & Semiconductors (AREA)
- Electromagnetism (AREA)
- Photometry And Measurement Of Optical Pulse Characteristics (AREA)
Abstract
The disclosure relates to the technical field of artificial intelligence, and provides an infrared sensing and calculation integrated chip and a working method thereof. The infrared sensing and calculating integrated chip comprises an infrared detector array and a sensing and calculating integrated processing chip electrically connected with the infrared detector array; the infrared detector array is used for detecting infrared thermal radiation signals and outputting current signals based on the infrared thermal radiation signals under the action of the sensing and calculation integrated processing chip; the sensing and calculating integrated processing chip is used for controlling the infrared detector array to output current signals based on the infrared thermal radiation signals, and obtaining the operation result of convolution operation. The system can simultaneously realize the induction and convolution operation of infrared heat radiation, avoids the problem of data handling, greatly saves the data transmission bandwidth, improves the efficiency ratio of convolution operation, further reduces the size and weight of the system, has the characteristics of high integration level, high efficiency ratio and small size and light weight, and is easy to be carried on various edge side devices to realize all-weather real-time artificial intelligence application.
Description
Technical Field
The disclosure relates to the technical field of artificial intelligence, in particular to an infrared sensing and calculating integrated chip and a working method thereof.
Background
In recent years, inspired by biological brains, edge side processors and special neural network acceleration chips begin to develop towards neuromorphic hardware architectures simulating human neural synapse behaviors, and the non-von hardware processor architecture based on neuromorphic calculation breaks the limitation of the traditional von Neumann storage wall, improves the energy efficiency of a system through highly parallel operation, and is suitable for the requirements of various end side AI tasks on the energy efficiency. Crosspoint-based synapse array structures, such as a one-transistor one-memristor (1T 1R) crosspoint array structure, can mimic the interconnect topology of human brain synapses for implementing Artificial Intelligence (AI) algorithms. In the "integrated memory" structure, a resistive non-volatile memory (RRAM) is usually used as a memory to store a weight matrix for neural network operation, and the RRAM is connected to a Metal-Oxide-Semiconductor Field-Effect Transistor (MOSFET), i.e., a MOS gate Transistor, to control erasing and writing of the memory through word lines and bit lines, and to implement operation of the convolutional neural network.
Although the storage and computation integrated architecture solves the problems of power consumption and time delay caused by the data handling problem of the storage wall to a certain extent, in the end-side equipment, a front-end sensor is still required to sense a detection signal, an analog-to-digital converter is also required to quantize the detected signal, and then various end-side AI tasks can be completed through a storage and computation integrated chip or other types of AI chips. Data interaction between the sensor and the back-end AI chip requires a high data transmission bandwidth, which also causes problems of power consumption and data transmission delay.
The image sensor and various focal plane detectors are used as direct windows of end-side equipment interacting with the environment, and play a very important role. Currently, an intelligent image sensor mounted on an end side generally accesses an image processing chip (ISP), a neural network dedicated processor (NPU), a Field Programmable Gate Array (FPGA), and the like through a conventional image sensor to implement AI tasks such as target recognition and tracking. However, this brings about problems of a data transmission bandwidth between the image sensor and the back-end image processing chip, a power consumption delay, and a light weight of the entire end-side device.
Disclosure of Invention
The present disclosure is directed to at least one of the problems of the prior art, and provides an infrared sensing and calculation integrated chip and a working method thereof.
In one aspect of the present disclosure, an infrared calculation integrated chip is provided. The infrared sensing and calculating integrated chip comprises an infrared detector array and a sensing and calculating integrated processing chip electrically connected with the infrared detector array;
the infrared detector array is used for detecting infrared thermal radiation signals and outputting current signals based on the infrared thermal radiation signals under the action of the sensing and calculation integrated processing chip;
and the sensing and calculating integrated processing chip is used for controlling the infrared detector array to output current signals based on the infrared thermal radiation signals so as to obtain an operation result of convolution operation.
Optionally, the sensing and computing integrated processing chip includes a bias voltage generating module, a gating device array, a column amplification and conversion module, and a control module, wherein:
the first ends of the infrared detectors in each row are respectively and electrically connected with the bias voltage generation module and the control module through corresponding row output lines;
the first end of each row of the gating devices is electrically connected with the second end of the corresponding row of the infrared detectors, the second end of each row of the gating devices is electrically connected with the corresponding amplification conversion module through the corresponding row output line, and the control end of each row of the gating devices is electrically connected with the control module; wherein the content of the first and second substances,
the control module is used for controlling the bias voltage generation module to apply bias voltage to the infrared detector array; and the number of the first and second groups,
the control module is also used for controlling the gating device array to gate according to a preset gating mode and adjusting the bias voltage so that the infrared detector array outputs a current signal under the combined action of the infrared thermal radiation signal and the adjusted bias voltage;
and the column amplification conversion module is used for amplifying the current signal and converting the amplified current signal into a voltage signal to obtain an operation result.
Optionally, the gating device adopts an MOS gate tube;
the drain electrode of the MOS gate tube is a first end of the gating device, the source electrode of the MOS gate tube is a second end of the gating device, and the grid electrode of the MOS gate tube is a control end of the gating device.
Optionally, the amplifying and converting module includes a capacitive transconductance amplifier and a sample-and-hold circuit, wherein:
the input end of the capacitor transconductance amplifier is electrically connected with the corresponding column output line, and the output end of the capacitor transconductance amplifier is electrically connected with the input end of the sampling hold circuit.
Optionally, the infrared sensing and computation integrated chip further includes a buffer and an analog-to-digital converter, and the analog-to-digital converter is electrically connected to the output end of the sample-and-hold circuit through the buffer.
Optionally, the infrared detector comprises a thermistor or a photo-resistor type device.
In another aspect of the present disclosure, an operating method of an infrared sensing and calculation integrated chip is provided, which is applied to the infrared sensing and calculation integrated chip described above. The working method of the infrared sensing and calculating integrated chip comprises the following steps:
detecting an infrared thermal radiation signal by an infrared detector array;
the sensing and calculating integrated processing chip controls the infrared detector array to output current signals based on the infrared thermal radiation signals, and operation results of convolution operation are obtained.
Optionally, when the sensing and computation integrated processing chip comprises the bias voltage generation module, the gating device array, the column amplification conversion module and the control module,
the sensing and calculating integrated processing chip controls the infrared detector array to output current signals based on infrared thermal radiation signals to obtain an operation result of convolution operation, and the method comprises the following steps of:
the control module controls the bias voltage generation module to apply bias voltage to the infrared detector array;
the control module controls the gating device array to gate according to a preset gating mode, adjusts the bias voltage and configures the adjusted bias voltage to the infrared detector array;
the infrared detector array outputs a current signal under the combined action of the infrared thermal radiation signal and the adjusted bias voltage;
the column amplification conversion module amplifies the current signal, converts the amplified current signal into a voltage signal and obtains an operation result.
Alternatively, the current signal is expressed as the following formula (1) according to kirchhoff's law:
wherein the content of the first and second substances,g ij indicating the second in the infrared detector arrayiGo to the firstjThe conductance of the infrared detectors of the column, and is determined from the infrared thermal radiation signal,i=1,2,…,mindicating the number of rows in the infrared detector array,mrepresenting the number of rows of the infrared detector array,j=1,2,…,nindicating the column number in the infrared detector array,nindicating the number of columns of the infrared detector array,V ini indicating the second in the infrared detector arrayiVoltage across the line infrared detector isiAdjusted bias voltage corresponding to line infrared detector，I j Indicating the second in the infrared detector arrayjAnd current signals on column output lines corresponding to the column infrared detectors.
Alternatively, the current signal is expressed as the following formula (2) according to kirchhoff's law:
wherein the content of the first and second substances,g ij indicating the second in the infrared detector arrayiGo to the firstjThe conductance of the infrared detectors of the column, and is determined from the infrared thermal radiation signal,i=1,2,…,mindicating the number of rows in the infrared detector array,mrepresenting the number of rows of the infrared detector array,j=1,2,…,nindicating the column number in the infrared detector array,nindicating the number of columns of the infrared detector array,V inki indicating the second in the infrared detector arrayiFirst of the line infrared detectorkVoltage across the group, and isiThe first corresponding to the line infrared detectorkThe adjusted bias voltage of the bank is set,I jk indicating the second in the infrared detector arrayjA second one on the column output line corresponding to the column infrared detectorkThe set of current signals is then used to generate a set of current signals,k=1,2,…,t，tindicating the number of sets set in advance.
Compared with the prior art, the infrared detector array and the sensing and calculating integrated processing chip are combined together, and the single-chip integrated sensing and calculating integrated framework can simultaneously realize the sensing of infrared heat radiation and convolution operation, avoid the problem of data carrying during convolution operation, greatly save the data transmission bandwidth from a front-end sensor to a rear-end processing chip, effectively solve the problems of poor integration level, high data transmission bandwidth and the like caused by the adoption of a discrete intelligent processing chip of the existing intelligent image sensor, also improve the efficiency ratio of convolution operation, further reduce the size and the weight of a system, have the characteristics of high integration level, high efficiency ratio and small size and light weight, and can be easily carried on various edge side equipment to realize all-day and all-weather real-time artificial intelligent application.
Drawings
One or more embodiments are illustrated by way of example in the accompanying drawings, which correspond to the figures in which like reference numerals refer to similar elements and which are not to scale unless otherwise specified.
Fig. 1 is a schematic structural diagram of an infrared sensing and computing integrated chip according to an embodiment of the present disclosure;
fig. 2 is a schematic diagram of an array structure of an infrared sensing and computing integrated chip according to another embodiment of the disclosure;
fig. 3 is a schematic structural diagram of a bias voltage generation module according to another embodiment of the present disclosure;
fig. 4 is a schematic circuit diagram of an infrared sensing and computing integrated chip according to another embodiment of the disclosure;
fig. 5 is a flowchart of a working method of an infrared sensing and calculation integrated chip according to another embodiment of the present disclosure;
fig. 6 is a flowchart of step 520 in a method for operating an infrared sensing and computing integrated chip according to another embodiment of the disclosure.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present disclosure more apparent, the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. However, it will be appreciated by those of ordinary skill in the art that in various embodiments of the disclosure, numerous technical details are set forth in order to provide a better understanding of the disclosure. However, the technical solution claimed in the present disclosure can be implemented without these technical details and various changes and modifications based on the following embodiments. The following embodiments are divided for convenience of description, and should not constitute any limitation to the specific implementation of the present disclosure, and the embodiments may be mutually incorporated and referred to without contradiction.
One embodiment of the disclosure relates to an infrared sensing and calculation integrated chip, which comprises an infrared detector array and a sensing and calculation integrated processing chip electrically connected with the infrared detector array.
Referring to fig. 1 and 2 together, the infrared detector array includes a plurality of infrared detectors arranged in an array form for detecting infrared thermal radiation signals and outputting current signals based on the infrared thermal radiation signals under the action of the sensing and calculating integrated processing chip.
Illustratively, the infrared detector may comprise a thermistor or thermistor type device. The infrared detector is used as a core component of the infrared sensing and calculating integrated chip, and resistance change can occur after receiving infrared heat radiation signals, so that the sensing of the infrared heat radiation signals is realized by utilizing the photoresistance characteristics of the infrared detector comprising thermosensitive or photosensitive resistor devices, and the infrared heat radiation power is represented by the resistance change of the infrared detector comprising the thermosensitive or photosensitive resistor devices.
The resistance of an infrared detector can be characterized by its conductance. For example, as shown in fig. 1 and 2, when the infrared detector array is m rows and n columns, the conductance of the infrared detector in the 1 st row is respectively represented as g 11 、g 12 、g 13 、……、g 1n The conductance of the infrared detector of line 2 is denoted as g 21 、g 22 、g 23 、……、g 2n The conductance of the infrared detector of line 3 is respectively expressed as g 31 、g 32 、g 33 、……、g 3n … …, the conductance of the m-th row infrared detector is denoted g, respectively m1 、g m2 、g m3 、……、g mn 。
And the sensing and calculating integrated processing chip is used for controlling the infrared detector array to output current signals based on the infrared thermal radiation signals so as to obtain an operation result of convolution operation.
Specifically, the sensing and calculation integrated processing chip can configure the weight information related to the convolution operation to the infrared detector array in a voltage signal mode, so that the infrared detector array can output a current signal based on the weight information in the voltage signal mode and the infrared thermal radiation signal, and thus the operation result of the convolution operation is obtained.
It should be noted that the sense integrated processing chip can be implemented by a Complementary Metal Oxide Semiconductor (CMOS) standard process. The infrared detector array and the sensing and calculation integrated processing chip can realize monolithic interconnection and integration through a subsequent process mode.
Compared with the prior art, the method and the device have the advantages that the infrared detector array and the sensing and calculating integrated processing chip are combined together, the sensing and convolution operation of infrared heat radiation can be realized simultaneously through the single-chip integrated sensing and calculating integrated framework, the data carrying problem in the convolution operation is avoided, the data transmission bandwidth from the front-end sensor to the rear-end processing chip can be greatly saved, the problems of poor integration level, high data transmission bandwidth and the like caused by the fact that the existing intelligent image sensor adopts a discrete intelligent processing chip are effectively solved, the efficiency ratio of the convolution operation is improved, the size and the weight of the system are further reduced, the method and the device have the advantages of being high in integration level, high in efficiency ratio and small in weight, and the device can be easily carried on various edge side devices to realize all-day and all-weather real-time artificial intelligence application.
Illustratively, as shown in fig. 1, the sense all-in-one processing chip comprises a bias voltage generation module, a gating device array, a column amplification conversion module and a control module.
The first ends of the infrared detectors in each row are respectively and electrically connected with the bias voltage generation module and the control module through corresponding row output lines. That is, as shown in fig. 1, the first end of each infrared detector is electrically connected to the bias voltage generation module and the control module through the corresponding row output line, respectively.
The first end of each row of the gating devices is electrically connected with the second end of the corresponding row of the infrared detectors, the second end of each row of the gating devices is electrically connected with the corresponding amplification conversion module through the corresponding row output line, and the control end of each row of the gating devices is electrically connected with the control module.
Specifically, the bias voltage generation module is used for providing bias voltage for the infrared detector array, the bias voltage can be configured to weight information related to convolution operation, and the bias voltage is configured to the infrared detector array through the control module.
Illustratively, as shown in fig. 3, the bias voltage generating module may include a reference voltage generating module and a digital-to-analog converter (DAC). The reference voltage generating module generates reference voltage, the reference voltage is converted into programmable analog voltage through the digital-to-analog converter, and the programmable analog voltage is configured to the corresponding row output line under the control of the control module to provide corresponding bias voltage for each row of infrared detectors.
For example, as shown in fig. 3, when the infrared detector array has m rows and n columns, the infrared detectors in the 1 st, 2 nd, … … and m rows correspond to the bias voltage V respectively sk1 、V sk2 、……、V skm . For another example, as shown in fig. 1 and fig. 2, the bias voltage generating module may provide bias voltages V for the infrared detectors in the rows respectively sk1 、V sk2 、V sk3 、……、V skm The current signals on the column output lines corresponding to the infrared detectors in each column can be respectively represented as I 1 、I 2 、I 3 、……、I n 。
As shown in fig. 1, the gating device array includes a plurality of gating devices arranged in an array form, and each gating device corresponds to one infrared detector in the infrared detector array. That is to say, the gating devices in the gating device array correspond to the infrared detectors in the infrared detector array one by one to form an array cross structure. In the array cross structure, a first end of each infrared detector is electrically connected with a corresponding row output line, a second end of each infrared detector is electrically connected with a first end of a corresponding gating device, a second end of each gating device is electrically connected with a corresponding column output line, and a control end of each gating device is electrically connected with a control module.
The gating device is used for gating according to a preset gating mode under the control of the control module, adjusting the bias voltage through bias and gating control, and configuring the adjusted bias voltage to the corresponding infrared detector, so that the adjusted bias voltage is configured to the corresponding infrared detector as weight information related to convolution operation, and the configuration of the weight information is completed. It should be noted that the preset gating manner may be set according to actual needs, and the present embodiment does not limit this.
Illustratively, as shown in fig. 2, the gating device may be a MOS gate tube. And a gating device array consisting of MOS (metal oxide semiconductor) gating tubes and an infrared detector array form an integrated sensing and calculating cross array structure. And the drain electrode of the MOS gate tube is the first end of the gate device and is electrically connected with the second end of the corresponding infrared detector. And the source level of the MOS gate tube is the second end of the gate device and is electrically connected with the corresponding amplification conversion module through the corresponding column output line. The gate of the MOS gate tube is a control end of the gating device and is electrically connected with the control module, the gating of the infrared detector can be realized under the control of the control module, and the current output by the infrared detector can be finely adjusted.
In the embodiment, the MOS gate tube is used as the gate device, so that the size and the weight of the gate device array can be reduced, the energy consumption is further reduced, and the efficiency ratio is improved.
The control module is used for controlling the bias voltage generation module to apply bias voltage to the infrared detector array. Specifically, the control module may configure the bias voltage to the row output line by timing, so as to apply the bias voltage to the infrared detector array through the row output line.
The control module is also used for controlling the gating device array to gate according to a preset gating mode and adjusting the bias voltage so that the infrared detector array outputs a current signal under the combined action of the infrared thermal radiation signal and the adjusted bias voltage.
And the column amplification conversion module is used for amplifying the current signal and converting the amplified current signal into a voltage signal to obtain an operation result. Specifically, as shown in fig. 1 and 2, the column amplification conversion module includes a plurality of amplification conversion modules, each amplification conversion module corresponds to each column of gating devices, and is electrically connected to the second ends of the gating devices in the corresponding column through a corresponding column output line. Each amplification conversion module can amplify the current signal on the corresponding column output line respectively and convert the amplified current signal into a voltage signal, so that the current signal I on the column output line is realized 1 、I 2 、I 3 、……、I n Amplifying and converting the amplified currentThe signal is converted into a voltage signal to obtain the operation result of convolution operation.
The embodiment further avoids the problem of data handling during convolution operation, saves data transmission bandwidth and improves the efficiency ratio of convolution operation.
Illustratively, as shown in fig. 4, the amplification conversion module includes a capacitive transconductance amplifier and a sample-and-hold circuit, wherein: the input end of the capacitor transconductance amplifier is electrically connected with the corresponding column output line, and the output end of the capacitor transconductance amplifier is electrically connected with the input end of the sampling hold circuit.
Specifically, the capacitive transconductance amplifier is configured to amplify a current signal on the column output line and convert the amplified current signal into a voltage signal. As shown in fig. 4, the sample-and-hold circuit includes a sampling capacitor and a switch (not shown in the figure) for maintaining the stability of the voltage signal before performing analog-to-digital conversion on the voltage signal output by the capacitor transconductance amplifier, thereby ensuring the accuracy of the analog-to-digital conversion.
Illustratively, in conjunction with fig. 1, the infrared sensor-integrated chip further includes a buffer (not shown) and an analog-to-digital converter (ADC), and the ADC is electrically connected to an output terminal of the sample-and-hold circuit (not shown) through the buffer.
Specifically, in conjunction with FIG. 4, the sample-and-hold circuit outputs a voltage signal V out Voltage signal V out The voltage signal V is input into an analog-to-digital converter after being acted by a buffer out And converting the digital signals into digital signals to realize digital quantization of the operation result.
The buffer is arranged between the sampling holding circuit and the analog-to-digital converter, so that the driving capability can be increased, and the capacity with load can be improved, thereby improving the signal quality and further improving the precision of the analog-to-digital conversion.
The circuit principle of the infrared integrated chip shown in fig. 1 will be described below with reference to fig. 2 and 4.
The first end of each infrared detector in the infrared detector array is respectively and electrically connected with the bias voltage generation module and the control module through the corresponding row output line so as to enable the bias voltage to be generatedThe voltage generation module generates a bias voltage V under the control of the control module sk1 、V sk2 、V sk3 、……、V skm And a bias voltage V is applied through the corresponding row output line sk1 、V sk2 、V sk3 、……、V skm Respectively configured to the infrared detectors of the corresponding rows.
When the gating devices in the gating device array are MOS gating tubes, the drain electrodes of the MOS gating tubes are respectively and electrically connected with the second ends of the corresponding infrared detectors, the source electrodes of the MOS gating tubes are respectively and electrically connected with the corresponding column output lines, the grid electrodes of the MOS gating tubes are respectively and electrically connected with the control module, and the control module enables grid voltage V to be applied to the grid electrodes FID1 、……、V FIDm And the MOS gate tubes are respectively configured to the corresponding rows, so that the MOS gate tubes are controlled to gate according to a preset gating mode, the bias voltage is adjusted, and the adjusted bias voltage is configured to the corresponding infrared detector.
Each column output line is electrically connected with the input end of a corresponding capacitor transconductance amplifier in the amplification and conversion module, the output end of each capacitor transconductance amplifier is electrically connected with the input end of a corresponding sample-and-hold circuit, and the output end of the sample-and-hold circuit is electrically connected with the analog-to-digital converter through a buffer, so that current signals output by each column output line are amplified and converted into voltage signals through each capacitor transconductance amplifier, and digital quantization of the voltage signals is realized through the sample-and-hold circuit, the buffer and the analog-to-digital converter.
Another embodiment of the present disclosure relates to a working method of an infrared sensing and calculation integrated chip, which is applied to the infrared sensing and calculation integrated chip provided in the above embodiment of the present disclosure. For a specific structure of the infrared sensing and calculation integrated chip, reference may be made to the above description of the embodiment of the infrared sensing and calculation integrated chip, and details are not described here.
As shown in fig. 5, the working method of the infrared sensing and computing integrated chip includes the following steps:
in step 510, the infrared detector array detects infrared thermal radiation signals.
Specifically, the resistance of the infrared detector in the infrared detector array changes after receiving the infrared thermal radiation signal, so that the infrared thermal radiation signal is sensed by utilizing the photoresistance characteristic of the infrared detector.
And step 520, controlling the infrared detector array to output current signals based on the infrared thermal radiation signals by the sensing and calculating integrated processing chip to obtain a calculation result of convolution operation.
Specifically, the integrated sensing and calculating processing chip can configure the weight information related to convolution operation to the infrared detector array in a voltage signal mode, so that the infrared detector array outputs a current signal by combining the weight information in the form of the voltage signal on the basis of resistance change of the infrared thermal radiation signal.
Compared with the prior art, the infrared sensing and calculating integrated chip formed by combining the infrared detector array and the sensing and calculating integrated processing chip can simultaneously realize the sensing and convolution operation of infrared heat radiation, effectively solves the problems of data carrying, high data transmission bandwidth and the like caused by the convolution operation of the discrete intelligent processing chip of the existing intelligent image sensor, improves the efficiency ratio of the convolution operation, and can be applied to end-side equipment to realize all-day and all-weather real-time artificial intelligent learning tasks.
Illustratively, when the sense-all-in-one processing chip comprises a bias voltage generating module, a gating device array, a column amplification conversion module and a control module, as shown in fig. 6, the step 520 comprises the following steps:
and step 521, the control module controls the bias voltage generation module to apply bias voltage to the infrared detector array. Specifically, the control module may first control the bias voltage generation module to generate the bias voltage. Then, the control module controls the bias voltage generation module to apply a bias voltage to the infrared detector array through the row output line.
For example, when the offset voltage generating module includes a reference voltage generating module and a digital-to-analog converter, the reference voltage generating module may be used to generate a reference voltage, the digital-to-analog converter may be used to convert the reference voltage into a programmable analog voltage, and the control module may be used to configure the programmable analog voltage to a corresponding row output line as the offset voltage corresponding to the infrared detector array.
And 522, controlling the gating device array to gate according to a preset gating mode by the control module, adjusting the bias voltage, and configuring the adjusted bias voltage to the infrared detector array.
Specifically, the control module may control the gating device array to gate according to a predetermined gating manner, adjust the bias voltage through bias and gating control, and configure the adjusted bias voltage to the infrared detector array.
It should be noted that, in this embodiment, the specific forms of the control mode and the preset gating mode are not limited, as long as the gating device array can gate and read the current signal output by the infrared detector array according to the preset gating mode under the control of the control module.
Specifically, the infrared detectors in the infrared detector array generate resistance changes after receiving infrared thermal radiation signals, and output current signals based on the changed resistances under the action of the adjusted bias voltage.
In step 524, the column amplification conversion module amplifies the current signal and converts the amplified current signal into a voltage signal to obtain an operation result.
Specifically, the column amplification and conversion module may amplify a current signal output by each column output line, and convert the amplified current signal into a voltage signal.
For example, when the column amplification and conversion module includes a plurality of amplification and conversion modules, each amplification and conversion module corresponds to each column gating device and is electrically connected to the second end of the corresponding column gating device through a corresponding column output line, the current signal output by each column output line may be amplified by each amplification and conversion module, and the amplified current signal may be converted into a voltage signal.
For example, when the amplification and conversion module includes a capacitive transconductance amplifier and a sample-and-hold circuit, the capacitive transconductance amplifier may amplify a current signal output by a corresponding column output line, and convert the amplified current signal into a voltage signal.
Illustratively, when the infrared calculation integrated chip further includes a buffer and an analog-to-digital converter, the operating method of the infrared calculation integrated chip further includes the following steps:
and inputting the voltage signal output by the column amplification conversion module into an analog-to-digital converter after the action of the buffer to obtain a digital signal corresponding to the voltage signal.
Illustratively, after the digital signal is obtained, the digital signal can be output to the outside of the chip, so that the digital signal can be subjected to subsequent processing according to actual needs.
The following describes a calculation principle of the infrared sensing and calculating integrated chip to realize convolution operation with reference to fig. 1.
Will be the first in the infrared detector arrayiGo to the firstjThe conductance of the infrared detector of the column is expressed asg ij ，i=1,2,…,mIndicating the number of rows in the infrared detector array,mrepresenting the number of rows of the infrared detector array,j=1,2,…,nindicating the column number in the infrared detector array,nindicating the number of columns of the infrared detector array. I.e. the size of the infrared detector array ism×nThe conductance of the infrared detector of line 1 is respectively expressed as g 11 、g 12 、g 13 、……、g 1n The conductance of the infrared detector of line 2 is denoted as g 21 、g 22 、g 23 、……、g 2n The conductance of the infrared detector of line 3 is respectively expressed as g 31 、g 32 、g 33 、……、g 3n … …, the conductance of the m-th row infrared detector is denoted g, respectively m1 、g m2 、g m3 、……、g mn . Wherein the content of the first and second substances,g ij may be determined based on the infrared thermal radiation signal detected by the corresponding infrared detector.
Will be the first in the infrared detector arrayiLine infrared detectionThe voltage across the device is represented asV ini The numbers 1,2, …,mthe voltages at the two ends of the line infrared detector can be respectively expressed asV in1 、V in2 、……、V inm . Voltage acrossV ini Is the firstiAnd correspondingly adjusting the bias voltage of the line infrared detector.
Will be the first in the infrared detector arrayjThe current signal on the column output line corresponding to the column infrared detector is expressed asI j The numbers 1,2, …, nthe current signals on the column output lines corresponding to the column infrared detectors can be respectively represented as I 1 、I 2 、……、I n . According to kirchhoff's law, the column current signal I 1 、I 2 、……、I n Can be respectively expressed as:
converting the current signal obtained according to kirchhoff's law into a matrix form, which can be expressed as the following formula (1):
in particular, in order to realize the matrix multiplication operation which is a key step in the convolution operation, the matrix multiplication operation can be realized by configuring a plurality of groups of voltages at two ends of the infrared detector array, namely the adjusted bias voltage.
Exemplary, orderk=1,2,…,t，tIndicating a preset number of groups, and arranging the infrared detectors in the arrayiFirst of the line infrared detectorkThe voltage across the group is represented asV inki In the infrared detector arrayjA second one on the column output line corresponding to the column infrared detectorkThe group current signal is represented asI jk . Then, the 1,2, …,mthe voltage across the 1 st group of the line infrared detector can be expressed asV in11 、V in12 、……、V in1m 1,2, …,mthe voltage across the 2 nd group of the line infrared detector can be expressed asV in21 、V in22 、……、V in2m … …, 1,2, …,mfirst of the line infrared detectortThe voltage across the stack can be expressed asV int1 、V int2 、……、V intm . 1,2, …, nthe 1 st set of current signals on the column output lines corresponding to the column infrared detectors can be represented as I 11 、I 21 、……、I n1 1,2, …, nthe 2 nd set of current signals on the column output lines corresponding to the column infrared detectors can be represented as I 12 、I 22 、……、I n2 … …, 1,2, …, na second row output line corresponding to the row infrared detectortThe group current signal can be represented as I t1 、I t2 、……、I tn 。
The sets of current signals can be obtained based on the above formula (1) according to kirchhoff's law, and the sets of current signals are combined together and can be expressed as the following formula (2):
through the combination of multiple groups of voltages at two ends and multiple gating modes, the configuration of multiple kinds of weight information related to convolution operation can be realized, so that the corresponding convolution operation is completed based on the configured weight information, and a processing result after the convolution operation is obtained.
It will be understood by those of ordinary skill in the art that the foregoing embodiments are specific embodiments for practicing the present disclosure, and that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure in practice.
Claims (8)
1. An infrared sensing and calculation integrated chip is characterized by comprising an infrared detector array and a sensing and calculation integrated processing chip electrically connected with the infrared detector array;
the infrared detector array is used for detecting infrared thermal radiation signals and outputting current signals based on the infrared thermal radiation signals under the action of the sensing and calculation integrated processing chip;
the sensing and calculating integrated processing chip is used for controlling the infrared detector array to output the current signal based on the infrared thermal radiation signal to obtain an operation result of convolution operation;
the sensing and computing integrated processing chip comprises a bias voltage generation module, a gating device array, a column amplification conversion module and a control module, wherein:
first ends of the infrared detectors in each row are respectively and electrically connected with the bias voltage generation module and the control module through corresponding row output lines;
the first end of each row of the gating device is electrically connected with the second end of the infrared detector in the corresponding row, the second end of each row of the gating device is electrically connected with the corresponding amplification conversion module through the corresponding row output line, and the control end of each row of the gating device is electrically connected with the control module; wherein the content of the first and second substances,
the control module is used for controlling the bias voltage generation module to apply bias voltage to the infrared detector array; and the number of the first and second groups,
the control module is further configured to control the gating device array to gate according to a preset gating manner, and adjust the bias voltage, so that the infrared detector array outputs the current signal under the combined action of the infrared thermal radiation signal and the adjusted bias voltage;
the column amplification conversion module is used for amplifying the current signal and converting the amplified current signal into a voltage signal to obtain the operation result.
2. The infrared sensing and calculation integrated chip according to claim 1, wherein the gating device adopts an MOS gate tube;
the drain electrode of the MOS gate tube is the first end of the gating device, the source stage of the MOS gate tube is the second end of the gating device, and the grid electrode of the MOS gate tube is the control end of the gating device.
3. The infrared integrated sensor chip of claim 2, wherein the amplification conversion module comprises a capacitive transconductance amplifier and a sample-and-hold circuit, wherein:
the input end of the capacitor transconductance amplifier is electrically connected with the corresponding column output line, and the output end of the capacitor transconductance amplifier is electrically connected with the input end of the sampling hold circuit.
4. The infrared algoritic chip of claim 3, characterized in that it further comprises a buffer and an analog-to-digital converter, said analog-to-digital converter being electrically connected to the output of the sample-and-hold circuit through said buffer.
5. An infrared sensory and computational integrated chip according to any one of claims 1 to 4, characterized in that the infrared detector comprises a thermistor or thermistor-like device.
6. An operating method of an infrared sensory and computational integrated chip, which is applied to the infrared sensory and computational integrated chip of any one of claims 1 to 5, the operating method comprising the steps of:
the infrared detector array detects infrared thermal radiation signals;
the sensing and calculating integrated processing chip controls the infrared detector array to output current signals based on the infrared thermal radiation signals to obtain a calculation result of convolution operation;
when the integrated sensing and computing processing chip comprises a bias voltage generation module, a gating device array, a column amplification conversion module and a control module,
the sensing and calculating integrated processing chip controls the infrared detector array to output current signals based on the infrared thermal radiation signals to obtain a calculation result of convolution operation, and the sensing and calculating integrated processing chip comprises the following steps of:
the control module controls the bias voltage generation module to apply bias voltage to the infrared detector array;
the control module controls the gating device array to gate according to a preset gating mode, adjusts the bias voltage, and configures the adjusted bias voltage to the infrared detector array;
the infrared detector array outputs the current signal under the combined action of the infrared thermal radiation signal and the adjusted bias voltage;
the column amplification conversion module amplifies the current signal, converts the amplified current signal into a voltage signal, and obtains the operation result.
7. The method of claim 6, wherein the current signal is expressed according to kirchhoff's law as the following equation (1):
wherein the content of the first and second substances,g ij represents the second in the infrared detector arrayiGo to the firstjThe conductance of the infrared detectors of the column, and is determined on the basis of the infrared thermal radiation signals detected by the corresponding infrared detectors,i=1,2,…,mrepresents the number of rows in the infrared detector array,mrepresenting the number of rows of the infrared detector array,j=1,2,…,nrepresenting a column number in the infrared detector array,nrepresenting the number of columns of the infrared detector array,V ini represents the first in the infrared detector arrayiVoltage across the line infrared detector isiThe offset voltage after the adjustment corresponding to the line infrared detector,I j represents the second in the infrared detector arrayjAnd current signals on column output lines corresponding to the column infrared detectors.
8. The method of claim 6, wherein the current signal is expressed according to kirchhoff's law as the following equation (2):
wherein the content of the first and second substances,g ij represents the second in the infrared detector arrayiGo to the firstjThe conductance of the infrared detectors of the column, and is determined on the basis of the infrared thermal radiation signals detected by the corresponding infrared detectors,i=1,2,…,mrepresents the number of rows in the infrared detector array,mrepresenting the number of rows of the infrared detector array,j=1,2,…,nrepresenting a column number in the infrared detector array,nrepresenting the number of columns of the infrared detector array,V inki represents the second in the infrared detector arrayiFirst of the line infrared detectorkVoltage across the group, and isiThe first corresponding to the line infrared detectorkThe set of adjusted said bias voltages is then,I jk represents the second in the infrared detector arrayjA second row output line corresponding to the row infrared detectorkThe set of current signals is then used to generate a set of current signals,k=1,2,…,t，tindicating the number of sets set in advance.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211194541.3A CN115295544B (en) | 2022-09-29 | 2022-09-29 | Infrared sensing and calculation integrated chip and working method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211194541.3A CN115295544B (en) | 2022-09-29 | 2022-09-29 | Infrared sensing and calculation integrated chip and working method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115295544A CN115295544A (en) | 2022-11-04 |
| CN115295544B true CN115295544B (en) | 2022-12-20 |
Family
ID=83833642
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211194541.3A Active CN115295544B (en) | 2022-09-29 | 2022-09-29 | Infrared sensing and calculation integrated chip and working method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115295544B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112949848A (en) * | 2021-03-19 | 2021-06-11 | 清华大学 | Light sensing and calculation integrated display unit, display array, operation method and equipment thereof |
| CN113489925A (en) * | 2021-06-01 | 2021-10-08 | 中国科学院上海技术物理研究所 | Focal plane detector reading circuit for realizing convolution calculation |
| CN113596361A (en) * | 2021-08-02 | 2021-11-02 | 电子科技大学 | Sense-memory-computation integrated circuit structure for realizing positive and negative weight calculation in pixel |
| CN114199374A (en) * | 2021-11-22 | 2022-03-18 | 复旦大学 | Sensing and calculation integrated single photon imaging chip and method |
| CN114362738A (en) * | 2021-12-08 | 2022-04-15 | 中山大学 | Capacitance array-based sensing calculation system and control method thereof |
-
2022
- 2022-09-29 CN CN202211194541.3A patent/CN115295544B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112949848A (en) * | 2021-03-19 | 2021-06-11 | 清华大学 | Light sensing and calculation integrated display unit, display array, operation method and equipment thereof |
| CN113489925A (en) * | 2021-06-01 | 2021-10-08 | 中国科学院上海技术物理研究所 | Focal plane detector reading circuit for realizing convolution calculation |
| CN113596361A (en) * | 2021-08-02 | 2021-11-02 | 电子科技大学 | Sense-memory-computation integrated circuit structure for realizing positive and negative weight calculation in pixel |
| CN114199374A (en) * | 2021-11-22 | 2022-03-18 | 复旦大学 | Sensing and calculation integrated single photon imaging chip and method |
| CN114362738A (en) * | 2021-12-08 | 2022-04-15 | 中山大学 | Capacitance array-based sensing calculation system and control method thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115295544A (en) | 2022-11-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Long et al. | ReRAM-based processing-in-memory architecture for recurrent neural network acceleration | |
| CN112183739B (en) | Hardware architecture of memristor-based low-power-consumption pulse convolution neural network | |
| US20240170060A1 (en) | Data processing method based on memristor array and electronic apparatus | |
| CN111406265A (en) | Neural network circuit with non-volatile synapse array | |
| US11157803B2 (en) | Neuromorphic device including a synapse having a variable resistor and a transistor connected in parallel with each other | |
| US20220047200A1 (en) | Information processing circuit and information processing method | |
| WO2023045160A1 (en) | Data processing apparatus and data processing method | |
| KR20210154502A (en) | Neural network apparatus performing floating point operation and operating method of the same | |
| US20230113627A1 (en) | Electronic device and method of operating the same | |
| CN113596361B (en) | Sense-memory-computation integrated circuit structure for realizing positive and negative weight calculation in pixel | |
| CN114400031B (en) | Complement mapping RRAM (resistive random access memory) storage and calculation integrated chip and electronic equipment | |
| CN108712621B (en) | Moving object detection circuit based on memristor and CMOS | |
| WO2023217017A1 (en) | Variational inference method and device for bayesian neural network based on memristor array | |
| KR20220010362A (en) | Neural network apparatus and operating method of the same | |
| KR20220004430A (en) | Apparatus for performing in memory processing and computing apparatus having the same | |
| CN115295544B (en) | Infrared sensing and calculation integrated chip and working method thereof | |
| CN116601710A (en) | Dynamic configuration of sense circuits for simulating different operations in a resistor crossbar array | |
| Wei et al. | Emerging Memory-Based Chip Development for Neuromorphic Computing: Status, Challenges, and Perspectives | |
| CN114861902A (en) | Processing unit, operation method thereof and computing chip | |
| US11863899B2 (en) | CMOS image sensor, image sensor unit and signal transmission methods therefor | |
| CN115458005A (en) | Data processing method, integrated storage and calculation device and electronic equipment | |
| US11468146B2 (en) | Array of integrated pixel and memory cells for deep in-sensor, in-memory computing | |
| CN115471710B (en) | Infrared detection recognition system and method | |
| US11106971B2 (en) | Neuromorphic device and a synapse network including a post-synaptic neuron having a subtracting circuit | |
| US20240046083A1 (en) | Memristive cellular neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |