CN113612942A - Convolution vision image sensor - Google Patents
Convolution vision image sensor Download PDFInfo
- Publication number
- CN113612942A CN113612942A CN202110811321.XA CN202110811321A CN113612942A CN 113612942 A CN113612942 A CN 113612942A CN 202110811321 A CN202110811321 A CN 202110811321A CN 113612942 A CN113612942 A CN 113612942A
- Authority
- CN
- China
- Prior art keywords
- signal
- detection unit
- signal modulation
- pixel
- pixel structure
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 claims abstract description 70
- 230000000007 visual effect Effects 0.000 claims abstract description 24
- 229910052751 metal Inorganic materials 0.000 claims description 38
- 239000002184 metal Substances 0.000 claims description 38
- 230000003321 amplification Effects 0.000 claims description 14
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 14
- 230000010287 polarization Effects 0.000 claims description 9
- 239000000463 material Substances 0.000 claims description 7
- 230000003287 optical effect Effects 0.000 claims description 7
- OAICVXFJPJFONN-UHFFFAOYSA-N Phosphorus Chemical compound [P] OAICVXFJPJFONN-UHFFFAOYSA-N 0.000 claims description 6
- VYPSYNLAJGMNEJ-UHFFFAOYSA-N Silicium dioxide Chemical compound O=[Si]=O VYPSYNLAJGMNEJ-UHFFFAOYSA-N 0.000 claims description 6
- 239000011810 insulating material Substances 0.000 claims description 6
- 239000004065 semiconductor Substances 0.000 claims description 6
- SDDGNMXIOGQCCH-UHFFFAOYSA-N 3-fluoro-n,n-dimethylaniline Chemical compound CN(C)C1=CC=CC(F)=C1 SDDGNMXIOGQCCH-UHFFFAOYSA-N 0.000 claims description 3
- JBRZTFJDHDCESZ-UHFFFAOYSA-N AsGa Chemical compound [As]#[Ga] JBRZTFJDHDCESZ-UHFFFAOYSA-N 0.000 claims description 3
- 229910002601 GaN Inorganic materials 0.000 claims description 3
- 229910001218 Gallium arsenide Inorganic materials 0.000 claims description 3
- JMASRVWKEDWRBT-UHFFFAOYSA-N Gallium nitride Chemical compound [Ga]#N JMASRVWKEDWRBT-UHFFFAOYSA-N 0.000 claims description 3
- HITXEXPSQXNMAN-UHFFFAOYSA-N bis(tellanylidene)molybdenum Chemical compound [Te]=[Mo]=[Te] HITXEXPSQXNMAN-UHFFFAOYSA-N 0.000 claims description 3
- FFBGYFUYJVKRNV-UHFFFAOYSA-N boranylidynephosphane Chemical compound P#B FFBGYFUYJVKRNV-UHFFFAOYSA-N 0.000 claims description 3
- 229920001577 copolymer Polymers 0.000 claims description 3
- 229910052732 germanium Inorganic materials 0.000 claims description 3
- GNPVGFCGXDBREM-UHFFFAOYSA-N germanium atom Chemical compound [Ge] GNPVGFCGXDBREM-UHFFFAOYSA-N 0.000 claims description 3
- KQHQLIAOAVMAOW-UHFFFAOYSA-N hafnium(4+) oxygen(2-) zirconium(4+) Chemical compound [O--].[O--].[O--].[O--].[Zr+4].[Hf+4] KQHQLIAOAVMAOW-UHFFFAOYSA-N 0.000 claims description 3
- HFGPZNIAWCZYJU-UHFFFAOYSA-N lead zirconate titanate Chemical compound [O-2].[O-2].[O-2].[O-2].[O-2].[Ti+4].[Zr+4].[Pb+2] HFGPZNIAWCZYJU-UHFFFAOYSA-N 0.000 claims description 3
- 229910052451 lead zirconate titanate Inorganic materials 0.000 claims description 3
- CWQXQMHSOZUFJS-UHFFFAOYSA-N molybdenum disulfide Chemical compound S=[Mo]=S CWQXQMHSOZUFJS-UHFFFAOYSA-N 0.000 claims description 3
- RVTZCBVAJQQJTK-UHFFFAOYSA-N oxygen(2-);zirconium(4+) Chemical compound [O-2].[O-2].[Zr+4] RVTZCBVAJQQJTK-UHFFFAOYSA-N 0.000 claims description 3
- UGFMBZYKVQSQFX-UHFFFAOYSA-N para-methoxy-n-methylamphetamine Chemical compound CNC(C)CC1=CC=C(OC)C=C1 UGFMBZYKVQSQFX-UHFFFAOYSA-N 0.000 claims description 3
- 229920003229 poly(methyl methacrylate) Polymers 0.000 claims description 3
- 239000004926 polymethyl methacrylate Substances 0.000 claims description 3
- MUSFRELEIXGPKU-UHFFFAOYSA-N selanylidenepalladium Chemical compound [Pd]=[Se] MUSFRELEIXGPKU-UHFFFAOYSA-N 0.000 claims description 3
- 235000012239 silicon dioxide Nutrition 0.000 claims description 3
- 239000000377 silicon dioxide Substances 0.000 claims description 3
- 239000010409 thin film Substances 0.000 claims description 3
- 229910001928 zirconium oxide Inorganic materials 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 2
- 238000012545 processing Methods 0.000 abstract description 7
- 239000010410 layer Substances 0.000 description 58
- 230000004044 response Effects 0.000 description 23
- 238000003384 imaging method Methods 0.000 description 21
- 238000000034 method Methods 0.000 description 9
- 230000035945 sensitivity Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000013528 artificial neural network Methods 0.000 description 6
- 238000007781 pre-processing Methods 0.000 description 6
- 210000001525 retina Anatomy 0.000 description 5
- 230000000946 synaptic effect Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 210000001328 optic nerve Anatomy 0.000 description 3
- 238000012216 screening Methods 0.000 description 3
- 238000012546 transfer Methods 0.000 description 3
- 229910003090 WSe2 Inorganic materials 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 210000004027 cell Anatomy 0.000 description 2
- 238000005265 energy consumption Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000010365 information processing Effects 0.000 description 2
- 239000002346 layers by function Substances 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 229910052698 phosphorus Inorganic materials 0.000 description 2
- 210000000857 visual cortex Anatomy 0.000 description 2
- PNEYBMLMFCGWSK-UHFFFAOYSA-N aluminium oxide Inorganic materials [O-2].[O-2].[O-2].[Al+3].[Al+3] PNEYBMLMFCGWSK-UHFFFAOYSA-N 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013529 biological neural network Methods 0.000 description 1
- 230000033228 biological regulation Effects 0.000 description 1
- 239000000969 carrier Substances 0.000 description 1
- 229910052593 corundum Inorganic materials 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000005137 deposition process Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000009792 diffusion process Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 230000003278 mimic effect Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 229910021421 monocrystalline silicon Inorganic materials 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000000206 photolithography Methods 0.000 description 1
- 239000004033 plastic Substances 0.000 description 1
- 229920003023 plastic Polymers 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 230000002207 retinal effect Effects 0.000 description 1
- 239000000523 sample Substances 0.000 description 1
- 230000008054 signal transmission Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000004528 spin coating Methods 0.000 description 1
- 238000004544 sputter deposition Methods 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
- 210000000225 synapse Anatomy 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 229910001845 yogo sapphire Inorganic materials 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/40—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
- H04N25/46—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled by combining or binning pixels
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
- H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
- H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
- H01L27/144—Devices controlled by radiation
- H01L27/146—Imager structures
- H01L27/14601—Structural or functional details thereof
- H01L27/14603—Special geometry or disposition of pixel-elements, address-lines or gate-electrodes
- H01L27/14605—Structural or functional details relating to the position of the pixel elements, e.g. smaller pixel elements in the center of the imager compared to pixel elements at the periphery
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
- H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
- H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
- H01L27/144—Devices controlled by radiation
- H01L27/146—Imager structures
- H01L27/14601—Structural or functional details thereof
- H01L27/14609—Pixel-elements with integrated switching, control, storage or amplification elements
- H01L27/14612—Pixel-elements with integrated switching, control, storage or amplification elements involving a transistor
- H01L27/14616—Pixel-elements with integrated switching, control, storage or amplification elements involving a transistor characterised by the channel of the transistor, e.g. channel having a doping gradient
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Power Engineering (AREA)
- Electromagnetism (AREA)
- Condensed Matter Physics & Semiconductors (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Microelectronics & Electronic Packaging (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Solid State Image Pick-Up Elements (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
Abstract
The invention discloses a convolution visual image sensor, which comprises a pixel array, a signal processing unit and a signal processing unit, wherein the pixel array comprises a plurality of pixel structures, and each pixel structure comprises a detection unit and N signal modulation units; the N signal modulation units in the same pixel structure comprise a first signal modulation unit connected with the detection unit in the pixel structure and a second signal modulation unit connected with the detection unit in the adjacent pixel structure; the N signal modulation units are electrically connected in parallel and then are converged to form an output circuit of the pixel structure, and N is more than or equal to 2; the invention realizes that the electric signals acquired by the detection units in the adjacent pixel structures are transmitted to the pixel structures, different convolution weights are given to the pixel structures and the electric signals acquired by the adjacent pixel structures by the signal modulation units, and the actual pixel signals of the pixel structures are acquired after the convolution weights.
Description
Technical Field
The invention belongs to the technical field of image sensors, and particularly relates to a convolution vision image sensor.
Background
The invention of CCD and CMOS imaging sensor based on semiconductor technology opens up digital imaging technology and thoroughly changes the recording mode of human about image. However, the existing imaging system is inferior to human visual experience in terms of high-throughput data acquisition, real-time target analysis and decision making. This is because its imaging sensor elements (PN junction, schottky junction, avalanche diode, photoconductive device, etc.) have a specific response sensitivity and fixed connections after packaging, and merely undertake converting optical signals into electrical signals for transmission to a post-processor unit, and cannot achieve processing of visual information like in the retina. To mimic the information processing characteristics of the optic neural network, image data is typically transmitted to an external processor, such functions being obtained by means of artificial neural network algorithms such as the documents US 10,755,136B 2; US 10,460,231B 2; CN 108898191A. The architecture, in which image acquisition, storage and analysis are separated, allows a large amount of data to be transported between the levels, placing extremely high demands on the speed of data storage and the computing power of the computing chip, while at the same time creating an additional high energy consumption, for example, in document US 10,755,126B 2. Compared with the prior art, the human visual system has a simple structure, and the image information can be preprocessed through the retina, the optic nerve and the visual cortex, so that the data is greatly simplified in the transmission process, and the key characteristics are reserved. Studies have shown that the information transmitted into the posterior optic nerve and visual cortex through the retina is only about one percent of the retinal imaging data. The preprocessing and feature screening of the image information enable the human visual system to show efficient image classification and recognition capability, and can realize quick response and decision under lower energy consumption. By analogy with the structure and the characteristics of human eyes, the image preprocessing and screening algorithm is integrated in the imaging sensor, so that the application bottleneck of the existing digital imaging system in an artificial vision system is expected to be broken through, and the method is applied to the fields of national defense early warning, automatic driving, intelligent robots, vision restoration and the like.
The high efficiency of the human visual system comes from the high plasticity of the optic nerve network, so that the human visual system can evolve through learning and complete partial information preprocessing and feature screening in the retina. The photoelectric detector with response plasticity characteristic and the imaging sensor thereof can be constructed to realize human-like eye vision imaging. At present, the scheme for realizing integrated visual information processing is mainly to design a detection unit, and simulate synaptic weights in a biological neural network by using adjustable response sensitivity in a positive interval and a negative interval of the detection unit, so as to realize the functions of processing, encoding and decoding specific signals. For example, an artificial visual neural network scheme composed of adjustable PN junction detectors is provided in Nature,2020,579,62-66 by professor t.mueller of vienna science and technology university, the gate controls P, N doping states of different regions in two-dimensional WSe2 to modulate the direction of the PN junction and the built-in field intensity, and the detector with continuously adjustable sensitivity in positive and negative intervals is realizedAnd (4) units. Based on the detector with plastic response, the team embeds different response weights in unit pixels based on a sub-pixel structure, so that the high efficiency of a machine vision system is proved, and the machine vision system can be quickly converged to the recognition rate of 0.96 under the noise intensity of 0.3. Also, since this type of system avoids the conversion between analog and digital signals in existing digital imaging and analysis systems, it can achieve ultra-fast target identification within 10 ns. The university of Nanjing, Miao and the university of Massachusetts, Yang Jianhua, USA, collaboratively propose a visual image sensor based on positive and negative photoconductive response in Sci.adv.,2020,6, eaba 6173. The device adjusts WSe through external grid voltage2Photon-generated carriers in the channel are hBN and Al2O3The binding in the medium causes grating effect and positive and negative photoconduction, and has high response sensitivity. Different convolution kernels can be embedded in the sensor by utilizing adjustable positive and negative photoconductive response sensitivity among different pixels, so that convolution image preprocessing effects such as contrast enhancement, DOG (difference of gaussian), edge enhancement and the like are realized.
Currently, existing visual sensor solutions all rely on the adjustment of the imaging detection unit response sensitivity to simulate synaptic weights in neural networks. However, in this scheme, when imaging, the pixel weight needs to be adjusted repeatedly to complete the functions of classifying and identifying specific targets, and the versatility in different scenes is insufficient. The sub-pixel structure is also not beneficial to improving the spatial resolution and the signal-to-noise ratio. In contrast, the convolution type image sensor does not depend on specific external training to obtain the response weight, only applies the convolution kernel with specific weight combination according to different target preprocessing functions, and has better universality on partial information preprocessing functions simulating retina. However, the current approach of simulating synaptic weights using the response sensitivity of the probe unit reduces the effective density of pixels (the scale factor is the square of the convolution kernel size) when implementing a single-pair convolution imaging. If a complete pixel array image needs to be obtained, the convolution kernel weight center needs to be refreshed for multiple times, but the time difference in the process can cause the imaging distortion problem. In this respect, the existing solutions are very challenging for the integration and signal readout of high resolution imaging systems, and also severely limit their imaging efficiency, affecting the decision and feedback capability of the system.
Disclosure of Invention
In view of the above defects or improvement requirements of the prior art, the present invention provides a convolution visual image sensor, which aims to transmit an electrical signal acquired by a detection unit in an adjacent pixel structure to the present pixel structure, and use a signal modulation unit to give different convolution weights to the present pixel structure and the electrical signal acquired by the adjacent pixel structure, and obtain an actual pixel signal of the present pixel structure after convolution weighting, thereby solving the technical problem that the response sensitivity of the adjacent pixel needs to be refreshed for many times during imaging by using a detection unit response sensitivity simulation synapse weight mode and a dependent gate voltage regulation and control method so as to obtain single-pair convolution imaging.
To achieve the above object, according to one aspect of the present invention, there is provided a convolution vision image sensor including a pixel array including a plurality of pixel structures, each pixel structure including a detection unit and N signal modulation units; the N signal modulation units in the same pixel structure comprise a first signal modulation unit connected with the detection unit in the pixel structure and a second signal modulation unit connected with the detection unit in the adjacent pixel structure; the N signal modulation units are electrically connected in parallel and then are converged to form an output circuit of the pixel structure, and N is more than or equal to 2;
the detection unit is used for converting detected optical signals into electric signals and transmitting the electric signals to a signal modulation unit connected with the detection unit, the first signal modulation unit is used for carrying out positive and negative gain amplification on the electric signals acquired by the pixel structure of the detection unit to obtain a first weight current signal, the second signal modulation unit is used for carrying out positive and negative gain amplification on the electric signals acquired by the adjacent pixel structure to obtain a second weight current signal, and the output circuit is used for carrying out convolution weighting on the first weight current signal and the second weight current signal and then outputting the pixel signal of the pixel structure.
Preferably, the signal modulation unit comprises a metal connecting electrode, a bottom gate dielectric layer, a transistor channel layer, a metal contact electrode, a top gate dielectric layer and a top gate electrode from bottom to top; the detection unit is connected with the signal modulation unit through a metal connecting electrode.
Preferably, the material of the transistor channel layer is a bipolar semiconductor thin film, the top gate dielectric layer is a ferroelectric dielectric layer, and the bottom gate dielectric layer is an insulating material; the signal modulation unit is converted between a P type and an N type and is modulated in a memorability mode through residual polarization in the ferroelectric medium layer, and therefore positive and negative gain amplification of electric signals is achieved.
Preferably, the material of the transistor channel layer is one of tungsten selenide, molybdenum telluride, boron phosphide, black phosphorus and palladium selenide; the top gate dielectric layer is made of one of vinylidene fluoride-trifluoroethylene copolymer, lead zirconate titanate and hafnium zirconium oxide; the bottom gate dielectric layer is made of one of silicon dioxide, zirconium oxide and PMMA.
Preferably, the top gate dielectric layer is made of an insulating material; positive and negative gain amplification of the electrical signal is achieved by the application of a continuous top gate voltage.
Preferably, the detection unit is a PN junction detection unit, a schottky junction detection unit or a detection unit with a photoconductive structure, and the detection unit is made of one of molybdenum sulfide, gallium nitride, germanium, gallium arsenide and black phosphorus.
Preferably, one pixel structure includes 1 detection unit and 9 signal modulation units, the 9 signal modulation units are arranged in an array manner of 3 rows and 3 columns, the central 1 is a first signal modulation unit connected with the detection unit of the pixel structure, and the remaining 8 are second signal modulation units respectively connected with the detection units of the adjacent 8 pixel structures.
Preferably, when the detection unit is connected with the signal modulation unit through the metal connection electrodes, the connection mode includes a boundary connection mode or a diagonal connection mode, the boundary connection mode is that the plurality of metal connection electrodes are arranged on the same plane, projections of the plurality of metal connection electrodes on the plane are not overlapped, the diagonal connection mode is that the plurality of metal connection electrodes are respectively arranged on different planes, projections of the plurality of metal connection electrodes on the same plane are overlapped, and there is no contact between metal electrodes on different planes at the projection overlapping position.
In general, the above technical solutions contemplated by the present invention can achieve the following advantageous effects compared to the prior art.
(1) In the invention, a detection unit and N signal modulation units are integrated in one pixel structure, and the N signal modulation units are not only connected with the detection unit in the pixel structure, but also connected with the detection units in the adjacent pixel structures. Therefore, the electric signals acquired by the detection units in the adjacent pixel structures can be transmitted to the pixel structure in real time, different convolution weights are given to the electric signals acquired by the pixel structure and the adjacent pixel structures by the signal modulation units, and the actual pixel signals of the pixel structure are acquired after the electric signals with different weights are subjected to convolution weighting by the output circuit. When the N signal modulation units are electrically connected in parallel and then are converged to form the output circuit of the pixel structure, the total current of the parallel circuit is equal to the sum of the currents of all branches, and the current signals after being connected in parallel can realize direct weighted addition in real time; thereby realizing the convolution weighting process of the pixel structure. The problem that the response weight of the pixel needs to be updated for multiple times in the conventional convolution imaging is solved.
Therefore, the convolution image sensor can locally store the convolution kernel weight in the signal modulation unit, and improves the imaging efficiency. The image sensor avoids analog-digital signal conversion in the traditional sensor, so the convolution image sensor has higher signal processing speed and is suitable for high-speed vision sensors and machine vision systems.
(2) In the invention, the ferroelectric medium layer is adopted to adjust the conduction type of the channel of the optical fiber, and the residual polarization in the ferroelectric medium layer is used for realizing the conversion and the memory modulation between the P type and the N type, thereby obtaining the adjustment of signal amplification gain in a positive and negative interval, simulating the synaptic weight in a neural network, realizing the real-time switching of a convolution kernel through the gain adjustment of the signal modulation unit, and obtaining the real-time characteristic enhancement function of image information. Meanwhile, as the ferroelectric material has the characteristic of non-volatilization long-term retention, when the ferroelectric dielectric layer is adopted as the top gate dielectric layer, the conductive type of the channel can be adjusted by applying pulse voltage to the top gate electrode, the gate voltage does not need to be continuously retained, and the power consumption of the system is greatly reduced.
Drawings
FIG. 1 is a schematic structural diagram of functional layers of a convolution visual image sensor provided by an embodiment of the invention;
FIG. 2 is a cross-sectional view of the integrated structure of a detection unit and a signal modulation unit within a single pixel structure of the convolution vision image sensor provided by the embodiment of the present invention in FIG. 1;
fig. 3 (a) is a schematic diagram of modulating unit amplifying positive gain with ferroelectric polarization according to an embodiment of the present invention; fig. 3 (b) is a schematic diagram of amplifying a negative gain by the modulation unit of the ferroelectric polarization adjustment signal according to the embodiment of the present invention, and fig. 3 (c) is a schematic diagram of amplifying a positive gain by the modulation unit of the ferroelectric polarization adjustment signal according to the embodiment of the present invention; fig. 3 (d) is a schematic diagram of the operation of modulating unit to amplify negative gain with ferroelectric polarization according to the embodiment of the present invention;
FIG. 4 is a schematic diagram of the layout and connections within the different functional layers of the convolutional visual image sensor shown in FIG. 1;
FIG. 5 is a connection diagram within the detection layer for implementing inter-pixel response transfer for the convolutional visual image sensor of FIG. 1.
The same reference numbers will be used throughout the drawings to refer to the same or like elements or structures, wherein:
101-a sensing layer; 102-a signal modulation layer; 103 an output layer; 104-pixel structure; 105-a detection unit; 106-a signal modulation unit; 106 a-a first signal modulation unit; 106 b-a second signal modulation unit; 107-convolution weights; 201-metal connection electrode; 202-bottom gate dielectric layer; 203-transistor channel layer; 204-metal contact electrodes; 205-top gate dielectric layer; 206-top gate electrode.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
The embodiment of the invention provides a convolution vision image sensor, which is shown in fig. 1 and comprises a pixel array, a signal processing unit and a signal processing unit, wherein the pixel array comprises a plurality of pixel structures 104, and each pixel structure 104 comprises a detection unit 105 and N signal modulation units 106; the N signal modulation units 106 in the same pixel structure 104 include a first signal modulation unit 106a connected to the detection unit 105 in the pixel structure 104, and a second signal modulation unit 106b connected to the detection unit 105 in the adjacent pixel structure 104; the N signal modulation units 106 are electrically connected in parallel and then are converged to form an output circuit of the pixel structure 104, wherein N is more than or equal to 2; the detection unit 105 is configured to convert a detected optical signal into an electrical signal and transmit the electrical signal to a signal modulation unit 106 connected to the detection unit, the first signal modulation unit 106a is configured to perform positive and negative gain amplification on the electrical signal acquired by the pixel structure 104 to obtain a first weighted current signal, the second signal modulation unit 106b is configured to perform positive and negative gain amplification on the electrical signal acquired by the adjacent pixel structure 104 to obtain a second weighted current signal, and the output circuit is configured to perform convolution weighting on the first weighted current signal and the second weighted current signal and output the pixel signal of the pixel structure 104.
Specifically, when the N signal modulation units 106 form an array, the signal modulation units connect the detection units in the adjacent pixels to each other in a convolution kernel matrix manner. During imaging work, the detection unit converts an optical signal into a voltage signal and transmits the signal to the N signal modulation units 106, and the signal is subjected to weight modulation corresponding to the convolution kernel matrix and then is summarized and output as a current or voltage signal which is output as a pixel. When the N signal modulation units are electrically connected in series and then are converged to form the output circuit of the pixel structure, the total voltage of the series circuit is equal to the sum of the voltages, and the voltage signals after series can be directly weighted and summed in real time; thus, the convolution weighting process of the pixel structure is realized.
With continued reference to fig. 1, the convolution visual image sensor provided in this embodiment can be understood as a multi-layer structure, and after the optical signal is responded by the sensing layer 101, the signal modulation layer 102 performs convolution weighting on the responses of neighboring pixels, and finally forms an image signal output on the output layer 103. Convolution weights 107 are embodied by different amplification gains of corresponding transistors within modulation unit 106. When the analog-digital converter works, A-I pixel responses are modulated by the modulation unit 106 and then are input into the same electrode for output so as to simulate convolution operation.
Referring to fig. 2, the signal modulation unit 106 includes, from bottom to top, a metal connection electrode 201, a bottom gate dielectric layer 202, a transistor channel layer 203, a metal contact electrode 204, a top gate dielectric layer 205, and a top gate electrode 206; the detection unit 105 is connected to the signal modulation unit 106 through a metal connection electrode 201. The transistor channel layer 203 is made of a bipolar semiconductor thin film, the top gate dielectric layer 205 is a ferroelectric dielectric layer, and the bottom gate dielectric layer 202 is made of an insulating material; the signal modulation unit 106 realizes the transformation between the P type and the N type and the memory modulation through the residual polarization in the ferroelectric layer, thereby realizing the positive and negative gain amplification of the electric signal.
Specifically, the material of the transistor channel layer 203 is one of tungsten selenide, molybdenum telluride, boron phosphide, black phosphorus and palladium selenide; the top gate dielectric layer 205 is made of one of vinylidene fluoride-trifluoroethylene copolymer, lead zirconate titanate and hafnium zirconium oxide; the bottom gate dielectric layer 202 is made of one of silicon dioxide, zirconium oxide and PMMA.
The top gate dielectric layer 205 is made of an insulating material; positive and negative gain amplification of the electrical signal is achieved by the application of a continuous top gate voltage.
The detection unit 105 is a PN junction detection unit, a schottky junction detection unit or a photoconductive detection unit, and the detection unit 105 is made of one of molybdenum sulfide, gallium nitride, germanium, gallium arsenide and black phosphorus.
In the preparation process, the detection unit 105, the metal connection electrode 201, the bottom gate dielectric layer 202, the transistor channel layer 203, the metal contact electrode 204, the top gate dielectric layer 205 and the top gate electrode 206 are respectively constructed from bottom to top in a standard semiconductor process. For example, a PN junction type detecting unit 105 is constructed by a mask and diffusion doping process with single crystal silicon as a substrate, and a photoresponsive electric signal is connected to a signal modulating unit 106 in a neighboring pixel at the upper side thereof with a metal connecting electrode 201; further, a bottom gate dielectric layer 202, a WSe2 bipolar semiconductor channel layer 203 and a metal contact electrode 204 are sequentially constructed on the upper layer of the transistor, so that a transistor is formed to modulate and output the photoresponse voltage; then, the P (VDF-TrFE) ferroelectric dielectric layer 205 is continuously built by a spin coating process, and the top gate electrode 206 is prepared on the upper surface thereof by photolithography and sputtering deposition processes.
Fig. 3 (a) - (d) show the modulation principle of the response photovoltage of the detection unit 105 by the signal modulation unit 106 in the sensor array shown in fig. 1. To simulate positive and negative convolution weights in the modulation cell, the image sensor is operated with the top gate pulse voltages 301, 304 reversing the polarization 307, 308 within the ferroelectric layer, thereby modulating the conductivity type of the transistor channel P, N within the cell such that its transfer curve under bottom gate modulation exhibits either positive 309 or negative 310 gain. At this time, the photoresponsive voltages 302, 305 are applied to the signal modulation unit, which increases 303 or decreases 306 the output current, respectively.
One pixel structure 104 includes 1 detection unit 105 and 9 signal modulation units 106, the 9 signal modulation units 106 are arranged in an array of 3 rows and 3 columns, the central 1 is a first signal modulation unit 106a connected to the detection unit 105 of the pixel structure 104, and the remaining 8 are second signal modulation units 106b respectively connected to the detection units 105 of the adjacent 8 pixel structures 104.
When the detection unit 105 and the signal modulation unit 106 are connected through the metal connection electrode 201, the connection mode includes a boundary connection mode or a diagonal connection mode, the boundary connection mode is that a plurality of metal connection electrodes are arranged on the same plane, projections of the plurality of metal connection electrodes on the plane are not overlapped, the diagonal connection mode is that the plurality of metal connection electrodes are respectively arranged on different planes, projections of the plurality of metal connection electrodes on the same plane are overlapped, and there is no contact between metal electrodes on different planes at the projection overlapping position.
To illustrate the convolution weighting of the responses of neighboring pixels, the pixel structure of fig. 1 is further described below with reference to fig. 4. As shown in fig. 4, each pixel structure is composed of a detection layer 401, a signal modulation layer 402, and a top gate layer 403. The area shown at 404 is represented by the pixel a size and comprises a detection unit 408 and metal connections 405, 406. Wherein the metal connection 406 further extends to an electrode 407 within a neighboring pixel to pass the a-pixel response to the neighboring pixel for subsequent signal weighting. In contrast, the pixel a also includes metal electrodes 409 and 410 extending from neighboring pixels. In constructing the array, the connection structure shown in fig. 5 can be used to complete the response signal transmission between neighboring pixels. For any adjacent pixels a and E, their mutual signal transfer is accomplished by a pair of metal connections 501 and 502. In particular, for the array shown in FIG. 5, two types of connections are included, namely boundary connections 504 and diagonal connections 503. The boundary connection can be completed in the same layer because of no intercrossing, and the diagonal connection can be completed in an upper layer and a lower layer by adopting a through hole process because of the intercrossing.
As shown in fig. 4, in the signal modulation layer 402, the modulation transistors 411, 412, and 413 correspond to the metal connection terminals 405, 407, and 409 in the detection layer 401, respectively, and the optical response voltage modulates the threshold voltage and conductance of the transistor channel by a capacitive coupling method. Where 411, 413 gate voltage modulation inputs come from 405, 407, corresponding to the photoresponse voltage of the pixel 404. And 412 the gate voltage input 409 is from the neighbor pixel photoresponse voltage. For the 3 × 3 convolution kernel in this embodiment, there are 9 modulation transistors in the pixel area, and to complete the convolution weighting, all the signal modulation transistors in the pixel a area 404 are connected in parallel, and their modulation signals are summed up as the pixel output. A convolutional image sensor array constructed in accordance with the present invention provides a top gate electrode in the top gate layer 403 corresponding to the modulation transistor in the signal modulation layer 402 to vary the modulation transistor gain with the top gate pulse voltages 301,304 in the fig. 3 operating principle.
It should be noted that the size of the convolution kernel in the convolution image sensor may also be expanded to different sizes (e.g., 2x2, 4x4, etc.) or a plurality of signal modulation layers 402 and 403 may be applied in series to simulate the function of a multilayer convolution neural network.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (8)
1. A convolutive visual image sensor, characterized by comprising a pixel array comprising a number of pixel structures (104), each pixel structure (104) comprising a detection unit (105) and N signal modulation units (106); the N signal modulation units (106) in the same pixel structure (104) comprise a first signal modulation unit (106a) connected with the detection unit (105) in the pixel structure (104) and a second signal modulation unit (106b) connected with the detection unit (105) in the adjacent pixel structure (104); the N signal modulation units (106) are electrically connected in parallel and then are converged to form an output circuit of the pixel structure (104), and N is more than or equal to 2;
the detection unit (105) is configured to convert a detected optical signal into an electrical signal and transmit the electrical signal to a signal modulation unit (106) connected to the detection unit, the first signal modulation unit (106a) is configured to perform positive and negative gain amplification on the electrical signal acquired by the pixel structure (104) of the detection unit to obtain a first weight current signal, the second signal modulation unit (106b) is configured to perform positive and negative gain amplification on the electrical signal acquired by an adjacent pixel structure (104) to obtain a second weight current signal, and the output circuit is configured to perform convolution weighting on the first weight current signal and the second weight current signal and output the pixel signal of the pixel structure (104).
2. The convolutional visual image sensor of claim 1 wherein the signal modulating unit (106) comprises, from bottom to top, a metal connection electrode (201), a bottom gate dielectric layer (202), a transistor channel layer (203), a metal contact electrode (204), a top gate dielectric layer (205), and a top gate electrode (206); the detection unit (105) and the signal modulation unit (106) are connected by a metal connection electrode (201).
3. The convolutive visual image sensor of claim 2, wherein the material of the transistor channel layer (203) is a bipolar semiconductor thin film, the top gate dielectric layer (205) is a ferroelectric dielectric layer, and the bottom gate dielectric layer (202) is an insulating material; the signal modulation unit (106) realizes the transformation between a P type and an N type and the memorability modulation through the residual polarization in the ferroelectric layer, thereby realizing the positive and negative gain amplification of the electric signal.
4. The convolutional visual image sensor of claim 3 wherein the material of the transistor channel layer (203) is one of tungsten selenide, molybdenum telluride, boron phosphide, black phosphorus, and palladium selenide; the top gate dielectric layer (205) is made of one of vinylidene fluoride-trifluoroethylene copolymer, lead zirconate titanate and hafnium zirconium oxide; the bottom gate dielectric layer (202) is made of one of silicon dioxide, zirconium oxide and PMMA.
5. The convolutional visual image sensor of claim 2 wherein the top gate dielectric layer (205) is an insulating material; positive and negative gain amplification of the electrical signal is achieved by the application of a continuous top gate voltage.
6. The convolutional visual image sensor of claim 1 wherein said detection unit (105) is a PN junction type detection unit, a schottky junction type detection unit or a photoconductive type detection unit, and the material of said detection unit (105) is one of molybdenum sulfide, gallium nitride, germanium, gallium arsenide and black phosphorus.
7. The convolutional visual image sensor of claim 1, wherein one pixel structure (104) comprises 1 detection unit (105) and 9 signal modulation units (106), the 9 signal modulation units (106) are arranged in an array of 3 rows and 3 columns, the central 1 is a first signal modulation unit (106a) connected to the detection unit (105) of the pixel structure (104), and the remaining 8 are second signal modulation units (106b) respectively connected to the detection units (105) of the adjacent 8 pixel structures (104).
8. The convolution visual image sensor according to claim 2, wherein when the detection unit (105) and the signal modulation unit (106) are connected through the metal connection electrode (201), the connection mode includes a boundary connection mode or a diagonal connection mode, the boundary connection mode is a mode in which the plurality of metal connection electrodes are arranged on the same plane, projections of the plurality of metal connection electrodes on the plane do not coincide, the diagonal connection mode is a mode in which the plurality of metal connection electrodes are respectively arranged on different planes, projections of the plurality of metal connection electrodes on the same plane coincide, and there is no contact between metal electrodes on different planes at the position where the projections coincide.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110811321.XA CN113612942B (en) | 2021-07-19 | 2021-07-19 | Convolution vision image sensor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110811321.XA CN113612942B (en) | 2021-07-19 | 2021-07-19 | Convolution vision image sensor |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113612942A true CN113612942A (en) | 2021-11-05 |
| CN113612942B CN113612942B (en) | 2022-09-20 |
Family
ID=78337833
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110811321.XA Active CN113612942B (en) | 2021-07-19 | 2021-07-19 | Convolution vision image sensor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113612942B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6011295A (en) * | 1997-07-22 | 2000-01-04 | Foveonics, Inc. | Neural network active pixel cell |
| CN107147856A (en) * | 2017-03-30 | 2017-09-08 | 深圳大学 | A kind of pixel cell and its denoising method, dynamic visual sensor, imaging device |
| US20190052821A1 (en) * | 2016-03-14 | 2019-02-14 | Insightness Ag | A Vision Sensor, a Method of Vision Sensing, and a Depth Sensor Assembly |
| WO2020145142A1 (en) * | 2019-01-08 | 2020-07-16 | ソニー株式会社 | Solid-state image capturing element and signal processing method for same, and electronic instrument |
| CN111669527A (en) * | 2020-07-01 | 2020-09-15 | 浙江大学 | Convolution operation framework in CMOS image sensor |
| CN112511769A (en) * | 2020-11-05 | 2021-03-16 | 北京大学深圳研究生院 | Image sensor pixel circuit and image sensing array |
| US20210084246A1 (en) * | 2019-09-18 | 2021-03-18 | Sony Semiconductor Solutions Corporation | Solid-state imaging device and imaging device with combined dynamic vision sensor and imaging functions |
| US20210168314A1 (en) * | 2019-12-02 | 2021-06-03 | Sony Semiconductor Solutions Corporation | Solid-state imaging device and imaging device with combined dynamic vision sensor and imaging functions |
-
2021
- 2021-07-19 CN CN202110811321.XA patent/CN113612942B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6011295A (en) * | 1997-07-22 | 2000-01-04 | Foveonics, Inc. | Neural network active pixel cell |
| US20190052821A1 (en) * | 2016-03-14 | 2019-02-14 | Insightness Ag | A Vision Sensor, a Method of Vision Sensing, and a Depth Sensor Assembly |
| CN107147856A (en) * | 2017-03-30 | 2017-09-08 | 深圳大学 | A kind of pixel cell and its denoising method, dynamic visual sensor, imaging device |
| WO2020145142A1 (en) * | 2019-01-08 | 2020-07-16 | ソニー株式会社 | Solid-state image capturing element and signal processing method for same, and electronic instrument |
| US20210084246A1 (en) * | 2019-09-18 | 2021-03-18 | Sony Semiconductor Solutions Corporation | Solid-state imaging device and imaging device with combined dynamic vision sensor and imaging functions |
| US20210168314A1 (en) * | 2019-12-02 | 2021-06-03 | Sony Semiconductor Solutions Corporation | Solid-state imaging device and imaging device with combined dynamic vision sensor and imaging functions |
| CN111669527A (en) * | 2020-07-01 | 2020-09-15 | 浙江大学 | Convolution operation framework in CMOS image sensor |
| CN112511769A (en) * | 2020-11-05 | 2021-03-16 | 北京大学深圳研究生院 | Image sensor pixel circuit and image sensing array |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113612942B (en) | 2022-09-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Kim et al. | Bio‐inspired artificial vision and neuromorphic image processing devices | |
| Liao et al. | Neuromorphic vision sensors: Principle, progress and perspectives | |
| Ji et al. | Artificial perception built on memristive system: Visual, auditory, and tactile sensations | |
| JP2809954B2 (en) | Apparatus and method for image sensing and processing | |
| Lee et al. | In-sensor image memorization and encoding via optical neurons for bio-stimulus domain reduction toward visual cognitive processing | |
| US20230186060A1 (en) | Novel neuromorphic vision system | |
| US5255362A (en) | Photo stimulated and controlled imaging neural network | |
| CN113612942B (en) | Convolution vision image sensor | |
| Bae et al. | Hetero‐integrated InGaAs photodiode and oxide memristor‐based artificial optical nerve for in‐sensor NIR image processing | |
| Gong et al. | Bioinspired Artificial Visual System Based on 2D WSe2 Synapse Array | |
| Lange et al. | Optical neural chips | |
| Al-Hardan et al. | Unleashing the potential of gallium oxide: A paradigm shift in optoelectronic applications for image sensing and neuromorphic computing applications | |
| JP2021082818A (en) | Nonvolatile memory element and operation method for the same | |
| CN114531555B (en) | Bionic visual sensor with enhanced end-side profile | |
| US6011295A (en) | Neural network active pixel cell | |
| Wang et al. | Digital image processing realized by memristor-based technologies | |
| JP3607074B2 (en) | Apparatus for image sensing and processing | |
| KR102296316B1 (en) | Nonvolatile memory device and operating method of the same | |
| Liao et al. | A Gate Multiplexing Architecture‐Based Artificial Visual Sensor and Memory System | |
| US20220155454A1 (en) | Analysis portion, time-of-flight imaging device and method | |
| Wu et al. | CMOS-compatible retinomorphic Si photodetector for motion detection | |
| CN113091895A (en) | Method for realizing machine vision function through coordinatable photoelectric detector | |
| Rafiq et al. | Investigation of ReRAM Variability on Flow-Based Edge Detection Computing Using HfO 2-Based ReRAM Arrays | |
| JPH0490015A (en) | Optical neurocomputer | |
| CN113903856A (en) | Novel memristor based on neural network visual perception, and preparation method and application thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |