WO2022166188A1 - 光人工神经网络智能芯片、智能处理设备及制备方法 - Google Patents

光人工神经网络智能芯片、智能处理设备及制备方法 Download PDF

Info

Publication number
WO2022166188A1
WO2022166188A1 PCT/CN2021/115965 CN2021115965W WO2022166188A1 WO 2022166188 A1 WO2022166188 A1 WO 2022166188A1 CN 2021115965 W CN2021115965 W CN 2021115965W WO 2022166188 A1 WO2022166188 A1 WO 2022166188A1
Authority
WO
WIPO (PCT)
Prior art keywords
neural network
artificial neural
filter layer
optical
layer
Prior art date
Application number
PCT/CN2021/115965
Other languages
English (en)
French (fr)
Inventor
崔开宇
杨家伟
熊健
黄翊东
刘仿
冯雪
张巍
Original Assignee
清华大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 清华大学 filed Critical 清华大学
Publication of WO2022166188A1 publication Critical patent/WO2022166188A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/067—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using optical means
    • G06N3/0675—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using optical means using electro-optical, acousto-optical or opto-electronic means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/10—Interfaces, programming languages or software development kits, e.g. for simulating neural networks
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/145—Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissue
    • A61B5/14532—Measuring characteristics of blood in vivo, e.g. gas concentration, pH value; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid, cerebral tissue for measuring glucose, e.g. by tissue impedance measurement
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Definitions

  • the present application relates to the technical field of artificial intelligence, and in particular, to an optical artificial neural network intelligent chip, an intelligent processing device and a preparation method.
  • the existing intelligent recognition technology usually needs to image a person or object first, and then go through the steps of image preprocessing, feature extraction, and feature matching to realize the recognition of the person or object.
  • image preprocessing image preprocessing
  • feature extraction feature matching
  • feature matching feature matching
  • the embodiments of the present application provide an optical artificial neural network intelligent chip, an intelligent processing device and a preparation method.
  • an embodiment of the present application provides an optical artificial neural network smart chip, including: an optical filter layer, an image sensor, and a processor; the optical filter layer corresponds to the input layer of the artificial neural network and the input layer to the linear layer
  • the connection weight of the image sensor corresponds to the linear layer of the artificial neural network;
  • the processor corresponds to the nonlinear layer and the output layer of the artificial neural network;
  • the optical filter layer is disposed on the surface of the photosensitive region of the image sensor, the optical filter layer includes a light modulation structure, and the optical filter layer is used for entering the light through the light modulation structure pair
  • the incident light at different positions of the modulation structure is subjected to different spectral modulations respectively, so as to obtain incident light-carrying information corresponding to different positions on the surface of the photosensitive region;
  • the incident light-carrying information includes light intensity distribution information, spectral information , the angle information of the incident light and the phase information of the incident light
  • the image sensor is used to convert the incident light-carrying information corresponding to the different position points after being modulated by the optical filter layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processing
  • the electrical signal is an image signal modulated by the optical filter layer
  • the processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network.
  • optical artificial neural network intelligent chip is used for intelligent processing tasks of the target object;
  • the intelligent processing tasks include at least one or more of intelligent perception, intelligent recognition and intelligent decision-making tasks;
  • the reflected light, transmitted light and/or radiated light of the target object enters the trained optical artificial neural network smart chip, and the intelligent processing result of the target object is obtained; the intelligent processing result at least includes the intelligent perception result and the intelligent recognition result and/or one or more of the Smart Decision Results;
  • the trained optical artificial neural network smart chip refers to the optical artificial neural network smart chip including the trained optical modulation structure, image sensor and processor; the trained optical modulation structure, image sensor and processor refers to Using the input training samples and output training samples corresponding to the intelligent processing tasks, the optical artificial neural network smart chips including different light modulation structures, image sensors and processors with different fully connected parameters and nonlinear activation parameters are used to perform The trained light modulation structures, image sensors and processors that satisfy the training convergence conditions.
  • the different light modulation structures are obtained by using a computer.
  • the design and realization of the optical simulation design method are obtained by using a computer.
  • the light modulation structures in the optical filter layer include regular structures and/or irregular structures; and/or, the light modulation structures in the optical filter layer include discrete structures and/or continuous structures.
  • the optical filter layer has a single-layer structure or a multi-layer structure.
  • the light modulation structure in the optical filter layer includes a unit array composed of a plurality of micro/nano units, each micro/nano unit corresponds to one or more pixels on the image sensor; the structures of each micro/nano unit are the same or different.
  • micro-nano unit includes a regular structure and/or an irregular structure; and/or, the micro-nano unit includes a discrete structure and/or a continuous structure.
  • micro-nano unit includes a plurality of groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.
  • each group of micro-nano structure arrays has the function of broadband filtering or narrow-band filtering.
  • each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.
  • micro-nano unit has polarization independent properties.
  • micro-nano unit has quadruple rotational symmetry.
  • optical filter layer is composed of one or more filter layers
  • the filter layer is made of one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, and perovskite materials; and/or, the filter layer is made of photonic crystals, metasurfaces, random A filter layer prepared by one or more of a structure, a nanostructure, a metal surface plasmon SPP micro-nano structure, and a tunable Fabry-Perot resonant cavity.
  • the semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and/or, the nanometer
  • the structure includes one or more of nanodot 2D materials, nanopillar 2D materials, and nanowire 2D materials.
  • the thickness of the optical filter layer is 0.1 ⁇ ⁇ 10 ⁇ , where ⁇ represents the central wavelength of the incident light
  • the image sensor is any one or more of the following:
  • CMOS image sensor CIS charge coupled element CCD, single photon avalanche diode SPAD array and focal plane photodetector array.
  • the types of the artificial neural network include: feedforward neural network.
  • a light-transmitting medium layer is provided before the optical filter layer and the image sensor.
  • the image sensor is front-illuminated, and includes: a metal wire layer and a light detection layer arranged from top to bottom, and the optical filter layer is integrated on the side of the metal wire layer away from the light detection layer; or ,
  • the image sensor is back-illuminated and includes: a light detection layer and a metal line layer arranged from top to bottom, and the light filter layer is integrated on the side of the light detection layer away from the metal line layer.
  • an embodiment of the present application provides an intelligent processing device, including: the optical artificial neural network intelligent chip as described in the first aspect.
  • the intelligent processing device includes one or more of a smart phone, an intelligent computer, an intelligent identification device, an intelligent perception device, and an intelligent decision-making device.
  • an embodiment of the present application provides a method for preparing an optical artificial neural network smart chip, including:
  • the optical filter layer is used to perform different spectral modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain different spectral modulations on the surface of the photosensitive region.
  • the image sensor is used to convert the incident light-carrying information corresponding to the different position points after being modulated by the optical filter layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processing
  • the processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different positions to obtain the output signal of the artificial neural network; the electrical signal is the image signal modulated by the optical filter layer.
  • preparing an optical filter layer containing a light modulation structure on the surface of the photosensitive area of the image sensor including:
  • an optical filter layer containing a light modulation structure is obtained;
  • the input training samples and the output training samples corresponding to the intelligent processing task are used to analyze the optical modulation structure, image sensor and
  • the optical artificial neural network smart chip of the processor with different fully connected parameters and nonlinear activation parameters is trained to obtain an optical modulation structure, an image sensor and a processor that satisfy the training convergence conditions.
  • the embodiment of the present application also provides an optical artificial neural network face recognition chip, which is used for face recognition processing tasks, including: an optical filter layer, an image sensor and a processor; the optical filter layer corresponds to the artificial neural network. the input layer and the connection weight between the input layer and the linear layer, the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the nonlinear layer and the output layer of the artificial neural network;
  • the optical filter layer is disposed on the surface of the photosensitive region of the image sensor, the optical filter layer includes a light modulation structure, and the optical filter layer is used for entering the light through the light modulation structure pair
  • the incident light at different positions of the modulation structure is subjected to different spectral modulations respectively, so as to obtain incident light-carrying information corresponding to different positions on the surface of the photosensitive region;
  • the incident light-carrying information includes light intensity distribution information, spectral information , the angle information of the incident light and the phase information of the incident light; the incident light is the reflected light of the user's face;
  • the image sensor is used to convert the incident light-carrying information corresponding to the different position points after being modulated by the optical filter layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processing
  • the electrical signal is an image signal modulated by the optical filter layer
  • the processor is used to perform full connection processing and nonlinear activation processing on electrical signals corresponding to different position points to obtain a face recognition processing result.
  • the optical artificial neural network face recognition chip includes a trained light modulation structure, an image sensor and a processor;
  • the trained light modulation structures, image sensors and processors refer to the use of input training samples and output training samples corresponding to face recognition tasks, and the use of input training samples and output training samples corresponding to face recognition tasks.
  • the optical modulation structure, image sensor and processor that satisfy the training convergence condition are obtained by training the optical artificial neural network face recognition chip of the processor with different nonlinear activation parameters;
  • the input training samples are incident light reflected by different faces in different lighting environments; the output training samples include corresponding face recognition results.
  • the different light modulation structures are passed through. It is designed and implemented by means of computer optical simulation design.
  • the light modulation structures in the optical filter layer include regular structures and/or irregular structures; and/or, the light modulation structures in the optical filter layer include discrete structures and/or continuous structures.
  • the optical filter layer has a single-layer structure or a multi-layer structure.
  • the light modulation structure in the optical filter layer includes a unit array composed of a plurality of micro/nano units, each micro/nano unit corresponds to one or more pixels on the image sensor; the structures of each micro/nano unit are the same or different.
  • micro-nano unit includes a regular structure and/or an irregular structure; and/or, the micro-nano unit includes a discrete structure and/or a continuous structure.
  • micro-nano unit includes a plurality of groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.
  • each group of micro-nano structure arrays has the function of broadband filtering or narrow-band filtering.
  • each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.
  • micro-nano unit has polarization independent properties.
  • the polarization-independent micro-nano unit has quadruple rotational symmetry.
  • optical filter layer is composed of one or more filter layers
  • the filter layer is made of one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, and perovskite materials; and/or, the filter layer is made of photonic crystals, metasurfaces, random A filter layer prepared by one or more of a structure, a nanostructure, a metal surface plasmon SPP micro-nano structure, and a tunable Fabry-Perot resonant cavity.
  • the semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and/or, the nanometer
  • the structure includes one or more of nanodot 2D materials, nanopillar 2D materials, and nanowire 2D materials.
  • the thickness of the optical filter layer is 0.1 ⁇ ⁇ 10 ⁇ , where ⁇ represents the center wavelength of the incident light.
  • the embodiment of the present application also provides a face recognition device, including the optical artificial neural network face recognition chip as described above.
  • the embodiment of the present application also provides a preparation method of the optical artificial neural network face recognition chip as described above, including:
  • the optical filter layer is used to perform different spectral modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain different spectral modulations on the surface of the photosensitive region.
  • the image sensor is used to convert the incident light-carrying information corresponding to the different position points after being modulated by the optical filter layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processing
  • the processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain a face recognition processing result; the electrical signals are image signals modulated by the optical filter layer, so The incident light is the reflected light of the human face under different lighting environments.
  • the preparation method of the optical artificial neural network face recognition chip also includes: a training process for the optical artificial neural network face recognition chip, specifically including:
  • the optical artificial neural network face containing different light modulation structures, image sensors and processors with different fully connected parameters and different nonlinear activation parameters
  • the recognition chip is trained to obtain a light modulation structure, an image sensor and a processor that meet the training convergence conditions, and the light modulation structure, image sensor and processor that meet the training convergence conditions are used as the trained light modulation structure, image sensor and processor.
  • the different light modulation structures are passed through. It is designed and implemented by means of computer optical simulation design.
  • the embodiment of the present application also provides an optical artificial neural network blood sugar detection chip, which is used for blood sugar detection tasks, including: an optical filter layer, an image sensor and a processor; the optical filter layer corresponds to the input layer of the artificial neural network and The connection weight between the input layer and the linear layer, the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the nonlinear layer and the output layer of the artificial neural network;
  • the optical filter layer is disposed on the surface of the photosensitive region of the image sensor, the optical filter layer includes a light modulation structure, and the optical filter layer is used for entering the light through the light modulation structure pair
  • the incident light at different positions of the modulation structure is subjected to different spectral modulations respectively, so as to obtain incident light-carrying information corresponding to different positions on the surface of the photosensitive region;
  • the incident light-carrying information includes light intensity distribution information, spectral information , the angle information of the incident light and the phase information of the incident light; wherein, the incident light includes the reflected light and/or the transmitted light of the part to be measured on the human body;
  • the image sensor is used to convert the incident light-carrying information corresponding to the different position points after being modulated by the optical filter layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processing device;
  • the electrical signal is an image signal modulated by the optical filter layer;
  • the processor is used to perform full connection processing and nonlinear activation processing on electrical signals corresponding to different position points to obtain blood glucose detection results.
  • the optical artificial neural network blood sugar detection chip includes a trained light modulation structure, an image sensor and a processor;
  • the trained light modulation structures, image sensors and processors refer to the use of input training samples and output training samples corresponding to the blood glucose detection task, which include different light modulation structures, image sensors, and different fully connected parameters and different parameters.
  • the optical modulation structure, the image sensor and the processor satisfying the training convergence condition are obtained by training the optical artificial neural network blood glucose detection chip of the processor of the nonlinear activation parameter;
  • the input training samples include incident light reflected and transmitted by the parts to be measured on the human body with different blood sugar values; the output training samples include corresponding blood sugar values.
  • the different light modulation structures are obtained by using Design and implementation of computer optical simulation design.
  • the light modulation structures in the optical filter layer include regular structures and/or irregular structures; and/or, the light modulation structures in the optical filter layer include discrete structures and/or continuous structures.
  • the optical filter layer has a single-layer structure or a multi-layer structure.
  • the light modulation structure in the optical filter layer includes a unit array composed of a plurality of micro/nano units, each micro/nano unit corresponds to one or more pixels on the image sensor; the structures of each micro/nano unit are the same or different.
  • micro-nano unit includes a regular structure and/or an irregular structure; and/or, the micro-nano unit includes a discrete structure and/or a continuous structure.
  • micro-nano unit includes a plurality of groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.
  • each group of micro-nano structure arrays has the function of broadband filtering or narrow-band filtering.
  • each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.
  • micro-nano unit has polarization independent properties.
  • the polarization-independent micro-nano unit has quadruple rotational symmetry.
  • optical filter layer is composed of one or more filter layers
  • the filter layer is made of one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, and perovskite materials; and/or, the filter layer is made of photonic crystals, metasurfaces, random A filter layer prepared by one or more of a structure, a nanostructure, a metal surface plasmon SPP micro-nano structure, and a tunable Fabry-Perot resonant cavity.
  • the semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and/or, the nanometer
  • the structure includes one or more of nanodot 2D materials, nanopillar 2D materials, and nanowire 2D materials.
  • the thickness of the optical filter layer is 0.1 ⁇ ⁇ 10 ⁇ , where ⁇ represents the center wavelength of the incident light.
  • the embodiment of the present application also provides an intelligent blood glucose detector, including the above-mentioned optical artificial neural network blood glucose detection chip.
  • the embodiment of the present application also provides a preparation method of the above-mentioned optical artificial neural network blood glucose detection chip, including:
  • the optical filter layer is used to perform different spectral modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain different spectral modulations on the surface of the photosensitive region.
  • the image sensor is used to convert the incident light-carrying information corresponding to the different position points after being modulated by the optical filter layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processing
  • the processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different positions to obtain blood glucose detection results; the electrical signals are image signals modulated by the optical filter layer, and the incident The light includes reflected light and/or transmitted light of the body part to be measured.
  • the preparation method of the optical artificial neural network blood sugar detection chip also includes: a training process for the optical artificial neural network blood sugar detection chip, specifically including:
  • the optical artificial neural network blood glucose detection chip containing different light modulation structures, image sensors and processors with different fully connected parameters and different nonlinear activation parameters
  • the light modulation structure, image sensor and processor satisfying the training convergence condition are obtained by training, and the light modulation structure, image sensor and processor satisfying the training convergence condition are taken as the trained light modulation structure, image sensor and processor.
  • the different light modulation structures are obtained by using Design and implementation of computer optical simulation design.
  • the embodiment of the present application also provides an optical artificial neural network intelligent agricultural precision control chip, which is used for agricultural precision control intelligent processing tasks, including: an optical filter layer, an image sensor and a processor; the optical filter layer corresponds to the artificial neural network.
  • the optical filter layer is disposed on the surface of the photosensitive region of the image sensor, the optical filter layer includes a light modulation structure, and the optical filter layer is used for entering the light through the light modulation structure pair
  • the incident light at different positions of the modulation structure is subjected to different spectral modulations respectively, so as to obtain incident light-carrying information corresponding to different positions on the surface of the photosensitive region;
  • the incident light-carrying information includes light intensity distribution information, spectral information , the angle information of the incident light and the phase information of the incident light;
  • the incident light includes reflected light, transmitted light and/or radiated light of an agricultural object;
  • the agricultural object includes crops and/or soil;
  • the image sensor is used to convert the incident light-carrying information corresponding to the different position points after being modulated by the optical filter layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processing
  • the electrical signal is an image signal modulated by the optical filter layer
  • the processor is used to perform full connection processing and nonlinear activation processing on electrical signals corresponding to different position points to obtain agricultural precision control processing results;
  • the agricultural precision control intelligent processing tasks include one or more of soil fertility detection, pesticide spreading detection, trace element content detection, drug resistance detection, and crop growth status detection;
  • the agricultural precision control processing results include: : One or more of soil fertility test results, pesticide spray test results, trace element content test results, drug resistance test results, and crop growth status test results.
  • the optical artificial neural network intelligent agricultural precision control chip includes a trained light modulation structure, an image sensor and a processor;
  • the trained light modulation structures, image sensors and processors refer to the use of input training samples and output training samples corresponding to the agricultural precision control intelligent processing tasks, and the use of input training samples and output training samples corresponding to the agricultural precision control intelligent processing tasks.
  • the optical modulation structure, image sensor and processor that meet the training convergence conditions are obtained by training the optical artificial neural network intelligent agricultural precision control chip of the processor with full connection parameters and different nonlinear activation parameters;
  • the input training samples include incident light reflected, transmitted and/or radiated by soils with different fertility;
  • the output training samples include corresponding soil fertility;
  • the input training samples include incident light reflected, transmitted and/or radiated by soils with different pesticide application conditions; the output training samples include corresponding pesticide application conditions;
  • the input training samples include incident light reflected, transmitted and/or radiated by soils with different trace element content conditions; the output training samples include corresponding trace element content conditions;
  • the input training samples include incident light reflected, transmitted and/or radiated by soils with different drug resistances; the output training samples include corresponding drug resistances;
  • the input training samples include incident light reflected, transmitted and/or radiated by crops with different growth conditions; the output training samples include corresponding crop growth conditions.
  • the different light modulation structures It is designed and realized by adopting the computer optical simulation design method.
  • the light modulation structures in the optical filter layer include regular structures and/or irregular structures; and/or, the light modulation structures in the optical filter layer include discrete structures and/or continuous structures.
  • the optical filter layer has a single-layer structure or a multi-layer structure.
  • the light modulation structure in the optical filter layer includes a unit array composed of a plurality of micro/nano units, each micro/nano unit corresponds to one or more pixels on the image sensor; the structures of each micro/nano unit are the same or different.
  • micro-nano unit includes a regular structure and/or an irregular structure; and/or, the micro-nano unit includes a discrete structure and/or a continuous structure.
  • micro-nano unit includes a plurality of groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.
  • each group of micro-nano structure arrays has the function of broadband filtering or narrow-band filtering.
  • each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.
  • micro-nano unit has polarization independent properties.
  • micro-nano unit has quadruple rotational symmetry.
  • optical filter layer is composed of one or more filter layers
  • the filter layer is made of one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, and perovskite materials; and/or, the filter layer is made of photonic crystals, metasurfaces, random A filter layer prepared by one or more of a structure, a nanostructure, a metal surface plasmon SPP micro-nano structure, and a tunable Fabry-Perot resonant cavity.
  • the semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and/or, the nanometer
  • the structure includes one or more of nanodot 2D materials, nanopillar 2D materials, and nanowire 2D materials.
  • the thickness of the optical filter layer is 0.1 ⁇ ⁇ 10 ⁇ , where ⁇ represents the center wavelength of the incident light.
  • the embodiment of the present application also provides an intelligent agricultural control device, including the above-mentioned optical artificial neural network intelligent agricultural precision control chip.
  • Embodiments of the present application also provide a method for preparing an optical artificial neural network intelligent agricultural precision control chip as described above, including:
  • the optical filter layer is used to perform different spectral modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain different spectral modulations on the surface of the photosensitive region.
  • the image sensor is used to convert the incident light-carrying information corresponding to the different position points after being modulated by the optical filter layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processing
  • the processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different positions to obtain the agricultural precision control processing results; the electrical signals are image signals modulated by the optical filter layer, so
  • the incident light includes reflected light, transmitted light and/or radiated light of an agricultural object; the agricultural object includes crops and/or soil.
  • the preparation method of the optical artificial neural network intelligent agriculture precision control chip further includes: the training process of the optical artificial neural network intelligent agriculture precise control chip, specifically including:
  • the optical artificial neural network containing different light modulation structures, image sensors and processors with different fully connected parameters and different nonlinear activation parameters
  • the smart agriculture precision control chip is trained to obtain a light modulation structure, image sensor and processor that meet the training convergence conditions, and the light modulation structure, image sensor and processor that meet the training convergence conditions are used as the trained light modulation structure, image sensor and processor. processor.
  • the different light modulation structures It is designed and realized by adopting the computer optical simulation design method.
  • the embodiment of the present application also provides an optical artificial neural network smelting end point monitoring chip, which is used for the smelting end point monitoring task, including: an optical filter layer, an image sensor and a processor; the optical filter layer corresponds to the input of the artificial neural network layer and the connection weight from the input layer to the linear layer, the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the nonlinear layer and the output layer of the artificial neural network;
  • the optical filter layer is disposed on the surface of the photosensitive region of the image sensor, the optical filter layer includes a light modulation structure, and the optical filter layer is used for entering the light through the light modulation structure pair
  • the incident light at different positions of the modulation structure is subjected to different spectral modulations respectively, so as to obtain incident light-carrying information corresponding to different positions on the surface of the photosensitive region;
  • the incident light-carrying information includes light intensity distribution information, spectral information , the angle information of the incident light and the phase information of the incident light;
  • the incident light includes the reflected light, the transmitted light and/or the radiated light of the steelmaking furnace mouth;
  • the image sensor is used to convert the incident light-carrying information corresponding to the different position points after being modulated by the optical filter layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processing
  • the electrical signal is an image signal modulated by the optical filter layer
  • the processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain the monitoring result of the smelting end point;
  • the smelting end point monitoring task includes identifying the smelting end point, and the smelting end point monitoring result includes the smelting end point identification result.
  • the smelting end point monitoring task further includes identifying the carbon content and/or molten steel temperature during the smelting process, and the smelting end point monitoring result includes the identification result of the carbon content and/or molten steel temperature during the smelting process.
  • the optical artificial neural network smelting endpoint monitoring chip includes a trained light modulation structure, an image sensor and a processor;
  • the trained light modulation structures, image sensors and processors refer to using the input training samples and output training samples corresponding to the monitoring task of the smelting end point, to perform data analysis on the light modulation structures, image sensors, and fully-connected parameters including different light modulation structures, image sensors, and
  • the optical artificial neural network of the processor with different nonlinear activation parameters smelts the end point monitoring chip and obtains the optical modulation structure, the image sensor and the processor that satisfy the training convergence condition;
  • the input training sample includes the incident light reflected, transmitted and/or radiated from the steelmaking furnace mouth smelted to the end point and not smelted to the end point; the output training sample includes the determination result of whether the smelting to the end point is reached.
  • the input training sample also includes a steelmaking furnace port from smelting to different carbon contents and/or molten steel temperature. Reflected, transmitted and/or radiated incident light, the output training sample also includes the corresponding carbon content and/or molten steel temperature.
  • the different light modulation structures are passed through. It is designed and implemented by means of computer optical simulation design.
  • the light modulation structures in the optical filter layer include regular structures and/or irregular structures; and/or, the light modulation structures in the optical filter layer include discrete structures and/or continuous structures.
  • the optical filter layer has a single-layer structure or a multi-layer structure.
  • the light modulation structure in the optical filter layer includes a unit array composed of a plurality of micro/nano units, each micro/nano unit corresponds to one or more pixels on the image sensor; the structures of each micro/nano unit are the same or different.
  • micro-nano unit includes a regular structure and/or an irregular structure; and/or, the micro-nano unit includes a discrete structure and/or a continuous structure.
  • micro-nano unit includes a plurality of groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.
  • each group of micro-nano structure arrays has the function of broadband filtering or narrow-band filtering.
  • each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.
  • micro-nano unit has polarization independent properties.
  • micro-nano unit has quadruple rotational symmetry.
  • optical filter layer is composed of one or more filter layers
  • the filter layer is made of one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, and perovskite materials; and/or, the filter layer is made of photonic crystals, metasurfaces, random A filter layer prepared by one or more of a structure, a nanostructure, a metal surface plasmon SPP micro-nano structure, and a tunable Fabry-Perot resonant cavity.
  • the semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and/or, the nanometer
  • the structure includes one or more of nanodot 2D materials, nanopillar 2D materials, and nanowire 2D materials.
  • the thickness of the optical filter layer is 0.1 ⁇ ⁇ 10 ⁇ , where ⁇ represents the center wavelength of the incident light.
  • the embodiment of the present application also provides an intelligent smelting control device, which is characterized in that it includes the above-mentioned optical artificial neural network smelting end point monitoring chip.
  • the embodiment of the present application also provides a preparation method for monitoring the smelting end point of the optical artificial neural network as described above, including:
  • the optical filter layer is used to perform different spectral modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain different spectral modulations on the surface of the photosensitive region.
  • the image sensor is used to convert the incident light-carrying information corresponding to the different position points after being modulated by the optical filter layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processing
  • the processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different positions to obtain the monitoring result of the smelting end point;
  • the electrical signal is an image signal modulated by an optical filter layer, and the Incident light includes reflected light, transmitted light and/or radiated light from the steelmaking furnace mouth.
  • the preparation method of the optical artificial neural network smelting end point monitoring chip also includes: the training process of the optical artificial neural network smelting end point monitoring chip, specifically including:
  • the smelting endpoints of optical artificial neural networks containing different light modulation structures, image sensors and processors with different fully connected parameters and different nonlinear activation parameters were analyzed.
  • the monitoring chip is trained to obtain a light modulation structure, an image sensor and a processor that meet the training convergence conditions, and the light modulation structure, image sensor and processor that meet the training convergence conditions are used as the trained light modulation structure, image sensor and processor.
  • the different light modulation structures are passed through. It is designed and implemented by means of computer optical simulation design.
  • the optical artificial neural network smart chip, the smart processing device, and the preparation method provided by the embodiments of the present application realize a brand-new smart chip capable of realizing the artificial neural network function.
  • the optical filter layer is arranged on the image sensor.
  • the light filter layer includes a light modulation structure, and the light filter layer is used to perform different spectral modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to achieve different spectral modulations in the photosensitive area.
  • the surface of the area obtains the incident light-carrying information corresponding to different position points.
  • the image sensor is used to convert the incident light-carrying information corresponding to different position points into electrical signals corresponding to different position points.
  • the processor connected to the sensor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different positions to obtain the output signal of the artificial neural network.
  • the optical filter layer is used as the artificial neural network.
  • the image sensor is used as the linear layer of the artificial neural network.
  • the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight between the input layer and the linear layer, that is, in the smart chip.
  • the optical filter layer and the image sensor realize the related functions of the input layer and the linear layer in the artificial neural network, that is, the embodiments of the present application strip the input layer and the linear layer in the artificial neural network implemented by software in the prior art.
  • the two-layer structure of the input layer and the linear layer in the artificial neural network is realized by means of hardware, so that the subsequent use of the intelligent chip for artificial neural network intelligent processing does not require any more complicated processing corresponding to the input layer and the linear layer.
  • the signal processing and algorithm processing of the artificial neural network only need to be processed by the processor in the smart chip, which is fully connected to the electrical signal and related processing of nonlinear activation, which can greatly reduce the power consumption and delay of artificial neural network processing.
  • the optical filter layer is used as the input layer of the artificial neural network
  • the image sensor is used as the linear layer of the artificial neural network
  • the filtering effect of the optical filter layer on the incident light entering the optical filter layer is used as the input.
  • connection weight from the layer to the linear layer uses the optical filter layer and the image sensor to project the information carried by the incident light of the target object into an electrical signal, and then realizes the full connection processing and nonlinear activation processing of the electrical signal in the processor.
  • the embodiment of the present application can not only save the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art, but also the embodiment of the present application actually uses the image information, spectral information, incident light of the target object at the same time.
  • the angle and the phase information of the incident light that is, the light-carrying information at different points in the target object space.
  • the incident light-carrying information at different points in the target object space covers the image, composition, shape, and three-dimensional depth of the target object , structure and other information, so that when the identification processing is carried out according to the information carried by the incident light at different points in the target object space, it can cover the image, composition, shape, three-dimensional depth, structure and other multi-dimensional information of the target object, which can solve the background technology. It is difficult to ensure the accuracy of recognition by using the two-dimensional image information of the target object mentioned in part. For example, it is difficult to distinguish whether it is a real person or a picture. It can be seen that the optical artificial neural network chip provided in the embodiment of the present application can not only achieve low The effect of power consumption and low latency is also able to achieve high accuracy, which can be applied to intelligent processing tasks such as intelligent perception, recognition and/or decision-making.
  • FIG. 1 is a schematic structural diagram of an optical artificial neural network smart chip provided in a first embodiment of the present application
  • FIG. 2 is a schematic diagram of the identification principle of an optical artificial neural network intelligent chip provided by an embodiment of the present application
  • FIG. 3 is a schematic diagram of disassembly of an optical artificial neural network smart chip provided by an embodiment of the present application
  • FIG. 4 is a schematic diagram of a target object recognition process provided by an embodiment of the present application.
  • FIG. 5 is a top view of an optical filter layer provided by an embodiment of the present application.
  • FIG. 6 is a top view of another optical filter layer provided by an embodiment of the present application.
  • FIG. 7 is a top view of another optical filter layer provided by an embodiment of the present application.
  • FIG. 8 is a top view of another optical filter layer provided by an embodiment of the present application.
  • FIG. 9 is a top view of yet another optical filter layer provided by an embodiment of the present application.
  • FIG. 10 is a top view of yet another optical filter layer provided by an embodiment of the present application.
  • FIG. 11 is a schematic diagram of a broadband filtering effect of a micro-nano structure provided by an embodiment of the present application.
  • FIG. 12 is a schematic diagram of a narrow-band filtering effect of a micro-nano structure provided by an embodiment of the present application.
  • FIG. 13 is a schematic structural diagram of a front-illuminated image sensor provided by an embodiment of the present application.
  • FIG. 14 is a schematic structural diagram of a back-illuminated image sensor provided by an embodiment of the present application.
  • 15 is a schematic flowchart of a method for preparing an optical artificial neural network smart chip provided by the third embodiment of the present application.
  • 16 is a schematic diagram of a face recognition process provided by an embodiment of the present application.
  • FIG. 17 is a schematic diagram of a finger blood glucose detection process provided by an embodiment of the present application.
  • FIG. 18 is a schematic diagram of a wrist blood glucose detection process provided by an embodiment of the present application.
  • FIG. 19 is a schematic diagram of a process for identifying agricultural objects provided by an embodiment of the present application.
  • Fig. 20 is a three-dimensional schematic diagram of identifying or qualitatively analyzing crops and/or soil provided by an embodiment of the present application;
  • FIG. 21 is a schematic diagram of identifying the furnace mouth in the smelting process to determine the smelting end point according to an embodiment of the present application.
  • the existing intelligent recognition technology usually needs to image a person or object first, and then go through the steps of image preprocessing, feature extraction, and feature matching to realize the recognition of the person or object.
  • image preprocessing image preprocessing
  • feature extraction feature matching
  • feature matching feature matching
  • an embodiment of the present application provides an optical artificial neural network intelligent chip, wherein the optical filter layer in the intelligent chip corresponds to the input layer of the artificial neural network, the image sensor corresponds to the linear layer of the artificial neural network, and the optical filter layer corresponds to the input layer of the artificial neural network.
  • the filtering effect of the incident light of the optical filter layer corresponds to the connection weight between the input layer and the linear layer.
  • the optical filter layer and the image sensor are used to project the spatial spectral information of the target object into electrical signals, which are then implemented in the processor.
  • the embodiments of the present application can not only omit the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art, but also the practical embodiments of the present application.
  • the light intensity distribution information of the target object ie image information
  • spectral information, the angle of incident light and the phase information of the incident light are used at the same time, that is, the light carrying information at different points in the space of the target object, so that it can be obtained from the spatial image, spectrum , angle, and phase information, so that the accuracy of intelligent processing (such as intelligent recognition) can be improved.
  • optical artificial neural network chip provided by the embodiment of this application can not only achieve the effects of low power consumption and low delay, but also The accuracy of intelligent processing can be improved, so that it can be better applied to intelligent processing fields such as intelligent perception, identification and/or decision-making.
  • intelligent processing fields such as intelligent perception, identification and/or decision-making.
  • the optical artificial neural network smart chip provided by the first embodiment of the present application includes: an optical filter layer 1, an image sensor 2 and a processor 3;
  • the optical filter layer 1 corresponds to the artificial neural network
  • the input layer and the connection weight between the input layer and the linear layer, the image sensor 2 corresponds to the linear layer of the artificial neural network;
  • the processor 3 corresponds to the nonlinear layer and the output layer of the artificial neural network;
  • the optical filter layer 1 is disposed on the surface of the photosensitive region of the image sensor, the optical filter layer 1 includes a light modulation structure, and the optical filter layer 1 is used to enter the light-emitting diode through the light modulation structure pair.
  • the incident light at different positions of the light modulation structure is respectively subjected to spectral modulation of intensity modulation with wavelength changes, that is, different intensity modulation is performed on the incident light of different wavelengths, so as to obtain corresponding points corresponding to different positions on the surface of the photosensitive region.
  • the incident light-carrying information includes light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light;
  • the image sensor 2 is used to convert the incident light-carrying information corresponding to the different position points modulated by the optical filter layer 1 into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the the processor 3; the electrical signal is an image signal modulated by the optical filter layer;
  • the processor 3 is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different positions to obtain the output signal of the artificial neural network.
  • the optical filter layer 1 is disposed on the surface of the photosensitive region of the image sensor, the optical filter layer 1 includes a light modulation structure, and the optical filter layer 1 is used to pass the light modulation structure to the light modulation structure.
  • the incident light at the position points is subjected to different spectral modulation respectively, so as to obtain the information carried by the modulated incident light corresponding to the different position points on the surface of the photosensitive area.
  • the image sensor 2 is used for the corresponding
  • the information carried by the incident light is converted into electrical signals corresponding to different positions, that is, the image signals modulated by the optical filter layer.
  • the processor 3 is used to perform full connection processing on the electrical signals corresponding to different positions. Processed with nonlinear activation, the output signal of the artificial neural network is obtained.
  • the optical filter layer 1 includes a light modulation structure, and the light modulation structure correlates the incident light (such as reflected light, transmitted light, radiated light, etc. of the target to be identified) entering different positions of the light modulation structure.
  • Acting light performs spectral modulation with different intensities, so as to obtain information carried by the incident light corresponding to different positions on the surface of the photosensitive area of the image sensor 2 .
  • the modulation intensity is related to the specific structural form of the light modulation structure.
  • different light modulation structures can be designed (eg, by changing the shape and/or size parameters of the light modulation structure) to achieve different modulation strength.
  • the light modulation structures at different positions on the optical filter layer 1 have different spectral modulation effects on the incident light, and the modulation intensity of the light modulation structures on different wavelength components of the incident light corresponds to the artificial nerve
  • the connection strength of the network that is, the connection weight corresponding to the input layer and the input layer to the linear layer.
  • the optical filter layer 1 is composed of a plurality of optical filter units, and the optical modulation structure at different positions in each optical filter unit is different, so it has different spectral modulation effects on incident light;
  • the light modulation structures at different positions between the optical filter units may be the same or different, and thus have the same or different spectral modulation effects on the incident light.
  • the image sensor 2 converts the incident light-carrying information corresponding to different positions into electrical signals corresponding to different positions, and sends the electrical signals corresponding to different positions to the processor 3, and the image sensor 2 Corresponds to the linear layer of the neural network.
  • the processor 3 performs full connection processing and nonlinear activation processing on the electrical signals at different positions, thereby obtaining the output signal of the artificial neural network.
  • processor 3 corresponds to the nonlinear layer and the output layer of the neural network, and can also be understood to correspond to the remaining layers (all other layers) in the neural network except the input layer and the linear layer.
  • the processor 3 may be disposed in the smart chip, that is, the processor 3 may be disposed in the smart chip together with the filter layer 1 and the image sensor 2, or may be separately It is arranged outside the smart chip, and is connected to the image sensor 2 in the smart chip through a data line or a connecting device, which is not limited in this embodiment.
  • the processor 3 may be implemented by a computer, or by an ARM or FPGA circuit board with a certain computing capability, or by a microprocessor, which is not limited in this embodiment.
  • the processor 3 may be integrated in the smart chip, or may be provided independently of the smart chip. When the processor 3 is set independently of the smart chip, the electrical signal in the image sensor 2 can be read out to the processor 3 through the signal readout circuit, and then the processor 3 can read out the electrical signal. Full connection processing and nonlinear activation processing.
  • a nonlinear activation function may be used to implement, for example, a Sigmoid function, a Tanh function, a ReLU function, etc., which are not limited in this embodiment. .
  • the optical filter layer 1 corresponds to the input layer of the artificial neural network and the connection weight between the input layer and the linear layer
  • the image sensor 2 corresponds to the linear layer of the artificial neural network.
  • the processor 3 corresponds to the nonlinear layer and the output layer of the artificial neural network, fully connects the electrical signals at different positions, and obtains the output signal of the artificial neural network through the nonlinear activation function to realize the intelligent perception of specific targets. , identification and/or decision making.
  • the optical artificial neural network smart chip includes an optical filter layer 1, an image sensor 2 and a processor 3.
  • the processor 3 is implemented by a signal readout circuit and a computer.
  • the optical filter layer 1 in the optical artificial neural network smart chip corresponds to the input layer of the artificial neural network
  • the image sensor 2 corresponds to the linear layer of the artificial neural network
  • the processor 3 corresponds to the nonlinearity of the artificial neural network.
  • layer and output layer, the filtering effect of the optical filter layer 1 on the incident light entering the optical filter layer 1 corresponds to the connection weight between the input layer and the linear layer.
  • the optical filter layer in the smart chip provided in this embodiment
  • the related functions of the input layer and the linear layer in the artificial neural network are realized by hardware with the image sensor, so that the complex signal processing corresponding to the input layer and the linear layer does not need to be carried out when the intelligent chip is used for intelligent processing. and algorithm processing, which can greatly reduce the power consumption and delay of artificial neural network processing.
  • this embodiment utilizes the image information of the target object and the image information and spectral information at different points in space at the same time, the angle information of the incident light, the phase information of the incident light, etc., can more accurately realize the intelligence of the target object. deal with.
  • the processor 3 includes a signal readout circuit and a computer.
  • the signal readout circuit reads out the photocurrent response and transmits it to the computer, and the computer performs the full connection processing and nonlinear activation processing of the electrical signal, and finally outputs the result.
  • the light modulation structure on the optical filter layer 1 is integrated above the image sensor 2, modulates the incident light, and projects/connects the spectral information of the incident light to different pixels of the image sensor 2 to obtain a
  • the electrical signal of the incident light spectrum information and image information that is, after the incident light passes through the optical filter layer 1, is converted into an electrical signal by the image sensor 2 to form an image containing the spectral information of the incident light, and finally is connected to the image sensor 2.
  • the processor 3 processes electrical signals containing spectral information and image information of incident light.
  • the optical artificial neural network chip provided in this embodiment actually utilizes the image information, spectral information, angle of incident light and phase information of incident light of the target object at the same time, that is, the light-carrying information at different points in the target object space , it can be seen that since the information carried by the incident light at different points in the target object space covers the image, composition, shape, three-dimensional depth, structure and other information of the target object, the information carried by the incident light at different points in the target object space can be carried out according to the information carried by the incident light at different points in the space of the target object.
  • the recognition processing it can cover the image, composition, shape, three-dimensional depth, structure and other multi-dimensional information of the target object, so as to solve the problem that the two-dimensional image information of the target object mentioned in the background technology is difficult to ensure the accuracy of recognition. For example It is difficult to distinguish whether it is a real person or a picture. It realizes intelligent perception, recognition and/or decision-making functions for different application fields, and realizes a spectrum optical artificial neural network intelligent chip with low power consumption, low delay and high accuracy.
  • the optical artificial neural network smart chip provided by the embodiment of the present application includes an optical filter layer, an image sensor and a processor.
  • the optical filter layer is arranged on the surface of the photosensitive area of the image sensor, and the optical filter layer includes a light modulation structure.
  • the filter layer is used to perform different spectral modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain information carried by the incident light corresponding to different positions on the surface of the photosensitive area, and correspondingly Ground
  • the image sensor is used to convert the incident light-carrying information corresponding to different position points into electrical signals corresponding to different position points
  • the processor connected to the image sensor is used to convert the electrical signals corresponding to different position points.
  • the optical filter layer is used as the input layer of the artificial neural network
  • the image sensor is used as the linear layer of the artificial neural network.
  • the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight between the input layer and the linear layer, that is, the optical filter layer and the image sensor in the smart chip realize the input layer and the image sensor in the artificial neural network.
  • the related functions of the linear layer that is, the embodiment of the present application strips the input layer and the linear layer in the artificial neural network implemented by software in the prior art, and realizes the input layer and the linear layer in the artificial neural network by means of hardware.
  • the two-layer structure makes it unnecessary to perform complex signal processing and algorithm processing corresponding to the input layer and linear layer when using the smart chip for artificial neural network intelligent processing. It is enough to perform the related processing of full connection with the electrical signal and nonlinear activation, which can greatly reduce the power consumption and delay of the artificial neural network processing.
  • the optical filter layer is used as the input layer of the artificial neural network
  • the image sensor is used as the linear layer of the artificial neural network
  • the filtering effect of the optical filter layer on the incident light entering the optical filter layer is used as the input.
  • connection weight from the layer to the linear layer uses the optical filter layer and the image sensor to project the information carried by the incident light of the target object into an electrical signal, and then realizes the full connection processing and nonlinear activation processing of the electrical signal in the processor.
  • the embodiment of the present application can not only save the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art, but also the embodiment of the present application actually uses the image information, spectral information, incident light of the target object at the same time.
  • the angle information and the phase information of the incident light that is, the information carried by the incident light at different points in the space of the target object.
  • the information carried by the incident light at different points in the space of the target object covers the image, composition, shape, 3D depth, structure and other information, so that when the recognition processing is carried out according to the information carried by the incident light at different points in the target object space, it can cover the image, composition, shape, 3D depth, structure and other multi-dimensional information of the target object, so as to solve the problem. It is difficult to ensure the accuracy of recognition by using the two-dimensional image information of the target object mentioned in the background art section, for example, it is difficult to distinguish whether it is a real person or a picture.
  • optical artificial neural network chip provided by the embodiment of the present application can not only be able to Achieving the effect of low power consumption and low latency can also achieve the effect of high accuracy, which can be used for intelligent processing tasks such as intelligent perception, recognition and/or decision-making.
  • the optical artificial neural network intelligent chip is used for the intelligent processing task of the target object;
  • the intelligent processing task includes at least one of intelligent perception, intelligent recognition and intelligent decision-making tasks or more;
  • the reflected light, transmitted light and/or radiated light of the target object enters the trained optical artificial neural network smart chip, and the intelligent processing result of the target object is obtained; the intelligent processing result at least includes the intelligent perception result and the intelligent recognition result and/or one or more of the Smart Decision Results;
  • the trained optical artificial neural network intelligent chip refers to the optical artificial neural network intelligent chip including the trained optical modulation structure, image sensor and processor; the trained optical modulation structure, image sensor and processor refers to the use of
  • the input training samples and output training samples corresponding to the intelligent processing task are obtained by training optical artificial neural network smart chips including different light modulation structures, image sensors and processors with different full connection parameters and nonlinear activation parameters. Light modulation structures, image sensors, and processors that satisfy training convergence conditions.
  • the optical artificial neural network intelligent chip can be used for intelligent processing tasks of the target object, for example, including one or more tasks of intelligent perception, intelligent recognition and intelligent decision-making tasks.
  • intelligent perception refers to mapping the signals of the physical world to the digital world through hardware devices such as cameras, microphones or other sensors, and using cutting-edge technologies such as speech recognition and image recognition, and then converting these digital signals into the digital world.
  • Information is further advanced to the level of recognition, such as memory, understanding, planning, decision-making, and so on.
  • Intelligent recognition refers to the technology that uses computers to process, analyze and understand images to identify targets and objects in different modes.
  • intelligent recognition technology is generally divided into face recognition and commodity recognition. Face recognition is mainly used in security inspections. , identity verification and mobile payment, commodity identification is mainly used in the process of commodity circulation, especially in unmanned retail areas such as unmanned shelves and smart retail cabinets.
  • Intelligent decision-making refers to the solution of automatic organization and coordination of multi-model operation by computer, access and processing of data in a large number of databases, and corresponding data processing and numerical calculation.
  • the reflected light, transmitted light and/or radiated light of the target object enters into the trained optical artificial neural network smart chip to obtain the intelligent processing result of the target object.
  • the recognition task of the target object is used as an example for description. It can be understood that when using the smart chip to perform the recognition task, the optical artificial neural network smart chip needs to be trained first. Smart chip training refers to determining the light modulation structure suitable for the current recognition task through training, as well as the fully connected parameters and nonlinear activation parameters suitable for the current recognition task.
  • the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight between the input layer of the artificial neural network and the linear layer
  • the light modulation in the optical filter layer is changed.
  • the structure is equivalent to changing the connection weight from the input layer of the artificial neural network to the linear layer.
  • the smart chip can be used to perform the recognition task. Specifically, after the incident light carrying the image information of the target object and the spatial spectrum information enters the optical filter layer 1 of the trained smart chip, the light modulation structure in the optical filter layer 1 modulates the incident light, and the modulated optical filter layer 1 modulates the incident light. The intensity of the optical signal is detected by the image sensor 2 and converted into an electrical signal, and then the processor 3 performs full connection processing and nonlinear activation processing to obtain the recognition result of the target object.
  • the complete process for target object recognition is: the broad-spectrum light source 100 illuminates the target object 200, and then the reflected light or transmitted light of the target object is collected by the optical artificial neural network smart chip 300, or the target object directly
  • the externally radiated light is collected by the optical artificial neural network smart chip 300 and processed by the optical filter layer, the image sensor and the processor in the smart chip, and then the recognition result can be obtained.
  • the trained optical artificial neural network smart chip refers to the optical artificial neural network smart chip including the trained optical modulation structure, image sensor and processor; the trained optical modulation structure, image sensor and processor refers to Using the input training samples and output training samples corresponding to the intelligent processing tasks, the optical artificial neural network smart chips including different light modulation structures, image sensors and processors with different fully connected parameters and nonlinear activation parameters are used to perform The trained light modulation structures, image sensors and processors that satisfy the training convergence conditions.
  • the input training sample corresponding to the intelligent identification task is the identification object sample
  • the output training sample corresponding to the intelligent identification task is the identification result of the identification object sample.
  • optical artificial neural network intelligent chip provided in this embodiment can also be used for other intelligent processing tasks of the target object, such as tasks such as intelligent perception and intelligent decision-making.
  • the optical filter layer 1 is used as the input layer of the neural network
  • the image sensor 2 is used as the linear layer of the neural network.
  • the light modulation structure in the optical filter layer is used for the target object.
  • the modulation intensities of different wavelength components in the incident light are used as the connection weights from the input layer of the neural network to the linear layer.
  • the adjustment of the connection weights of the linear layers which in turn optimizes the training of the neural network.
  • the light modulation structure in this embodiment is obtained based on neural network training, and the optical simulation of the training sample is performed by a computer to obtain the sample modulation intensity of the light modulation structure in the training sample on the different wavelength components of the incident light of the target object in the intelligent processing task , take the sample modulation intensity as the connection weight from the input layer of the neural network to the linear layer, perform nonlinear activation, and use the training samples corresponding to the intelligent processing tasks to train the neural network until the neural network converges.
  • the structure acts as an optical filter layer corresponding to the intelligent processing task.
  • this embodiment not only saves the complex signals corresponding to the input layer and the linear layer in the prior art processing and algorithm processing, and the embodiment of the present application actually uses the image information, spectral information, angle information of incident light and phase information of incident light of the target object at the same time, that is, the incident light at different points in the target object space carries information, which is represented by It can be seen that since the information carried by the incident light at different points in the target object space covers the image, composition, shape, three-dimensional depth, structure and other information of the target object, the identification processing is performed according to the information carried by the incident light at different points in the target object space.
  • the optical artificial neural network chip provided in the embodiment of the present application can not only achieve the effect of low power consumption and low delay, but also achieve the effect of high accuracy, so that it can be used for applications Be prepared for intelligent processing tasks such as intellisense, recognition and/or decision making.
  • the different light modulation structures are designed and realized by adopting computer optical simulation design.
  • optical simulation allows the user to experience the product through a digital environment before making a physical prototype.
  • the optical modulation structure is designed through computer optical simulation, and the optical modulation structure is adjusted through optical simulation.
  • the neural network converges, the corresponding optical modulation structure is determined to be the final optical modulation structure size to be produced, which saves prototyping time and cost. cost, improve product efficiency, and easily solve complex optical problems.
  • the FDTD software can be used to simulate the design of the light modulation structure, and the light modulation structure can be changed in the optical simulation, so that the modulation intensity of the light modulation structure for different incident light can be accurately predicted, and it can be used as the input layer of the neural network.
  • the connection weight of the layer is used to train the optical artificial neural network smart chip to accurately obtain the optical modulation structure.
  • the light modulation structure in the optical filter layer includes a regular structure and/or an irregular structure; and/or, the light modulation structure in the optical filter layer includes Discrete structures and/or continuous structures.
  • the light modulation structure in the optical filter layer may only include regular structures, may only include irregular structures, or may include both regular structures and irregular structures.
  • the light modulation structure includes a regular structure may refer to: the minimum modulation unit included in the light modulation structure is a regular structure, for example, the minimum modulation unit may have regular shapes such as a rectangle, a square, and a circle.
  • the light modulation structure includes a regular structure can also refer to: the arrangement of the minimum modulation units included in the light modulation structure is regular, for example, the arrangement can be a regular array form, a circular form, a trapezoidal form, a polygonal form, etc.
  • the light modulation structure includes a regular structure may also refer to: the minimum modulation unit included in the light modulation structure is a regular structure, and the arrangement of the minimum modulation units is also regular.
  • the light modulation structure here includes an irregular structure may refer to: the minimum modulation unit included in the light modulation structure is an irregular structure, for example, the minimum modulation unit may be irregular shapes such as irregular polygons and random shapes.
  • the light modulation structure includes an irregular structure may also refer to: the arrangement of the minimum modulation units included in the light modulation structure is irregular, for example, the arrangement may be in the form of an irregular polygon, a random arrangement, and the like.
  • the light modulation structure includes an irregular structure may also refer to: the minimum modulation unit included in the light modulation structure is an irregular structure, and the arrangement of the minimum modulation unit is also irregular.
  • the light modulation structures in the optical filter layer may include discrete structures, may also include continuous structures, or may include both discrete structures and continuous structures.
  • the continuous modulation pattern here may refer to a linear pattern, a wavy line pattern, a zigzag line pattern, and the like.
  • discrete modulation pattern herein may refer to a modulation pattern formed by discrete figures (eg, discrete dots, discrete triangles, discrete stars, etc.).
  • the light modulation structure has different modulation effects on light of different wavelengths, and specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, and resonance enhancement.
  • specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, and resonance enhancement.
  • the optical filter layer is a single-layer structure or a multi-layer structure.
  • the optical filter layer may be a single-layer filter structure or a multi-layer filter structure, such as a two-layer, three-layer, four-layer and other multi-layer structure.
  • the optical filter layer 1 has a single-layer structure, and the thickness of the optical filter layer 1 is related to the target wavelength range.
  • the thickness of the grating structure can be 50 nm. ⁇ 5 ⁇ m.
  • the optical filter layer 1 since the function of the optical filter layer 1 is to spectrally modulate the incident light, it is preferably prepared from materials with high refractive index and low loss, for example, silicon, germanium, germanium-silicon materials, silicon compounds, germanium can be selected. Compounds, III-V group materials, etc. are prepared, wherein silicon compounds include but are not limited to silicon nitride, silicon dioxide, silicon carbide, and the like.
  • the optical filter layer 1 in order to form more or more complex connection weights between the input layer and the linear layer, preferably, the optical filter layer 1 can be set as a multi-layer structure, and the corresponding optical modulation of each layer The structure can be set to different structures, thereby increasing the spectral modulation capability of the optical filter layer to the incident light, so that more or more complex connection weights can be formed between the input layer and the linear layer, thereby improving the processing intelligence of the smart chip. accuracy at the time of the task.
  • the materials of each layer structure may be the same or different.
  • the first layer may be is a silicon layer
  • the second layer may be a silicon nitride layer.
  • the thickness of the optical filter layer 1 is related to the target wavelength range. For wavelengths of 400 nm to 10 ⁇ m, the total thickness of the multilayer structure may be 50 nm to 5 ⁇ m.
  • the light modulation structure in the optical filter layer includes a unit array composed of a plurality of micro-nano units, and each micro-nano unit corresponds to one or more units on the image sensor. Pixel point; the structure of each micro-nano unit is the same or different.
  • the light modulation structure in the form of an array structure.
  • the light modulation structure includes a unit array composed of a plurality of micro-nano units, and each micro-nano unit corresponds to one or more pixels on the image sensor.
  • the structures of each micro-nano unit may be the same or different.
  • the structure of each micro-nano unit may be periodic or aperiodic.
  • each micro-nano unit may further include multiple groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.
  • the optical filter layer 1 includes a plurality of repeated continuous or discrete micro-nano units, such as 55, 66, each micro-nano unit has the same structure (and each micro-nano unit is an aperiodic structure), and each micro-nano unit corresponds to one or more pixels on the image sensor 2; as shown in FIG. 6, the optical filter Layer 1 contains multiple repeating micro-nano units, such as 11, 22, 33, 44, 55, 66, and each micro-nano unit has the same structure (the difference from FIG. 5 is that each micro-nano unit in FIG. 6 has a periodic structure ), each micro-nano unit corresponds to one or more pixels on the image sensor 2; as shown in FIG.
  • the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55 , 66, each micro-nano unit has the same structure (and each micro-nano unit is a periodic structure), each micro-nano unit corresponds to one or more pixels on the image sensor 2, and the difference from FIG. 6 is that in FIG. 7
  • the unit shape of the periodic array in each micro-nano unit has quadruple rotational symmetry; as shown in FIG. The difference is that each micro-nano unit has a different structure, and each micro-nano unit corresponds to one or more pixels on the image sensor 2.
  • the optical filter layer 1 contains a plurality of mutually different micro-nano units.
  • the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, 66, and each micro-nano unit has the same structure.
  • each micro-nano unit has the same structure.
  • the units are composed of discrete aperiodic array structures, and each micro-nano unit corresponds to one or more pixels on the image sensor 2 .
  • the micro-nano unit has different modulation effects on light of different wavelengths, and the specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, and the like.
  • the specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, and the like.
  • the micro-nano units include regular structures and/or irregular structures; and/or, the micro-nano units include discrete structures and/or continuous structures.
  • the micro-nano unit may only include regular structures, may only include irregular structures, or may include both regular structures and irregular structures.
  • the micro-nano unit includes a regular structure may refer to: the minimum modulation unit included in the micro-nano unit is a regular structure, for example, the minimum modulation unit may have regular patterns such as rectangle, square and circle.
  • the micro-nano unit includes a regular structure can also refer to: the arrangement of the smallest modulation units contained in the micro-nano unit is regular, for example, the arrangement can be a regular array form, a circular form, a trapezoidal form, a polygonal form, etc.
  • the micro-nano unit includes a regular structure can also refer to: the minimum modulation unit included in the micro-nano unit is a regular structure, and the arrangement of the minimum modulation unit is also regular, etc.
  • the micro-nano unit includes an irregular structure may refer to: the minimum modulation unit included in the micro-nano unit is an irregular structure, for example, the minimum modulation unit may be irregular polygons, random shapes and other irregular shapes.
  • the micro-nano unit includes irregular structure may also refer to: the arrangement of the smallest modulation units included in the micro-nano unit is irregular, for example, the arrangement may be in the form of irregular polygons, random arrangement and the like.
  • the micro-nano unit includes an irregular structure may also refer to: the minimum modulation unit included in the micro-nano unit is an irregular structure, and the arrangement of the minimum modulation unit is also irregular.
  • the micro-nano units in the optical filter layer may include discrete structures, may also include continuous structures, or may include both discrete and continuous structures.
  • the micro-nano unit includes a continuous structure may refer to: the micro-nano unit is composed of continuous modulation patterns; here the micro-nano unit includes a discrete structure may refer to: the micro-nano unit is composed of discrete modulation patterns of.
  • the continuous modulation pattern here may refer to a linear pattern, a wavy line pattern, a zigzag line pattern, and the like.
  • discrete modulation pattern herein may refer to a modulation pattern formed by discrete figures (eg, discrete dots, discrete triangles, discrete stars, etc.).
  • micro-nano units have different modulation effects on light of different wavelengths, and specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, and resonance enhancement.
  • specific modulation methods include but are not limited to scattering, absorption, interference, surface plasmon, and resonance enhancement.
  • the micro-nano unit includes multiple groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.
  • the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, 66, and each micro-nano unit includes a plurality of groups of micro-nano units
  • the structure array for example, the micro/nano unit 11 includes four different micro/nano structure arrays 110 , 111 , 112 and 113
  • the filtering unit 44 includes four different micro/nano structure arrays 440 , 441 , 442 and 443 . As shown in FIG.
  • the optical filter layer 1 includes a plurality of micro-nano units, such as 11, 22, 33, 44, 55, 66, and each micro-nano unit includes a plurality of groups of micro-nano structure arrays, such as the micro-nano unit 11 Four identical micro-nano structure arrays 110 , 111 , 112 and 113 are included.
  • micro-nano unit including four groups of micro-nano structure arrays is used as an example for illustration, and it does not play a limiting role.
  • the micro-nano unit of the micro-nano structure array is used as an example for illustration, and it does not play a limiting role.
  • each group of micro-nano structure arrays in the micro-nano unit has different modulation effects on light of different wavelengths, and the modulation effects on input light between groups of filter structures are also different.
  • the specific modulation methods include: Not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc.
  • each group of micro-nano structure arrays has the function of broadband filtering or narrow-band filtering.
  • the micro-nano structure array obtains the modulation intensity of different wavelength components of the incident light of the target object by performing broadband filtering or narrow-band filtering on the incident light of the target object.
  • each group of micro-nano structure arrays in the optical filter layer has the function of broadband filtering or narrow-band filtering.
  • each group of micro-nano structure arrays may have broadband filtering effects, and they may all have narrow-band filtering effects, and some may have broadband filtering effects, and some may have narrow-band filtering effects.
  • the broadband filtering range and the narrow-band filtering range of each group of micro-nano structure arrays may also be the same or different.
  • it by designing the period, duty cycle, radius, side length and other size parameters of each group of micro-nano structures in the micro-nano unit, it has a narrow-band filtering effect, that is, only one (or less) wavelength of light can be used. pass.
  • the period, duty cycle, radius, side length and other dimensional parameters of each group of micro-nano structures in the micro-nano unit it has a broadband filtering effect, that is, light with more or all wavelengths can pass through.
  • the filtering state of each group of micro-nano structure arrays can be determined by performing broadband filtering, narrow-band filtering or a combination thereof according to the application scenario.
  • each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.
  • each group of micro-nano structure arrays may be all periodic structure arrays, may all be non-periodic structure arrays, or may be partially periodic structure arrays and partially non-periodic structure arrays.
  • the periodic structure array is easy to carry out optical simulation design, and the aperiodic structure array can realize more complex modulation effect.
  • the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, 66, and each micro-nano unit is composed of multiple groups of micro-nano structures
  • the arrays are composed of different micro-nano structure arrays, and the micro-nano structure arrays are non-periodic structures.
  • the aperiodic structure means that the shape of the modulation holes on the micro-nano structure array is arranged in a non-periodic arrangement. As shown in FIG.
  • the micro-nano unit 11 includes 4 different aperiodic structure arrays 110 , 111 , 112 and 113
  • the micro-nano unit 44 includes 4 different aperiodic structure arrays 440 , 441 , 442 and 443
  • the micro-nano structure array of sexual structure is designed by neural network data training for intelligent processing tasks in the early stage, and it is usually an irregular-shaped structure.
  • the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, 66, and each micro-nano unit is composed of multiple groups of micro-nano structure arrays. The structures of the structure arrays are different from each other. The difference from FIG.
  • the micro-nano structure array is a periodic structure.
  • the periodic structure means that the shape of the modulation holes on the micro-nano structure array is arranged in a periodic arrangement, and the size of the period is usually 20 nm to 50 ⁇ m.
  • the micro-nano unit 11 includes 4 different periodic structure arrays 110 , 111 , 112 and 113
  • the micro-nano unit 44 includes 4 different periodic structure arrays 440 , 441 , 442 and 443 .
  • the filter structure is designed by training the neural network data for intelligent processing tasks in the early stage, and it is usually an irregular-shaped structure. As shown in FIG.
  • the optical filter layer 1 includes a plurality of different micro-nano units, such as 11, 22, 33, 44, 55, 66, and each micro-nano unit is composed of multiple groups of micro-nano structure arrays.
  • the structures of the micro-nano structure arrays are different from each other, and the micro-nano structure arrays are periodic structures.
  • the periodic structure means that the shapes on the filter structure are arranged in a periodic arrangement, and the size of the period is usually 20 nm to 50 ⁇ m.
  • the micro-nano structure arrays of the micro-nano unit 11 and the micro-nano unit 12 are different from each other.
  • the micro-nano unit 11 includes four different periodic structure arrays 110 , 111 , 112 and 113
  • the micro-nano unit 44 includes four There are different periodic structure arrays 440 , 441 , 442 and 443
  • the periodic structure micro-nano structure arrays are designed by neural network data training for intelligent processing tasks in the early stage, and are usually irregular-shaped structures.
  • each micro-nano unit in Fig. 5 to Fig. 9 includes four groups of micro-nano structure arrays, and the four groups of micro-nano structure arrays are respectively formed with modulation holes of four different shapes.
  • the incident light has different modulation effects.
  • the micro-nano unit including four groups of micro-nano structure arrays is used as an example for illustration, and it does not play a limiting role.
  • the four different shapes may be a circle, a cross, a regular polygon and a rectangle (not limited thereto).
  • each group of micro-nano structure arrays in the micro-nano unit has different modulation effects on light of different wavelengths, and the modulation effects on the input light are also different between groups of micro-nano structure arrays.
  • the specific modulation method Including but not limited to scattering, absorption, interference, surface plasmon, resonance enhancement, etc.
  • the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, 66, Each micro-nano unit is composed of multiple groups of micro-nano structure arrays. The structures corresponding to the multiple groups of micro-nano structure arrays are different from each other.
  • the micro-nano structure array is a periodic structure. The difference from the above embodiment is that for any micro-nano structure array A cell contains one or more sets of empty structures for passing through incident light.
  • each micro-nano unit includes a set of micro-nano structure arrays and three sets of empty structures
  • the micro-nano unit 11 includes an aperiodic structure array 111
  • the micro-nano unit 22 includes an aperiodic structure array 221
  • the micro-nano unit 33 includes an aperiodic structure array 331
  • the micro-nano unit 44 includes an aperiodic structure array 441
  • the micro-nano unit 55 includes an aperiodic structure array 55
  • the micro-nano unit 66 includes an aperiodic structure array Array 661, wherein the array of micro-nano structures is used for different modulation of incident light.
  • micro-nano structure arrays can be set to include a set of micro-nano structure arrays and five Groups of empty structures or other numbers of micro-nano units of micro-nano structure arrays.
  • the array of micro-nano structures can be made of circular, cross-shaped, regular polygon and rectangular modulation holes (not limited thereto).
  • the multiple groups of micro-nano structure arrays included in the micro-nano unit may not contain empty structures, that is, the multiple groups of micro-nano structure arrays may be aperiodic structure arrays or periodic structure arrays.
  • the micro-nano unit has polarization independent characteristics.
  • the optical filter layer is insensitive to the polarization of incident light, thereby realizing an optical artificial neural network smart chip that is insensitive to incident angle and polarization.
  • the optical artificial neural network smart chip provided by the embodiment of the present application is not sensitive to the incident angle and polarization characteristics of incident light, that is, the measurement result will not be affected by the incident angle and polarization characteristics of incident light, thereby ensuring the stability of spectral measurement performance. , which can ensure the stability of intelligent processing, such as the stability of intelligent perception, the stability of intelligent recognition, the stability of intelligent decision-making and so on.
  • the micro-nano unit may also have polarization-dependent properties.
  • the micro-nano unit has quadruple rotational symmetry.
  • quadruple rotational symmetry is a specific case of polarization-independent characteristics.
  • the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, 66, Each micro-nano unit is composed of multiple groups of micro-nano structure arrays, the corresponding structures of the multiple groups of micro-nano structure arrays are different from each other, and the micro-nano structure arrays are periodic structures.
  • the corresponding structure of the array can be a circle, a cross, a regular polygon, a rectangle, etc.
  • the optical filter layer is composed of one or more filter layers
  • the filter layer is made of one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, and perovskite materials; and/or, the filter layer is made of photonic crystals, metasurfaces, random Structure, nanostructure, metal surface plasmon SPP (Surface Plasmon Polaritons, SPP) micro-nano structure, tunable Fabry-Perot cavity (Fabry-perot Cavity, FP cavity) one or more preparations filter layer.
  • SPP Surface Plasmon Polaritons
  • Fabry-perot Cavity, FP cavity tunable Fabry-Perot cavity
  • the semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and/or, the nanostructure includes nanostructures.
  • photonic crystals and the combination of metasurfaces and random structures can be compatible with CMOS technology and can have better modulation effects.
  • the micropores of micro-nano modulation structures can also be filled with other materials for surface smoothing; Mine can make use of the spectral modulation characteristics of the material itself to minimize the volume of a single modulation structure; SPP is small in size and can realize polarization-dependent light modulation; liquid crystal can be dynamically regulated by voltage to improve spatial resolution; adjustable Fabry-Perot The resonant cavity can be dynamically adjusted to improve the spatial resolution.
  • the thickness of the optical filter layer is 0.1 ⁇ ⁇ 10 ⁇ , where ⁇ represents the center wavelength of the incident light.
  • the overall size (area) of each micro-nano unit in the optical filter layer 1 is usually ⁇ 2 -105 ⁇ 2 , and the thickness is usually 0.1 ⁇ 10 ⁇ ( ⁇ represents the center wavelength of the incident light of the target object).
  • the overall size of each micro-nano unit is 0.5 ⁇ m 2 to 40000 ⁇ m 2
  • the dielectric material in the optical filter layer 1 is polysilicon
  • the thickness is 50 nm to 2 ⁇ m.
  • the image sensor is any one or more of the following:
  • CMOS image sensor Contact Image Sensor, CIS
  • Charge Coupled Device Charge Coupled Device, CCD
  • Single Photon Avalanche Diode Single Photon Avalanche Diode, SPAD
  • focal plane photodetector array CMOS image sensor (Contact Image Sensor, CIS), Charge Coupled Device (Charge Coupled Device, CCD), Single Photon Avalanche Diode (Single Photon Avalanche Diode, SPAD) array and focal plane photodetector array.
  • CMOS image sensor CIS to achieve monolithic integration at the wafer level can minimize the distance between the image sensor and the optical filter layer, which is beneficial for Reduce the size of the unit, reduce the size of the device and the cost of packaging, SPAD can be used for weak light detection, CCD can be used for strong light detection.
  • the optical filter layer and the image sensor can be fabricated by a complementary metal oxide semiconductor (Complementary Metal Oxide Semiconductor, CMOS) integrated process, which is beneficial to reduce device failure rate, improve device yield and reduce cost.
  • CMOS complementary Metal Oxide Semiconductor
  • the optical filter can be prepared by directly growing one or more layers of dielectric material on the image sensor, and then performing etching, depositing metal material before removing the sacrificial layer for etching, and finally removing the sacrificial layer. device layer.
  • the type of the artificial neural network includes: a feedforward neural network.
  • Feedforward Neural Network also known as Deep Feedforward Network (DFN), Multi-Layer Perceptron (MLP)
  • DNN Deep Feedforward Network
  • MLP Multi-Layer Perceptron
  • the feedforward neural network has a simple structure, is easy to implement on hardware, and has a wide range of applications. It can approximate any continuous function and square-integrable function with arbitrary precision, and can accurately implement any limited training sample set.
  • a feedforward network is a static nonlinear mapping. Complex nonlinear processing capabilities can be obtained through composite mapping of simple nonlinear processing units.
  • a light-transmitting medium layer is provided before the optical filter layer and the image sensor.
  • a light-transmitting medium layer is arranged between the optical filter layer and the image sensor, which can effectively separate the optical filter layer and the image sensor layer and avoid mutual interference between the two.
  • the image sensor is a front-illuminated type, including: a metal wire layer and a light detection layer arranged from top to bottom, and the optical filter layer is integrated in the metal wire layer the side away from the light detection layer; or,
  • the image sensor is back-illuminated and includes: a light detection layer and a metal line layer arranged from top to bottom, and the light filter layer is integrated on the side of the light detection layer away from the metal line layer.
  • the silicon detection layer 21 is below the metal wire layer 22
  • the optical filter layer 1 is directly integrated on the metal wire layer 22 .
  • FIG. 14 shows a back-illuminated image sensor
  • the silicon detection layer 21 is above the metal wire layer 22
  • the optical filter layer 1 is directly integrated on the silicon detection layer 21 .
  • the silicon detection layer 21 is above the metal wire layer 22, which can reduce the influence of the metal wire layer on the incident light, thereby improving the quantum efficiency of the device.
  • the optical filter layer is used as the input layer of the artificial neural network
  • the image sensor is used as the linear layer of the artificial neural network
  • the filtering effect of the optical filter layer on the incident light entering the optical filter layer is used.
  • the connection weight between the input layer and the linear layer the optical filter layer and the image sensor in the optical artificial neural network intelligent chip provided by this embodiment realize the related functions of the input layer and the linear layer in the artificial neural network by means of hardware, thereby This makes it unnecessary to perform complex signal processing and algorithm processing corresponding to the input layer and the linear layer when using the smart chip for subsequent intelligent processing, which can greatly reduce the power consumption and delay of the artificial neural network processing.
  • the present embodiment utilizes the image information of the target object and the spectral information at different points in space at the same time, the intelligent processing of the target object can be implemented more accurately.
  • an intelligent processing device including: the optical artificial neural network intelligent chip as described in the above embodiments.
  • the intelligent processing device includes one or more of a smart phone, an intelligent computer, an intelligent identification device, an intelligent perception device, and an intelligent decision-making device.
  • the intelligent processing device provided in this embodiment includes the optical artificial neural network smart chip described in the above embodiments, the intelligent processing device provided in this embodiment has all the beneficial effects of the optical artificial neural network smart chip described in the above embodiments , since the foregoing embodiment has already described this in detail, it will not be repeated in this embodiment.
  • FIG. 15 Another embodiment of the present application provides a method for preparing an optical artificial neural network smart chip according to the above-mentioned embodiment, as shown in FIG. 15 , which specifically includes the following steps:
  • Step 1510 preparing an optical filter layer including a light modulation structure on the surface of the photosensitive region of the image sensor
  • Step 1520 generating a processor capable of performing full connection processing and nonlinear activation processing on the signal
  • Step 1530 connect the image sensor and the processor
  • the optical filter layer is used to perform different spectral modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain different spectral modulations on the surface of the photosensitive region.
  • the image sensor is used to convert the incident light-carrying information corresponding to the different position points after being modulated by the optical filter layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processing
  • the processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain the output signal of the artificial neural network.
  • a light filter layer containing a light modulation structure is prepared on the surface of the photosensitive area of the image sensor, including:
  • an optical filter layer containing a light modulation structure is obtained;
  • the input training samples and the output training samples corresponding to the intelligent processing is trained to obtain an optical modulation structure, an image sensor and a processor that meet the training convergence conditions.
  • the optical filter layer 1 can be etched by directly growing one or more layers of dielectric materials on the image sensor 2 . Before the sacrificial layer is deposited, the metal material is deposited, and finally the sacrificial layer is removed for preparation.
  • each unit can have different modulation effects on light of different wavelengths within the target range, and the modulation effects are not sensitive to the incident angle and polarization.
  • Each unit in the optical filter layer 1 corresponds to one or more pixels on the image sensor 2 . 1 was prepared directly on 2.
  • the optical filter layer 1 can be directly etched on the silicon detector layer 21 of the back-illuminated image sensor. etched and then deposited metal for preparation.
  • the light modulation structure on the optical filter layer can be dry-etched by performing light modulation structure pattern on one or more preset materials. Dry etching is to directly photosensitive the image sensor. One or more layers of preset materials on the surface of the area are removed to obtain an optical filter layer containing a light modulation structure; or one or more layers of preset materials are imprinted and transferred, and imprint transfer is performed in
  • the required structure is prepared on other substrates by etching, and then the structure is transferred to the photosensitive area of the image sensor through materials such as PDMS to obtain an optical filter layer containing a light modulation structure; or by presetting one or more layers
  • the material is subjected to external dynamic control, and the external dynamic control is to use active materials, and then the external electrode can adjust the light modulation characteristics of the corresponding area by changing the voltage, so as to obtain an optical filter layer containing a light modulation structure; It is assumed that the material is printed by partition, and the partition printing is a technology of partition printing to obtain an optical filter layer containing
  • face recognition technology is a biometric identification technology, which is widely used in access control and attendance systems, criminal investigation systems, e-commerce and other fields.
  • the main process of face recognition includes face image collection, preprocessing, feature extraction, matching and recognition.
  • face image collection preprocessing
  • feature extraction feature extraction
  • matching recognition
  • this embodiment provides a new type of optoelectronic chip for accurate face recognition.
  • the chip consists of an optical filter layer to form the input layer of the optical artificial neural network and the input
  • the connection weight from the layer to the linear layer is composed of the image sensor to form the linear layer of the optical artificial neural network.
  • the embodiment of the present application also provides an optical artificial neural network face recognition chip, which is used for face recognition processing tasks, including: an optical filter layer, an image sensor and a processor; the optical filter layer corresponds to the artificial neural network. the input layer and the connection weight between the input layer and the linear layer, the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the nonlinear layer and the output layer of the artificial neural network;
  • the optical filter layer is disposed on the surface of the photosensitive region of the image sensor, the optical filter layer includes a light modulation structure, and the optical filter layer is used for entering the light through the light modulation structure pair
  • the incident light at different positions of the modulation structure is subjected to different spectral modulations respectively, so as to obtain incident light-carrying information corresponding to different positions on the surface of the photosensitive region;
  • the incident light-carrying information includes light intensity distribution information, spectral information , the angle information of the incident light and the phase information of the incident light;
  • the incident light includes the reflected light of the user's face;
  • the image sensor is used to convert the incident light-carrying information corresponding to the different position points after being modulated by the optical filter layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processing
  • the electrical signal is an image signal modulated by the optical filter layer
  • the processor is used to perform full connection processing and nonlinear activation processing on electrical signals corresponding to different position points to obtain a face recognition processing result.
  • this embodiment implements a brand-new optical artificial neural network face recognition chip capable of realizing the function of artificial neural network, which is used for the face recognition task.
  • the optical filter layer on the hardware chip is used as the input layer of the artificial neural network and the connection weight between the input layer and the linear layer, and the image sensor on the hardware chip is used as the linear layer of the artificial neural network.
  • the information is injected into the pre-trained hardware chip, and the artificial neural network analysis is performed on the spatial spectrum information of the face through the hardware chip to obtain the face recognition result. It should be noted that the embodiment of the present application realizes low power consumption and safety. Reliable fast and accurate face recognition.
  • the hardware structure on it - the optical filter layer corresponds to the input layer of the artificial neural network and the connection weight between the input layer and the linear layer, and the hardware structure on it -
  • the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the nonlinear layer and the output layer of the artificial neural network.
  • the optical filter layer is disposed on the surface of the photosensitive area of the image sensor, the optical filter layer includes a light modulation structure, and the optical filter layer is used to pass the light modulation structure to the incident light entering different positions of the light modulation structure. Different spectral modulations are respectively performed to obtain incident light-carrying information corresponding to different positions on the surface of the photosensitive area.
  • the image sensor is used to convert the incident light-carrying information corresponding to different positions into different positions.
  • the processor connected to the image sensor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different positions to obtain the output signal of the artificial neural network.
  • the optical filter layer is used as the input layer of the artificial neural network
  • the image sensor is used as the linear layer of the artificial neural network.
  • the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the input layer
  • the connection weight to the linear layer that is, the optical filter layer and the image sensor in the face recognition chip realize the relevant functions of the input layer and the linear layer in the artificial neural network, that is, the embodiment of the present application adopts the prior art.
  • the input layer and the linear layer in the artificial neural network implemented by software are stripped off, and the input layer and the linear layer in the artificial neural network are realized by hardware, so that the face recognition chip can be used in the subsequent artificial neural network.
  • Neural network face recognition processing does not need to perform complex signal processing and algorithm processing corresponding to the input layer and linear layer, only the processor in the face recognition chip needs to be fully connected with electrical signals and nonlinear activation correlation It can be processed, which can greatly reduce the power consumption and delay of artificial neural network face recognition.
  • the optical filter layer is used as the input layer of the artificial neural network
  • the image sensor is used as the linear layer of the artificial neural network
  • the filtering effect of the optical filter layer on the incident light entering the optical filter layer is used as the input.
  • connection weight from the layer to the linear layer uses the optical filter layer and the image sensor to project the spatial spectral information of the face into an electrical signal, and then realizes the full connection processing and nonlinear activation processing of the electrical signal in the processor. It can be seen that, The embodiments of the present application can not only save the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art, but also use the image information of the face, the spectral information, and the incident light's image information at the same time.
  • the angle information and the phase information of the incident light that is, the information carried by the incident light at different points in the face space
  • the information carried by the incident light at different points in the face space covers the image, composition, shape, three-dimensional Depth, structure and other information, so that when the recognition processing is carried out according to the information carried by the incident light at different points in the face space, it can cover the image, composition, shape, three-dimensional depth, structure and other multi-dimensional information of the face, so as to accurately Perform face recognition.
  • the optical artificial neural network face recognition chip includes a trained light modulation structure, an image sensor and a processor;
  • the trained light modulation structures, image sensors and processors refer to the use of input training samples and output training samples corresponding to face recognition tasks, and the use of input training samples and output training samples corresponding to face recognition tasks.
  • the optical modulation structure, image sensor and processor that satisfy the training convergence condition are obtained by training the optical artificial neural network face recognition chip of the processor with different nonlinear activation parameters;
  • the input training samples include incident light reflected by different faces; the output training samples include corresponding face recognition results.
  • the different light modulation structures are passed through. It is designed and implemented by means of computer optical simulation design.
  • an artificial neural network photoelectric chip whose input is an object image and its frequency spectrum can be realized, and fast, accurate, safe and reliable living face recognition can be realized.
  • the faces of a large number of people can be collected first, and the weight from the input layer to the linear layer, that is, the system function of the optical filter layer, can be obtained through data training, and then the required design can be reversed.
  • the optical filter layer which is integrated above the image sensor.
  • a high-accuracy optical artificial neural network can be realized. Fast and accurate recognition of user faces.
  • the specific modulation pattern of the modulation structure on the optical filter layer is obtained by collecting the faces of a large number of people in the early stage and training and designing the artificial neural network data. shape structure.
  • the complete process of face recognition is: ambient light or other light sources illuminate the user's face, and then the reflected light is collected by the chip, and the recognition result is obtained after internal processing.
  • the chip actually utilizes the image information, spectral information, angle information of incident light and phase information of incident light at the same time, which improves the accuracy and security of face recognition, especially for non-living objects.
  • the face model was also able to accurately exclude it.
  • the chip partially implements an artificial neural network on the hardware, which improves the speed of face recognition.
  • the chip solution can use the existing CMOS process to achieve mass production, which reduces the size, power consumption and cost of the device.
  • the light modulation structures in the optical filter layer include regular structures and/or irregular structures; and/or, the light modulation structures in the optical filter layer include discrete structures and/or continuous structures.
  • the optical filter layer has a single-layer structure or a multi-layer structure.
  • the light modulation structure in the optical filter layer includes a unit array composed of a plurality of micro/nano units, each micro/nano unit corresponds to one or more pixels on the image sensor; the structures of each micro/nano unit are the same or different.
  • micro-nano unit includes a regular structure and/or an irregular structure; and/or, the micro-nano unit includes a discrete structure and/or a continuous structure.
  • micro-nano unit includes a plurality of groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.
  • each group of micro-nano structure arrays has the function of broadband filtering or narrow-band filtering.
  • each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.
  • micro-nano unit has polarization independent properties.
  • micro-nano unit has quadruple rotational symmetry.
  • optical filter layer is composed of one or more filter layers
  • the filter layer is made of one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, and perovskite materials; and/or, the filter layer is made of photonic crystals, metasurfaces, random A filter layer prepared by one or more of a structure, a nanostructure, a metal surface plasmon SPP micro-nano structure, and a tunable Fabry-Perot resonant cavity.
  • the semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and/or, the nanometer
  • the structure includes one or more of nanodot 2D materials, nanopillar 2D materials, and nanowire 2D materials.
  • the thickness of the optical filter layer is 0.1 ⁇ ⁇ 10 ⁇ , where ⁇ represents the center wavelength of the incident light.
  • the embodiments of the present application further provide a face recognition device, including the optical artificial neural network face recognition chip described in the above embodiments.
  • the face recognition device may be a portable face recognition device or a face recognition device installed in a fixed position. Since the face recognition device has similar beneficial effects to the above-mentioned optical artificial neural network face recognition chip, it will not be repeated here.
  • the embodiment of the present application also provides a preparation method of the optical artificial neural network face recognition chip as described above, including:
  • the optical filter layer is used to perform different spectral modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain different spectral modulations on the surface of the photosensitive region.
  • the image sensor is used to convert the incident light-carrying information corresponding to the different position points after being modulated by the optical filter layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processing
  • the processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain a face recognition processing result; the electrical signals are image signals modulated by the optical filter layer, so The incident light includes the reflected light of the human face.
  • the preparation method of the optical artificial neural network face recognition chip also includes: a training process for the optical artificial neural network face recognition chip, specifically including:
  • the optical artificial neural network face containing different light modulation structures, image sensors and processors with different fully connected parameters and different nonlinear activation parameters
  • the recognition chip is trained to obtain a light modulation structure, an image sensor and a processor that meet the training convergence conditions, and the light modulation structure, image sensor and processor that meet the training convergence conditions are used as the trained light modulation structure, image sensor and processor.
  • the different light modulation structures are passed through. It is designed and implemented by means of computer optical simulation design.
  • the optical artificial neural network face recognition chip based on the micro-nano modulation structure and the image sensor provided in this embodiment has the following effects: A.
  • the artificial neural network is partially embedded in the image sensor including various optical filter layers , to achieve safe, reliable, fast and accurate face recognition.
  • the preparation of the chip can be completed by one tap-out of the CMOS process, which is beneficial to reduce the failure rate of the device, improve the yield of the device, and reduce the cost.
  • Monolithic integration at the wafer level can minimize the distance between the sensor and the optical filter layer, which is conducive to reducing the size of the unit, reducing device volume and packaging costs.
  • optical artificial neural network face recognition chip it should be noted that, for the detailed structural description of the optical artificial neural network face recognition chip provided in this embodiment, reference may be made to the introduction of the optical artificial neural network chip in the foregoing embodiment, which is not described here to avoid repetition. In addition, for the detailed introduction of the optical artificial neural network face recognition chip preparation method, you can also refer to the introduction of the optical artificial neural network chip preparation method in the foregoing embodiment, which will not be repeated here.
  • the methods of non-invasive blood glucose detection mainly include optical and radiation methods, reverse iontophoresis analysis, electromagnetic wave method, ultrasonic method and tissue fluid extraction method.
  • the near-infrared spectral detection method mainly utilizes the relationship between the blood sugar concentration and its near-infrared spectral absorption, irradiates the skin with near-infrared light, and reflects the blood sugar concentration from the change in the intensity of the reflected light.
  • This method has the advantages of fast measurement, no need for chemical reagents and consumables, etc.
  • due to the large individual differences of the measured objects, and the obtained signal is very weak, so weak chemical extraction is performed in the selection of measurement sites, measurement conditions, and overlapping spectra.
  • the signal processing system is bulky and cannot be carried around.
  • a subcutaneous tissue fluid detection method which reflects the blood glucose concentration by measuring the glucose concentration of the tissue fluid exuded under the skin. According to this principle, a watch can be made to detect glucose, and the blood glucose concentration can be continuously monitored in real time.
  • this method has poor accuracy and slow response speed, so it is difficult to replace the existing invasive blood glucose meter. Therefore, non-invasive blood glucose testing has become an urgent need for the treatment of diabetes.
  • this embodiment provides a new type of optoelectronic chip for blood sugar detection.
  • the connection weight of the linear layer is composed of the image sensor to form the linear layer of the optical artificial neural network.
  • the embodiment of the present application also provides an optical artificial neural network blood sugar detection chip, which is used for blood sugar detection tasks, including: an optical filter layer, an image sensor and a processor; the optical filter layer corresponds to the input layer of the artificial neural network and The connection weight between the input layer and the linear layer, the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the nonlinear layer and the output layer of the artificial neural network;
  • the optical filter layer is disposed on the surface of the photosensitive region of the image sensor, the optical filter layer includes a light modulation structure, and the optical filter layer is used for entering the light through the light modulation structure pair
  • the incident light at different positions of the modulation structure is subjected to different spectral modulations respectively, so as to obtain incident light-carrying information corresponding to different positions on the surface of the photosensitive region;
  • the incident light-carrying information includes light intensity distribution information, spectral information , the angle information of the incident light and the phase information of the incident light;
  • the incident light includes the reflected light and/or the transmitted light of the body part to be measured;
  • the image sensor is used to convert the incident light-carrying information corresponding to the different position points after being modulated by the optical filter layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processing
  • the electrical signal is an image signal modulated by the optical filter layer
  • the processor is used to perform full connection processing and nonlinear activation processing on electrical signals corresponding to different positions to obtain blood glucose detection results.
  • this embodiment implements a brand-new optical artificial neural network blood glucose detection chip capable of realizing the function of artificial neural network, which is used for the task of blood glucose detection.
  • the optical filter layer above is used as the input layer of the artificial neural network and the connection weight between the input layer and the linear layer, and the image sensor on the hardware chip is used as the linear layer of the artificial neural network.
  • the information is injected into the pre-trained hardware chip, and the artificial neural network analysis is performed on the spatial spectral information of the part to be measured by the hardware chip to obtain the blood glucose detection result. It should be noted that the embodiment of the present application realizes low power consumption, Safe and reliable fast and accurate non-invasive blood glucose testing.
  • the hardware structure on it - the optical filter layer corresponds to the input layer of the artificial neural network and the connection weight between the input layer and the linear layer, and the hardware structure on it - the image.
  • the sensor corresponds to the linear layer of the artificial neural network;
  • the processor corresponds to the nonlinear layer and the output layer of the artificial neural network.
  • the optical filter layer is disposed on the surface of the photosensitive area of the image sensor, the optical filter layer includes a light modulation structure, and the optical filter layer is used to pass the light modulation structure to the incident light entering different positions of the light modulation structure. Different spectral modulations are respectively performed to obtain incident light-carrying information corresponding to different positions on the surface of the photosensitive area.
  • the image sensor is used to convert the incident light-carrying information corresponding to different positions into different positions.
  • the processor connected to the image sensor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different positions to obtain the output signal of the artificial neural network. It can be seen that in the blood sugar detection chip, the optical filter layer is used as the input layer of the artificial neural network, and the image sensor is used as the linear layer of the artificial neural network.
  • the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the input layer to
  • the connection weight of the linear layer that is, the optical filter layer and the image sensor in the blood sugar detection chip realize the related functions of the input layer and the linear layer in the artificial neural network, that is, the embodiment of the present application uses software in the prior art to realize
  • the input layer and linear layer in the artificial neural network are stripped off, and the input layer and the linear layer in the artificial neural network are realized by means of hardware, so that the blood glucose detection chip can be used in the subsequent artificial neural network blood sugar detection.
  • the processor in the face recognition chip only needs to perform the relevant processing of full connection with the electrical signal and nonlinear activation. In this way, the power consumption and delay of artificial neural network face recognition can be greatly reduced.
  • the optical filter layer is used as the input layer of the artificial neural network
  • the image sensor is used as the linear layer of the artificial neural network
  • the filtering effect of the optical filter layer on the incident light entering the optical filter layer is used as the input.
  • the connection weight from the layer to the linear layer uses the optical filter layer and the image sensor to project the spatial spectral information of the human body to be measured into an electrical signal, and then realizes the full connection processing and nonlinear activation processing of the electrical signal in the processor.
  • the embodiment of the present application can not only omit the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art, but also the embodiment of the present application actually simultaneously utilizes the image information of the blood in the part to be measured, the Spectral information, angle information of incident light, and phase information of incident light, that is, the information carried by incident light at different points in the blood space of the part to be tested. It covers the image, composition, shape, three-dimensional depth, structure and other information of the blood in the body to be tested, so that when the identification processing is carried out according to the information carried by the incident light at different points of the blood in the body to be tested, it can cover the blood of the body to be tested. image, composition, shape, three-dimensional depth, structure and other multi-dimensional information, so as to accurately detect blood sugar.
  • the optical artificial neural network blood sugar detection chip includes a trained light modulation structure, an image sensor and a processor;
  • the trained light modulation structures, image sensors and processors refer to the use of input training samples and output training samples corresponding to the blood glucose detection task, which include different light modulation structures, image sensors, and different fully connected parameters and different parameters.
  • the optical modulation structure, the image sensor and the processor satisfying the training convergence condition are obtained by training the optical artificial neural network blood glucose detection chip of the processor of the nonlinear activation parameter;
  • the input training samples include incident light reflected, transmitted and/or radiated by the part to be measured on the human body with different blood sugar values; the output training samples include corresponding blood sugar values.
  • the different light modulation structures are obtained by using Design and implementation of computer optical simulation design.
  • an artificial neural network photoelectric chip whose input is the image of the body to be measured and its frequency spectrum can be realized, and fast, accurate, safe and reliable non-invasive blood glucose detection can be realized.
  • a large number of spectral signal data corresponding to human body parts with blood sugar information can be collected first, and a computer can be used to simulate the response of incident light through the micro-nano modulation structure.
  • the required micro-nano modulation structure is integrated above the image sensor.
  • the fast and accurate detection of the user's blood sugar level can be achieved by performing algorithm restoration on the electrical signals modulated by the incident light of different wavelengths.
  • the specific modulation pattern of the modulation structure on the optical filter layer is obtained by collecting a large number of spectral signal data corresponding to the human body parts with blood sugar information in the early stage, and is designed by artificial neural network data training, which is usually irregular in shape. Structures, of course, may also be regular-shaped structures.
  • the complete process for blood sugar detection is as follows: ambient light or other light sources illuminate the part to be measured (finger) of the human body, and then the reflected light is collected by the chip, and the blood sugar detection result is obtained after internal processing.
  • the complete process for blood sugar detection is: ambient light or other light sources illuminate the part to be measured (wrist) of the human body, and then the reflected light is collected by the chip, and the blood sugar detection result is obtained after internal processing.
  • the chip actually utilizes the image information and spectral information of the part to be measured in the human body at the same time, which improves the accuracy and safety of blood glucose detection.
  • the chip partially implements an artificial neural network on the hardware, which improves the speed of blood sugar detection.
  • the chip solution can use the existing CMOS process to achieve mass production, which reduces the size, power consumption and cost of the device.
  • the light modulation structures in the optical filter layer include regular structures and/or irregular structures; and/or, the light modulation structures in the optical filter layer include discrete structures and/or continuous structures.
  • the optical filter layer has a single-layer structure or a multi-layer structure.
  • the light modulation structure in the optical filter layer includes a unit array composed of a plurality of micro/nano units, each micro/nano unit corresponds to one or more pixels on the image sensor; the structures of each micro/nano unit are the same or different.
  • micro-nano unit includes a regular structure and/or an irregular structure; and/or, the micro-nano unit includes a discrete structure and/or a continuous structure.
  • micro-nano unit includes a plurality of groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.
  • each group of micro-nano structure arrays has the function of broadband filtering or narrow-band filtering.
  • each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.
  • micro-nano unit has polarization independent properties.
  • micro-nano unit has quadruple rotational symmetry.
  • optical filter layer is composed of one or more filter layers
  • the filter layer is made of one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, and perovskite materials; and/or, the filter layer is made of photonic crystals, metasurfaces, random A filter layer prepared by one or more of a structure, a nanostructure, a metal surface plasmon SPP micro-nano structure, and a tunable Fabry-Perot resonant cavity.
  • the semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and/or, the nanometer
  • the structure includes one or more of nanodot 2D materials, nanopillar 2D materials, and nanowire 2D materials.
  • the thickness of the optical filter layer is 0.1 ⁇ ⁇ 10 ⁇ , where ⁇ represents the center wavelength of the incident light.
  • Embodiments of the present application further provide an intelligent blood glucose detector, including the optical artificial neural network blood glucose detection chip described in the above embodiments, and the blood glucose detector may be a wearable blood glucose detector. Since the intelligent blood glucose detector has similar beneficial effects to the above-mentioned optical artificial neural network blood glucose detection chip, it will not be repeated here.
  • the embodiment of the present application also provides a preparation method of the above-mentioned optical artificial neural network blood glucose detection chip, including:
  • the optical filter layer is used to perform different spectral modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain different spectral modulations on the surface of the photosensitive region.
  • the image sensor is used to convert the incident light-carrying information corresponding to the different position points after being modulated by the optical filter layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processing
  • the processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different positions to obtain blood glucose detection results; the electrical signals are image signals modulated by the optical filter layer, and the incident The light includes reflected light and/or transmitted light of the body part to be measured.
  • the preparation method of the optical artificial neural network blood sugar detection chip also includes: a training process for the optical artificial neural network blood sugar detection chip, specifically including:
  • the optical artificial neural network blood glucose detection chip containing different light modulation structures, image sensors and processors with different fully connected parameters and different nonlinear activation parameters
  • the light modulation structure, image sensor and processor satisfying the training convergence condition are obtained by training, and the light modulation structure, image sensor and processor satisfying the training convergence condition are taken as the trained light modulation structure, image sensor and processor.
  • the different light modulation structures are obtained by using Design and implementation of computer optical simulation design.
  • the optical artificial neural network blood sugar detection chip based on the micro-nano modulation structure and the image sensor provided in this embodiment has the following effects: A.
  • the artificial neural network is partially embedded in the image sensor including various optical filter layers, Realize safe, reliable, fast and accurate non-invasive blood glucose detection.
  • the preparation of the chip can be completed by one tap-out of the CMOS process, which is beneficial to reduce the failure rate of the device, improve the yield of the device, and reduce the cost.
  • Monolithic integration at the wafer level can minimize the distance between the sensor and the optical filter layer, which is conducive to reducing the size of the unit, reducing device volume and packaging costs.
  • Precision agriculture is a system that implements a complete set of modern agricultural operation technology and management based on spatial variation, positioning, timing and quantitative supported by information technology.
  • the spatial variation of soil properties and productivity determine the production goals of crops, carry out “systematic diagnosis, optimized formula, technical assembly, scientific management” of positioning, mobilize soil productivity, and achieve the same income or Higher income, improve the environment, efficiently utilize various agricultural resources, and obtain economic and environmental benefits.
  • the technical principle of precision agriculture is to adjust the input to crops according to the spatial differences in soil fertility and crop growth conditions.
  • identification of soil fertility and crop growth conditions is inaccurate and there is a large power consumption and time delay, it will seriously affect The development of precision agriculture.
  • this embodiment provides a new type of optoelectronic chip for agricultural precision control intelligent processing tasks, the chip is composed of an optical filter layer to form the input layer of the optical artificial neural network As well as the connection weight from the input layer to the linear layer, the linear layer of the optical artificial neural network is formed by the image sensor.
  • the image sensor By collecting the image information and spectral information of the farmland soil and crops, it can achieve fast, accurate, safe and reliable soil fertility and pesticide distribution. , identification and qualitative analysis of trace element content and crop growth status. The content of this embodiment will be explained and described in detail below.
  • the embodiment of the present application also provides an optical artificial neural network intelligent agricultural precision control chip, which is used for agricultural precision control intelligent processing tasks, including: an optical filter layer, an image sensor and a processor; the optical filter layer corresponds to the artificial neural network.
  • the optical filter layer is disposed on the surface of the photosensitive region of the image sensor, the optical filter layer includes a light modulation structure, and the optical filter layer is used for entering the light through the light modulation structure pair
  • the incident light at different positions of the modulation structure is subjected to different spectral modulations respectively, so as to obtain incident light-carrying information corresponding to different positions on the surface of the photosensitive region;
  • the incident light-carrying information includes light intensity distribution information, spectral information , the angle information of the incident light and the phase information of the incident light;
  • the incident light includes reflected light, transmitted light and/or radiated light of an agricultural object;
  • the agricultural object includes crops and/or soil;
  • the image sensor is used to convert the incident light-carrying information corresponding to the different position points after being modulated by the optical filter layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processing
  • the electrical signal is an image signal modulated by the optical filter layer
  • the processor is used to perform full connection processing and nonlinear activation processing on electrical signals corresponding to different position points to obtain agricultural precision control processing results;
  • the agricultural precision control intelligent processing tasks include one or more of soil fertility detection, pesticide spreading detection, trace element content detection, drug resistance detection, and crop growth status detection;
  • the agricultural precision control processing results include: : One or more of soil fertility test results, pesticide spray test results, trace element content test results, drug resistance test results, and crop growth status test results.
  • this embodiment implements a brand-new optical artificial neural network intelligent agricultural precision control chip capable of realizing the artificial neural network function, which is used for the intelligent processing task of agricultural precision control.
  • the artificial neural network is embedded in the hardware chip in this embodiment of the present application.
  • network, the optical filter layer on the hardware chip is used as the input layer of the artificial neural network and the connection weight between the input layer and the linear layer, and the image sensor on the hardware chip is used as the linear layer of the artificial neural network.
  • the spatial spectral information of crops are incident into the pre-trained hardware chip, and the artificial neural network analysis is performed on the spatial spectral information of farmland soil and crops through the hardware chip to obtain the agricultural precision control processing result.
  • this application implements The example realizes the identification and qualitative analysis of low power consumption, safe and reliable fast and accurate soil fertility, pesticide application, trace element content and crop growth status.
  • the hardware structure on it - the optical filter layer corresponds to the input layer of the artificial neural network and the connection weight between the input layer and the linear layer, and the hardware structure on it.
  • the image sensor corresponds to the linear layer of the artificial neural network;
  • the processor corresponds to the nonlinear layer and the output layer of the artificial neural network.
  • the optical filter layer is disposed on the surface of the photosensitive area of the image sensor, the optical filter layer includes a light modulation structure, and the optical filter layer is used to pass the light modulation structure to the incident light entering different positions of the light modulation structure. Different spectral modulations are respectively performed to obtain incident light-carrying information corresponding to different positions on the surface of the photosensitive area.
  • the image sensor is used to convert the incident light-carrying information corresponding to different positions into different positions.
  • the processor connected to the image sensor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different positions to obtain the output signal of the artificial neural network.
  • the optical filter layer is used as the input layer of the artificial neural network
  • the image sensor is used as the linear layer of the artificial neural network.
  • the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the input
  • the connection weight from the layer to the linear layer, that is, the optical filter layer and the image sensor in the intelligent agricultural precision control chip realize the related functions of the input layer and the linear layer in the artificial neural network, that is, the embodiment of the present application combines the existing technology
  • the input layer and linear layer in the artificial neural network implemented by software are stripped, and the input layer and linear layer in the artificial neural network are realized by hardware, so that the subsequent use of this intelligent agricultural precision control
  • the chip performs artificial neural network agricultural precision control intelligent processing, it does not need to perform complex signal processing and algorithm processing corresponding to the input layer and linear layer. It only needs to be fully connected to the electrical signal by the processor in the intelligent agricultural precision control chip.
  • the relevant processing of nonlinear activation is enough, which can greatly reduce the power consumption and delay in the precise control of artificial neural network intelligent agriculture.
  • the optical filter layer is used as the input layer of the artificial neural network
  • the image sensor is used as the linear layer of the artificial neural network
  • the filtering effect of the optical filter layer on the incident light entering the optical filter layer is used as the input.
  • the connection weight from the layer to the linear layer uses the optical filter layer and the image sensor to project the spatial spectral information of the face into an electrical signal, and then realizes the full connection processing and nonlinear activation processing of the electrical signal in the processor.
  • the embodiments of the present application can not only save the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art, but also utilize the image information, spectral information, incident information of farmland soil and crops at the same time in the embodiments of the present application.
  • the angle information of the light and the phase information of the incident light that is, the information carried by the incident light at different points in the farmland soil and crop space. It can be seen that the information carried by the incident light at different points in the farmland soil and crop space covers the farmland soil and crops.
  • the image, composition, shape, three-dimensional depth, structure and other information of the farmland soil and crops can cover the image, composition, shape, three-dimensional depth of the farmland soil and crops when the identification and processing are carried out according to the information carried by the incident light at different points in the farmland soil and crop space. , structure and other multi-dimensional information, so that the identification and qualitative analysis of soil fertility, pesticide application, trace element content and crop growth status can be accurately carried out.
  • the optical artificial neural network intelligent agricultural precision control chip includes a trained light modulation structure, an image sensor and a processor;
  • the trained light modulation structures, image sensors and processors refer to the use of input training samples and output training samples corresponding to the agricultural precision control intelligent processing tasks, and the use of input training samples and output training samples corresponding to the agricultural precision control intelligent processing tasks.
  • the optical modulation structure, image sensor and processor that meet the training convergence conditions are obtained by training the optical artificial neural network intelligent agricultural precision control chip of the processor with full connection parameters and different nonlinear activation parameters;
  • the input training samples include incident light reflected, transmitted and/or radiated by soils with different fertility;
  • the output training samples include corresponding soil fertility;
  • the input training samples include incident light reflected, transmitted and/or radiated by soils with different pesticide application conditions; the output training samples include corresponding pesticide application conditions;
  • the input training samples include incident light reflected, transmitted and/or radiated by soils with different trace element content conditions; the output training samples include corresponding trace element content conditions;
  • the input training samples include incident light reflected, transmitted and/or radiated by soils with different drug resistances; the output training samples include corresponding drug resistances;
  • the input training samples include incident light reflected, transmitted and/or radiated by crops with different growth conditions; the output training samples include corresponding crop growth conditions.
  • the different light modulation structures It is designed and realized by adopting the computer optical simulation design method.
  • a large number of soil samples and crop samples in different states and locations can be collected first, and the weight from the input layer to the linear layer can be obtained through data training, that is, the system of the optical filter layer. function, the required optical filter layer can be reverse engineered to be integrated over the image sensor.
  • data training that is, the system of the optical filter layer.
  • the required optical filter layer can be reverse engineered to be integrated over the image sensor.
  • the weight of the fully connected layer of electrical signals is further trained and optimized, and a high-accuracy optical artificial neural network can be realized. Identification and qualitative analysis of soil fertility, pesticide application, trace element content and crop growth status.
  • the specific modulation pattern of the modulation structure on the optical filter layer is obtained by collecting a large number of soil samples and crop samples in different states and positions in the early stage, and is designed by artificial neural network data training, which is usually an irregular shape structure. , and of course there may also be regular-shaped structures.
  • the complete process of agricultural object recognition is: ambient light or other light sources illuminate the agricultural object, and then the reflected light is collected by the chip, and the recognition result is obtained after internal processing.
  • the spectral data and images of farmland soil and crops are collected by the spectrum and image collector, and then the reflected light is collected by the precision agriculture control chip, and then processed by the internal algorithm of the processor to obtain the identification result.
  • the chip actually uses the image information and spectral information of farmland soil and crops at the same time to achieve safe, reliable, fast and accurate precision agricultural control.
  • the chip solution can use the existing CMOS process to achieve mass production, which reduces the size, power consumption and cost of the device.
  • the light modulation structures in the optical filter layer include regular structures and/or irregular structures; and/or, the light modulation structures in the optical filter layer include discrete structures and/or continuous structures.
  • the optical filter layer has a single-layer structure or a multi-layer structure.
  • the light modulation structure in the optical filter layer includes a unit array composed of a plurality of micro/nano units, each micro/nano unit corresponds to one or more pixels on the image sensor; the structures of each micro/nano unit are the same or different.
  • micro-nano unit includes a regular structure and/or an irregular structure; and/or, the micro-nano unit includes a discrete structure and/or a continuous structure.
  • micro-nano unit includes a plurality of groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.
  • each group of micro-nano structure arrays has the function of broadband filtering or narrow-band filtering.
  • each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.
  • micro-nano unit has polarization independent properties.
  • micro-nano unit has quadruple rotational symmetry.
  • optical filter layer is composed of one or more filter layers
  • the filter layer is made of one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, and perovskite materials; and/or, the filter layer is made of photonic crystals, metasurfaces, random A filter layer prepared by one or more of a structure, a nanostructure, a metal surface plasmon SPP micro-nano structure, and a tunable Fabry-Perot resonant cavity.
  • the semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and/or, the nanometer
  • the structure includes one or more of nanodot 2D materials, nanopillar 2D materials, and nanowire 2D materials.
  • the thickness of the optical filter layer is 0.1 ⁇ ⁇ 10 ⁇ , where ⁇ represents the center wavelength of the incident light.
  • the embodiments of the present application also provide an intelligent agricultural control device, including: the optical artificial neural network intelligent agricultural precision control chip as described in the above embodiments.
  • the intelligent agricultural control equipment may include fertilizer applicator equipment, drug sprayer equipment, drug resistance analysis equipment, unmanned aerial vehicle equipment, agricultural intelligent robot equipment, crop health analysis equipment, crop growth state monitoring equipment, and the like.
  • Another embodiment of the present application provides a method for preparing an optical artificial neural network intelligent agricultural precision control chip according to the above-mentioned embodiment, including:
  • the optical filter layer is used to perform different spectral modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain different spectral modulations on the surface of the photosensitive region.
  • the image sensor is used to convert the incident light-carrying information corresponding to the different position points after being modulated by the optical filter layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processing
  • the processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different positions to obtain the agricultural precision control processing results; the electrical signals are image signals modulated by the optical filter layer, so
  • the incident light includes reflected light, transmitted light and/or radiated light of an agricultural object; the agricultural object includes crops and/or soil.
  • the preparation method of the optical artificial neural network intelligent agriculture precision control chip further includes: the training process of the optical artificial neural network intelligent agriculture precise control chip, specifically including:
  • the optical artificial neural network containing different light modulation structures, image sensors and processors with different fully connected parameters and different nonlinear activation parameters
  • the smart agriculture precision control chip is trained to obtain a light modulation structure, image sensor and processor that meet the training convergence conditions, and the light modulation structure, image sensor and processor that meet the training convergence conditions are used as the trained light modulation structure, image sensor and processor. processor.
  • the different light modulation structures It is designed and realized by adopting the computer optical simulation design method.
  • the optical artificial neural network intelligent agricultural precision control chip based on the micro-nano modulation structure and image sensor has the following effects: A.
  • the artificial neural network is partially embedded in the image sensor including various optical filter layers To achieve safe, reliable, fast and accurate precision agricultural control.
  • Artificial neural network training and recognition can be introduced into soil, crops, etc., which is convenient for subsequent integration into industrial intelligent control systems such as drones and intelligent robots to achieve precise agricultural control in large areas, with high recognition accuracy and accurate qualitative analysis.
  • the preparation of the chip can be completed in one tape-out through the CMOS process, which is beneficial to reduce the failure rate of the device, improve the yield of the device, and reduce the cost.
  • Monolithic integration at the wafer level can minimize the distance between the sensor and the optical filter layer, which is conducive to reducing the size of the unit, reducing device volume and packaging costs.
  • optical artificial neural network intelligent agricultural precision control chip it should be noted that, for the detailed description of the structure of the optical artificial neural network intelligent agricultural precision control chip provided in this embodiment, reference may be made to the introduction of the optical artificial neural network intelligent chip in the foregoing embodiment. introduce. In addition, for the detailed introduction of the optical artificial neural network intelligent agricultural precision control chip preparation method, you can also refer to the introduction of the optical artificial neural network intelligent chip preparation method in the foregoing embodiment, and will not be repeated here.
  • Converter steelmaking is currently the most widely used and efficient steelmaking method in the world.
  • the control of smelting end point is one of the key technologies in converter production.
  • the accurate judgment of the end point is of great significance in improving the quality of molten steel and shortening the smelting cycle.
  • Accurate online detection of end-point carbon content and molten steel temperature has always been an urgent problem to be solved in the metallurgical industry all over the world.
  • the end point control in the industry mainly relies on manual experience, or complex large-scale instruments and equipment to measure furnace mouth temperature and qualitative measurement of slag residues, with low accuracy and high cost.
  • this embodiment provides a new type of optoelectronic chip for monitoring the smelting end point, the chip is composed of an optical filter layer to form the input layer and the input layer of the optical artificial neural network For the connection weight to the linear layer, the linear layer of the optical artificial neural network is formed by the image sensor.
  • the embodiment of the present application also provides an optical artificial neural network smelting end point monitoring chip, which is used for the smelting end point monitoring task, including: an optical filter layer, an image sensor and a processor; the optical filter layer corresponds to the input of the artificial neural network layer and the connection weight from the input layer to the linear layer, the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the nonlinear layer and the output layer of the artificial neural network;
  • the optical filter layer is disposed on the surface of the photosensitive region of the image sensor, the optical filter layer includes a light modulation structure, and the optical filter layer is used for entering the light through the light modulation structure pair
  • the incident light at different positions of the modulation structure is subjected to different spectral modulations respectively, so as to obtain incident light-carrying information corresponding to different positions on the surface of the photosensitive region;
  • the incident light-carrying information includes light intensity distribution information, spectral information , the angle information of the incident light and the phase information of the incident light;
  • the incident light includes the reflected light, the transmitted light and/or the radiated light of the steelmaking furnace mouth;
  • the image sensor is used to convert the incident light-carrying information corresponding to the different position points after being modulated by the optical filter layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processing
  • the electrical signal is an image signal modulated by the optical filter layer
  • the processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different position points to obtain the monitoring result of the smelting end point;
  • the smelting end point monitoring task includes identifying the smelting end point, and the smelting end point monitoring result includes the smelting end point identification result.
  • this embodiment implements a brand-new optical artificial neural network smelting end point monitoring chip capable of realizing the function of artificial neural network, which is used for the smelting end point monitoring task.
  • the optical filter layer on the hardware chip is used as the input layer of the artificial neural network and the connection weight between the input layer and the linear layer, and the image sensor on the hardware chip is used as the linear layer of the artificial neural network.
  • the spatial spectral information is incident into the pre-trained hardware chip, and the artificial neural network analysis is performed on the spatial spectral information of the steelmaking furnace mouth through the hardware chip to obtain the identification result of the smelting end point. Power consumption, safe and reliable fast and accurate smelting end point identification.
  • the hardware structure on it - the optical filter layer corresponds to the input layer of the artificial neural network and the connection weight between the input layer and the linear layer, and the hardware structure on it -
  • the image sensor corresponds to the linear layer of the artificial neural network; the processor corresponds to the nonlinear layer and the output layer of the artificial neural network.
  • the optical filter layer is disposed on the surface of the photosensitive area of the image sensor, the optical filter layer includes a light modulation structure, and the optical filter layer is used to pass the light modulation structure to the incident light entering different positions of the light modulation structure.
  • the image sensor is used to convert the incident light-carrying information corresponding to different positions into different positions.
  • the processor connected to the image sensor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different positions to obtain the output signal of the artificial neural network. It can be seen that in the smelting end monitoring chip, the optical filter layer is used as the input layer of the artificial neural network, and the image sensor is used as the linear layer of the artificial neural network.
  • connection weight to the linear layer that is, the optical filter layer and the image sensor in the smelting end-point monitoring chip realize the related functions of the input layer and the linear layer in the artificial neural network, that is, the embodiment of the present application adopts the prior art method.
  • the input layer and linear layer in the artificial neural network implemented by software are stripped off, and the two-layer structure of the input layer and the linear layer in the artificial neural network is realized by means of hardware, so that the subsequent use of the smelting end-point monitoring chip for artificial Neural network smelting end point identification processing does not need to perform complex signal processing and algorithm processing corresponding to the input layer and linear layer, only the processor in the smelting end point monitoring chip needs to be fully connected with the electrical signal and nonlinear activation correlation It can be processed, which can greatly reduce the power consumption and delay in monitoring the smelting end point of the artificial neural network.
  • the optical filter layer is used as the input layer of the artificial neural network
  • the image sensor is used as the linear layer of the artificial neural network
  • the filtering effect of the optical filter layer on the incident light entering the optical filter layer is used as the input.
  • the connection weight from the layer to the linear layer uses the optical filter layer and the image sensor to project the spatial spectral information of the steelmaking furnace mouth into an electrical signal, and then realizes the full connection processing and nonlinear activation processing of the electrical signal in the processor.
  • the embodiments of the present application can not only omit the complex signal processing and algorithm processing corresponding to the input layer and the linear layer in the prior art, but also utilize the image information and spectral information of the steel-making furnace mouth at the same time in the embodiments of the present application.
  • the angle information of the incident light and the phase information of the incident light that is, the information carried by the incident light at different points in the space of the steel-making furnace mouth.
  • the information carried by the incident light at different points of the steel-making furnace mouth space covers the steel-making furnace mouth space.
  • the image, composition, shape, three-dimensional depth, structure and other information of the furnace mouth so that the image, composition, shape, Three-dimensional depth, structure and other multi-dimensional information, so that the smelting end point can be accurately identified.
  • the smelting end point monitoring task further includes identifying the carbon content and/or molten steel temperature during the smelting process, and the smelting end point monitoring result includes the identification result of the carbon content and/or molten steel temperature during the smelting process.
  • the optical artificial neural network smelting endpoint monitoring chip includes a trained light modulation structure, an image sensor and a processor;
  • the trained light modulation structures, image sensors and processors refer to using the input training samples and output training samples corresponding to the monitoring task of the smelting end point, to perform data analysis on the light modulation structures, image sensors, and fully-connected parameters including different light modulation structures, image sensors, and
  • the optical artificial neural network of the processor with different nonlinear activation parameters smelts the end point monitoring chip and obtains the optical modulation structure, the image sensor and the processor that satisfy the training convergence condition;
  • the input training sample includes the incident light reflected, transmitted and/or radiated from the steelmaking furnace mouth smelted to the end point and not smelted to the end point; the output training sample includes the determination result of whether the smelting to the end point is reached.
  • the input training sample also includes a steelmaking furnace port from smelting to different carbon contents and/or molten steel temperature. Reflected, transmitted and/or radiated incident light, the output training sample also includes the corresponding carbon content and/or molten steel temperature.
  • the different light modulation structures are passed through. It is designed and implemented by means of computer optical simulation design.
  • a large number of images and spectral information of the converter furnace mouth at the end point can be collected first, and the weight from the input layer to the linear layer is obtained through data training, that is, the system function of the optical filter layer.
  • the required optical filter layer can be reverse engineered to be integrated over the image sensor.
  • a high-accuracy optical artificial neural network can be realized. , to complete the rapid and accurate identification of the smelting end point.
  • the specific modulation pattern of the modulation structure on the optical filter layer is obtained by collecting a large number of converter furnace mouth images and spectral information at the end point in the early stage, and is designed by artificial neural network data training, which is usually an irregular shape structure. , and of course there may also be regular-shaped structures.
  • the complete process of identifying the steelmaking furnace mouth to determine whether it is the smelting end point is: ambient light or other light sources illuminate the steelmaking furnace mouth, and then the reflected light is collected by the chip, and the identification result is obtained after internal processing. .
  • the chip actually utilizes the image information and spectral information of the steelmaking furnace mouth at the same time, which improves the accuracy of the identification of the smelting end point.
  • the chip partially realizes the artificial neural network on the hardware, which improves the speed of smelting end point recognition.
  • the chip solution can use the existing CMOS process to achieve mass production, which reduces the size, power consumption and cost of the device.
  • the light modulation structures in the optical filter layer include regular structures and/or irregular structures; and/or, the light modulation structures in the optical filter layer include discrete structures and/or continuous structures.
  • the optical filter layer has a single-layer structure or a multi-layer structure.
  • the light modulation structure in the optical filter layer includes a unit array composed of a plurality of micro/nano units, each micro/nano unit corresponds to one or more pixels on the image sensor; the structures of each micro/nano unit are the same or different.
  • micro-nano unit includes a regular structure and/or an irregular structure; and/or, the micro-nano unit includes a discrete structure and/or a continuous structure.
  • micro-nano unit includes a plurality of groups of micro-nano structure arrays, and the structures of each group of micro-nano structure arrays are the same or different.
  • each group of micro-nano structure arrays has the function of broadband filtering or narrow-band filtering.
  • each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.
  • micro-nano unit has polarization independent properties.
  • micro-nano unit has quadruple rotational symmetry.
  • optical filter layer is composed of one or more filter layers
  • the filter layer is made of one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, and perovskite materials; and/or, the filter layer is made of photonic crystals, metasurfaces, random A filter layer prepared by one or more of a structure, a nanostructure, a metal surface plasmon SPP micro-nano structure, and a tunable Fabry-Perot resonant cavity.
  • the semiconductor material includes one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed in a preset ratio, and direct bandgap compound semiconductor materials; and/or, the nanometer
  • the structure includes one or more of nanodot 2D materials, nanopillar 2D materials, and nanowire 2D materials.
  • the thickness of the optical filter layer is 0.1 ⁇ ⁇ 10 ⁇ , where ⁇ represents the center wavelength of the incident light.
  • Embodiments of the present application also provide an intelligent smelting control device, including: the optical artificial neural network smelting end point monitoring chip as described in the above embodiments.
  • the intelligent smelting control device may include various devices related to smelting process control, which is not limited in this embodiment.
  • the intelligent smelting control device provided in this embodiment has all the beneficial effects of the optical artificial neural network smelting end-point monitoring chip described in the above-mentioned embodiment. Since the above-mentioned embodiment has already described this in detail, this embodiment will not repeat it. .
  • the embodiment of the present application also provides a preparation method for monitoring the smelting end point of the optical artificial neural network as described above, including:
  • the optical filter layer is used to perform different spectral modulations on the incident light entering different positions of the light modulation structure through the light modulation structure, so as to obtain different spectral modulations on the surface of the photosensitive region.
  • the image sensor is used to convert the incident light-carrying information corresponding to the different position points after being modulated by the optical filter layer into electrical signals corresponding to the different position points, and send the electrical signals corresponding to the different position points to the processing
  • the processor is used to perform full connection processing and nonlinear activation processing on the electrical signals corresponding to different positions to obtain the monitoring result of the smelting end point;
  • the electrical signal is an image signal modulated by an optical filter layer, and the Incident light includes reflected light, transmitted light and/or radiated light from the steelmaking furnace mouth.
  • the preparation method of the optical artificial neural network smelting endpoint monitoring chip further includes: a training process for the optical artificial neural network smelting endpoint monitoring chip, specifically including:
  • the smelting endpoints of optical artificial neural networks containing different light modulation structures, image sensors and processors with different fully connected parameters and different nonlinear activation parameters
  • the monitoring chip is trained to obtain a light modulation structure, an image sensor and a processor that meet the training convergence conditions, and the light modulation structure, image sensor and processor that meet the training convergence conditions are used as the trained light modulation structure, image sensor and processor.
  • the different light modulation structures are passed through It is designed and implemented by means of computer optical simulation design.
  • the optical artificial neural network smelting endpoint monitoring chip based on the micro-nano modulation structure and image sensor provided in this embodiment has the following effects: A.
  • the artificial neural network is partially embedded in the image sensor including various optical filter layers , to achieve safe, reliable, fast and accurate smelting endpoint control.
  • B. Detectable samples include but are not limited to the end-point control of converter steelmaking, and artificial neural network training is introduced to detect the temperature and material elements of the smelting furnace mouth, which is very easy to integrate with the back-end industrial control system, and has high recognition accuracy and accurate qualitative analysis. .
  • the preparation of the chip can be completed in one tape-out through the CMOS process, which is beneficial to reduce the failure rate of the device, improve the yield of the device, and reduce the cost.
  • Monolithic integration at the wafer level can minimize the distance between the sensor and the optical filter layer, which is conducive to reducing the size of the unit, reducing device volume and packaging costs.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Neurology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Image Analysis (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)
  • Solid State Image Pick-Up Elements (AREA)

Abstract

一种光人工神经网络智能芯片、智能处理设备及制备方法，将光滤波器层作为人工神经网络的输入层，将图像传感器作为人工神经网络的线性层，将光滤波器层对进入光滤波器层的入射光的滤波作用作为输入层到线性层的连接权重，从而智能芯片中的光滤波器层和图像传感器以硬件的方式实现了人工神经网络中输入层和线性层的相关功能，使得后续在使用该智能芯片进行智能处理时不需要再进行与输入层和线性层对应的复杂的信号处理和算法处理，大幅降低人工神经网络处理时的功耗和延时。由于同时利用了目标对象空间不同点处的图像信息、光谱信息、入射光角度信息和入射光相位信息，从而可以更加准确地实现对目标对象的智能处理。

Description

光人工神经网络智能芯片、智能处理设备及制备方法
相关申请的交叉引用
本申请要求于2021年02月08日提交的申请号为2021101728410，发明名称为“光人工神经网络智能芯片、智能处理设备及制备方法”的中国专利申请的优先权，其通过引用方式全部并入本文。
技术领域
本申请涉及人工智能技术领域，尤其涉及一种光人工神经网络智能芯片、智能处理设备及制备方法。
背景技术
现有的智能识别技术，通常需要先对人或物体成像，再经过图像的预处理、特征提取、特征匹配等步骤，实现对人或物体的识别。然而，只利用人或物体的二维图像信息难以保证识别的准确性，例如难以区分真实的人脸和人脸照片；并且，成像过程需要将光学信息转换为数字电子信号，再传输到计算机中进行后续的算法处理，大量数据的传输和处理造成了较大的功耗和延时。
发明内容
针对现有技术存在的问题，本申请实施例提供一种光人工神经网络智能芯片、智能处理设备及制备方法。
具体地，本申请实施例提供了如下技术方案：
第一方面，本申请实施例提供一种光人工神经网络智能芯片，包括：光滤波器层、图像传感器和处理器；所述光滤波器层对应人工神经网络的输入层以及输入层到线性层的连接权重，所述图像传感器对应人工神经网络的线性层；所述处理器对应人工神经网络的非线性层以及输出层；
所述光滤波器层设置于所述图像传感器的感光区域的表面，所述光滤波器层包含有光调制结构，所述光滤波器层用于通过所述光调制结构对进入至所述光调制结构不同位置点处的入射光分别进行不同的频谱调制，以 在所述感光区域的表面得到与不同位置点对应的入射光携带信息；所述入射光携带信息包括光强度分布信息、光谱信息、所述入射光的角度信息以及所述入射光的相位信息
所述图像传感器用于将与不同位置点经光滤波器层调制后对应的入射光携带信息转换为与不同位置点对应的电信号，并将与不同位置点对应的电信号发送给所述处理器；所述电信号为经光滤波器层调制后的图像信号；
所述处理器用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到人工神经网络的输出信号。
进一步地，所述光人工神经网络智能芯片用于目标对象的智能处理任务；所述智能处理任务至少包括智能感知、智能识别和智能决策任务中的一种或多种；
目标对象的反射光、透射光和/或辐射光进入至训练好的光人工神经网络智能芯片中，得到所述目标对象的智能处理结果；所述智能处理结果至少包括智能感知结果、智能识别结果和/或智能决策结果中的一种或多种；
其中，训练好的光人工神经网络智能芯片是指包括训练好的光调制结构、图像传感器和处理器的光人工神经网络智能芯片；所述训练好的光调制结构、图像传感器和处理器是指利用与所述智能处理任务对应的输入训练样本和输出训练样本，对包含不同的光调制结构、图像传感器和具有不同的全连接参数与非线性激活参数的处理器的光人工神经网络智能芯片进行训练得到的满足训练收敛条件的光调制结构、图像传感器和处理器。
进一步地，在对包含不同的光调制结构、图像传感器和具有不同的全连接参数和非线性激活参数的处理器的光人工神经网络智能芯片进行训练时，所述不同的光调制结构通过采用计算机光学仿真设计的方式设计实现。
进一步地，所述光滤波器层中的光调制结构包含规则结构和/或不规则结构；和/或，所述光滤波器层中的光调制结构包含离散型结构和/或连续型结构。
进一步地，所述光滤波器层为单层结构或多层结构。
进一步地，所述光滤波器层中的光调制结构包含由多个微纳单元组成 的单元阵列，每个微纳单元对应图像传感器上的一个或多个像素点；各个微纳单元的结构相同或不同。
进一步地，所述微纳单元包含规则结构和/或不规则结构；和/或，所述微纳单元包含离散型结构和/或连续型结构。
进一步地，所述微纳单元包含有多组微纳结构阵列，各组微纳结构阵列的结构相同或不同。
进一步地，各组微纳结构阵列具有宽带滤波或窄带滤波的作用。
进一步地，各组微纳结构阵列为周期结构阵列或非周期结构阵列。
进一步地，所述微纳单元包含的多组微纳结构阵列中有一组或多组空结构。
进一步地，所述微纳单元具有偏振无关特性。
进一步地，所述微纳单元具有四重旋转对称性。
进一步地，所述光滤波器层由一层或多层滤波器层构成；
所述滤波器层是由半导体材料、金属材料、液晶、量子点材料、钙钛矿材料中的一种或多种制备；和/或，所述滤波器层是由光子晶体、超表面、随机结构、纳米结构、金属表面等离激元SPP微纳结构、可调法布里-珀罗谐振腔中的一种或多种制备的滤波器层。
进一步地，所述半导体材料包括硅、氧化硅、氮化硅、氧化钛、按照预设比例混合的复合材料以及直接带隙化合物半导体材料中的一种或多种；和/或，所述纳米结构包括纳米点二维材料、纳米柱二维材料和纳米线二维材料中的一种或多种。
进一步地，所述光滤波器层的厚度为0.1λ～10λ，其中λ表示入射光的中心波长
进一步地，所述图像传感器为下述中的任意一项或多项：
CMOS图像传感器CIS、电荷耦合元件CCD、单光子雪崩二极管SPAD阵列和焦平面光电探测器阵列。
进一步地，所述人工神经网络的类型包括：前馈神经网络。
进一步地，所述光滤波器层与所述图像传感器之前设置有透光介质层。
进一步地，所述图像传感器为前照式，包括：自上而下设置的金属线 层和光探测层，所述光滤波器层集成在所述金属线层远离所述光探测层的一面；或，
所述图像传感器为背照式，包括：自上而下设置的光探测层和金属线层，所述光滤波器层集成在所述光探测层远离所述金属线层的一面。
第二方面，本申请实施例提供一种智能处理设备，包括：如第一方面所述的光人工神经网络智能芯片。
进一步地，所述智能处理设备包括智能手机、智能电脑、智能识别设备、智能感知设备和智能决策设备中的一种或多种。
第三方面，本申请实施例提供光人工神经网络智能芯片的制备方法，包括：
在所述图像传感器的感光区域的表面制备包含有光调制结构的光滤波器层；
生成具备对信号进行全连接处理与非线性激活处理功能的处理器；
连接所述图像传感器和所述处理器；
其中，所述光滤波器层用于通过所述光调制结构对进入至所述光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息；所述入射光携带信息包括光强度分布信息、光谱信息、所述入射光的角度信息以及所述入射光的相位信息；
所述图像传感器用于将与不同位置点经光滤波器层调制后对应的入射光携带信息转换为与不同位置点对应的电信号，并将与不同位置点对应的电信号发送给所述处理器；所述处理器用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到人工神经网络的输出信号；所述电信号为经光滤波器层调制后的图像信号。
进一步地，在所述图像传感器的感光区域的表面制备包含有光调制结构的光滤波器层，包括：
在所述图像传感器的感光区域的表面生长一层或多层预设材料；
对所述一层或多层预设材料进行光调制结构图案的刻蚀，得到包含有光调制结构的光滤波器层；
或对所述一层或多层预设材料进行压印转移，得到包含有光调制结构 的光滤波器层；
或通过对所述一层或多层预设材料进行外加动态调制，得到包含有光调制结构的光滤波器层；
或对所述一层或多层预设材料进行分区打印，得到包含有光调制结构的光滤波器层；
或对所述一层或多层预设材料进行分区生长，得到包含有光调制结构的光滤波器层；
或对所述一层或多层预设材料进行量子点转移，得到包含有光调制结构的光滤波器层。
进一步地，当所述光人工神经网络智能芯片用于目标对象的智能处理任务时，利用与所述智能处理任务对应的输入训练样本和输出训练样本，对包含不同的光调制结构、图像传感器和具有不同的全连接参数和非线性激活参数的处理器的光人工神经网络智能芯片进行训练，得到满足训练收敛条件的光调制结构、图像传感器和处理器。
本申请实施例还提供了一种光人工神经网络人脸识别芯片，用于人脸识别处理任务，包括：光滤波器层、图像传感器和处理器；所述光滤波器层对应人工神经网络的输入层以及输入层到线性层的连接权重，所述图像传感器对应人工神经网络的线性层；所述处理器对应人工神经网络的非线性层以及输出层；
所述光滤波器层设置于所述图像传感器的感光区域的表面，所述光滤波器层包含有光调制结构，所述光滤波器层用于通过所述光调制结构对进入至所述光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息；所述入射光携带信息包括光强度分布信息、光谱信息、所述入射光的角度信息以及所述入射光的相位信息；所述入射光为用户人脸的反射光；
所述图像传感器用于将与不同位置点经光滤波器层调制后对应的入射光携带信息转换为与不同位置点对应的电信号，并将与不同位置点对应的电信号发送给所述处理器；所述电信号为经光滤波器层调制后的图像信号；
所述处理器用于将与不同位置点对应的电信号进行全连接处理与非 线性激活处理，得到人脸识别处理结果。
进一步地，所述光人工神经网络人脸识别芯片包括训练好的光调制结构、图像传感器和处理器；
所述训练好的光调制结构、图像传感器和处理器是指利用与人脸识别任务对应的输入训练样本以及输出训练样本，对包含不同的光调制结构、图像传感器和具有不同的全连接参数以及不同的非线性激活参数的处理器的光人工神经网络人脸识别芯片进行训练得到的满足训练收敛条件的光调制结构、图像传感器和处理器；
所述输入训练样本为在不同光照环境下由不同人脸反射的入射光；所述输出训练样本包括相应的人脸识别结果。
进一步地，在对包含不同的光调制结构、图像传感器和具有不同的全连接参数和非线性激活参数的处理器的光人工神经网络人脸识别芯片进行训练时，所述不同的光调制结构通过采用计算机光学仿真设计的方式设计实现。
进一步地，所述光滤波器层中的光调制结构包含规则结构和/或不规则结构；和/或，所述光滤波器层中的光调制结构包含离散型结构和/或连续型结构。
进一步地，所述光滤波器层为单层结构或多层结构。
进一步地，所述光滤波器层中的光调制结构包含由多个微纳单元组成的单元阵列，每个微纳单元对应图像传感器上的一个或多个像素点；各个微纳单元的结构相同或不同。
进一步地，所述微纳单元包含规则结构和/或不规则结构；和/或，所述微纳单元包含离散型结构和/或连续型结构。
进一步地，所述微纳单元包含有多组微纳结构阵列，各组微纳结构阵列的结构相同或不同。
进一步地，各组微纳结构阵列具有宽带滤波或窄带滤波的作用。
进一步地，各组微纳结构阵列为周期结构阵列或非周期结构阵列。
进一步地，所述微纳单元包含的多组微纳结构阵列中有一组或多组空结构。
进一步地，所述微纳单元具有偏振无关特性。
进一步地，所述偏振无关的微纳单元具有四重旋转对称性。
进一步地，所述光滤波器层由一层或多层滤波器层构成；
所述滤波器层是由半导体材料、金属材料、液晶、量子点材料、钙钛矿材料中的一种或多种制备；和/或，所述滤波器层是由光子晶体、超表面、随机结构、纳米结构、金属表面等离激元SPP微纳结构、可调法布里-珀罗谐振腔中的一种或多种制备的滤波器层。
进一步地，所述半导体材料包括硅、氧化硅、氮化硅、氧化钛、按照预设比例混合的复合材料以及直接带隙化合物半导体材料中的一种或多种；和/或，所述纳米结构包括纳米点二维材料、纳米柱二维材料和纳米线二维材料中的一种或多种。
进一步地，所述光滤波器层的厚度为0.1λ～10λ，其中λ表示入射光的中心波长。
本申请实施例还提供了一种人脸识别设备，包括如上面所述的光人工神经网络人脸识别芯片。
本申请实施例还提供了一种如上面所述的光人工神经网络人脸识别芯片的制备方法，包括：
在所述图像传感器的感光区域的表面制备包含有光调制结构的光滤波器层；
生成具备对信号进行全连接处理与非线性激活处理功能的处理器；
连接所述图像传感器和所述处理器；
其中，所述光滤波器层用于通过所述光调制结构对进入至所述光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息；所述入射光携带信息包括光强度分布信息、光谱信息、所述入射光的角度信息以及所述入射光的相位信息；
所述图像传感器用于将与不同位置点经光滤波器层调制后对应的入射光携带信息转换为与不同位置点对应的电信号，并将与不同位置点对应的电信号发送给所述处理器；所述处理器用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到人脸识别处理结果；所述电信号为经光滤波器层调制后的图像信号，所述入射光为不同光照环境下的人 脸的反射光。
进一步地，所述光人工神经网络人脸识别芯片的制备方法，还包括：对所述光人工神经网络人脸识别芯片的训练过程，具体包括：
利用与人脸识别任务对应的输入训练样本和输出训练样本，对包含不同的光调制结构、图像传感器和具有不同的全连接参数以及不同的非线性激活参数的处理器的光人工神经网络人脸识别芯片进行训练得到满足训练收敛条件的光调制结构、图像传感器和处理器，并将满足训练收敛条件的光调制结构、图像传感器和处理器作为训练好的光调制结构、图像传感器和处理器。
进一步地，在对包含不同的光调制结构、图像传感器和具有不同的全连接参数和非线性激活参数的处理器的光人工神经网络人脸识别芯片进行训练时，所述不同的光调制结构通过采用计算机光学仿真设计的方式设计实现。
本申请实施例还提供了一种光人工神经网络血糖检测芯片，用于血糖检测任务，包括：光滤波器层、图像传感器和处理器；所述光滤波器层对应人工神经网络的输入层以及输入层到线性层的连接权重，所述图像传感器对应人工神经网络的线性层；所述处理器对应人工神经网络的非线性层以及输出层；
所述光滤波器层设置于所述图像传感器的感光区域的表面，所述光滤波器层包含有光调制结构，所述光滤波器层用于通过所述光调制结构对进入至所述光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息；所述入射光携带信息包括光强度分布信息、光谱信息、所述入射光的角度信息以及所述入射光的相位信息；其中，所述入射光包括人体待测部位的反射光和/或透射光；
所述图像传感器用于将与不同位置点经光滤波器层调制后对应的入射光携带信息转换为与不同位置点对应的电信号，并将与不同位置点对应的电信号发送给所述处理器；所述电信号为经光滤波器层调制后的图像信号；
所述处理器用于将与不同位置点对应的电信号进行全连接处理与非 线性激活处理，得到血糖检测结果。
进一步地，所述光人工神经网络血糖检测芯片包括训练好的光调制结构、图像传感器和处理器；
所述训练好的光调制结构、图像传感器和处理器是指利用与血糖检测任务对应的输入训练样本以及输出训练样本，对包含不同的光调制结构、图像传感器和具有不同的全连接参数以及不同的非线性激活参数的处理器的光人工神经网络血糖检测芯片进行训练得到的满足训练收敛条件的光调制结构、图像传感器和处理器；
所述输入训练样本包括由具有不同血糖值的人体待测部位反射、透射的入射光；所述输出训练样本包括相应的血糖值。
进一步地，在对包含不同的光调制结构、图像传感器和具有不同的全连接参数和非线性激活参数的处理器的光人工神经网络血糖检测芯片进行训练时，所述不同的光调制结构通过采用计算机光学仿真设计的方式设计实现。
进一步地，所述光滤波器层中的光调制结构包含规则结构和/或不规则结构；和/或，所述光滤波器层中的光调制结构包含离散型结构和/或连续型结构。
进一步地，所述光滤波器层为单层结构或多层结构。
进一步地，所述光滤波器层中的光调制结构包含由多个微纳单元组成的单元阵列，每个微纳单元对应图像传感器上的一个或多个像素点；各个微纳单元的结构相同或不同。
进一步地，所述微纳单元包含规则结构和/或不规则结构；和/或，所述微纳单元包含离散型结构和/或连续型结构。
进一步地，所述微纳单元包含有多组微纳结构阵列，各组微纳结构阵列的结构相同或不同。
进一步地，各组微纳结构阵列具有宽带滤波或窄带滤波的作用。
进一步地，各组微纳结构阵列为周期结构阵列或非周期结构阵列。
进一步地，所述微纳单元包含的多组微纳结构阵列中有一组或多组空结构。
进一步地，所述微纳单元具有偏振无关特性。
进一步地，所述偏振无关微纳单元具有四重旋转对称性。
进一步地，所述光滤波器层由一层或多层滤波器层构成；
所述滤波器层是由半导体材料、金属材料、液晶、量子点材料、钙钛矿材料中的一种或多种制备；和/或，所述滤波器层是由光子晶体、超表面、随机结构、纳米结构、金属表面等离激元SPP微纳结构、可调法布里-珀罗谐振腔中的一种或多种制备的滤波器层。
进一步地，所述半导体材料包括硅、氧化硅、氮化硅、氧化钛、按照预设比例混合的复合材料以及直接带隙化合物半导体材料中的一种或多种；和/或，所述纳米结构包括纳米点二维材料、纳米柱二维材料和纳米线二维材料中的一种或多种。
进一步地，所述光滤波器层的厚度为0.1λ～10λ，其中λ表示入射光的中心波长。
本申请实施例还提供了一种智能血糖检测仪，包括如上面所述的光人工神经网络血糖检测芯片。
本申请实施例还提供了一种如上面所述的光人工神经网络血糖检测芯片的制备方法，包括：
在所述图像传感器的感光区域的表面制备包含有光调制结构的光滤波器层；
生成具备对信号进行全连接处理与非线性激活处理功能的处理器；
连接所述图像传感器和所述处理器；
其中，所述光滤波器层用于通过所述光调制结构对进入至所述光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息；所述入射光携带信息包括光强度分布信息、光谱信息、所述入射光的角度信息以及所述入射光的相位信息；
所述图像传感器用于将与不同位置点经光滤波器层调制后对应的入射光携带信息转换为与不同位置点对应的电信号，并将与不同位置点对应的电信号发送给所述处理器；所述处理器用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到血糖检测结果；所述电信号为经光滤波器层调制后的图像信号，所述入射光包括人体待测部位的反射光 和/或透射光。
进一步地，所述光人工神经网络血糖检测芯片的制备方法，还包括：对所述光人工神经网络血糖检测芯片的训练过程，具体包括：
利用与血糖检测任务对应的输入训练样本和输出训练样本，对包含不同的光调制结构、图像传感器和具有不同的全连接参数以及不同的非线性激活参数的处理器的光人工神经网络血糖检测芯片进行训练得到满足训练收敛条件的光调制结构、图像传感器和处理器，并将满足训练收敛条件的光调制结构、图像传感器和处理器作为训练好的光调制结构、图像传感器和处理器。
进一步地，在对包含不同的光调制结构、图像传感器和具有不同的全连接参数和非线性激活参数的处理器的光人工神经网络血糖检测芯片进行训练时，所述不同的光调制结构通过采用计算机光学仿真设计的方式设计实现。
本申请实施例还提供了一种光人工神经网络智能农业精准控制芯片，用于农业精准控制智能处理任务，包括：光滤波器层、图像传感器和处理器；所述光滤波器层对应人工神经网络的输入层以及输入层到线性层的连接权重，所述图像传感器对应人工神经网络的线性层；所述处理器对应人工神经网络的非线性层以及输出层；
所述光滤波器层设置于所述图像传感器的感光区域的表面，所述光滤波器层包含有光调制结构，所述光滤波器层用于通过所述光调制结构对进入至所述光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息；所述入射光携带信息包括光强度分布信息、光谱信息、所述入射光的角度信息以及所述入射光的相位信息；所述入射光包括农业对象的反射光、透射光和/或辐射光；所述农业对象包括农作物和/或土壤；
所述图像传感器用于将与不同位置点经光滤波器层调制后对应的入射光携带信息转换为与不同位置点对应的电信号，并将与不同位置点对应的电信号发送给所述处理器；所述电信号为经光滤波器层调制后的图像信号；
所述处理器用于将与不同位置点对应的电信号进行全连接处理与非 线性激活处理，得到农业精准控制处理结果；
其中，所述农业精准控制智能处理任务包括：土壤肥力检测、农药布撒检测、微量元素含量检测、抗药性检测以及农作物生长状况检测中的一种或多种；所述农业精准控制处理结果包括：土壤肥力检测结果、农药布撒检测结果、微量元素含量检测结果、抗药性检测结果以及农作物生长状况检测结果中的一种或多种。
进一步地，所述光人工神经网络智能农业精准控制芯片包括训练好的光调制结构、图像传感器和处理器；
所述训练好的光调制结构、图像传感器和处理器是指利用与所述农业精准控制智能处理任务对应的输入训练样本以及输出训练样本，对包含不同的光调制结构、图像传感器和具有不同的全连接参数以及不同的非线性激活参数的处理器的光人工神经网络智能农业精准控制芯片进行训练得到的满足训练收敛条件的光调制结构、图像传感器和处理器；
其中，所述输入训练样本包括由具备不同肥力的土壤反射、透射和/或辐射的入射光；所述输出训练样本包括相应的土壤肥力；
和/或，
所述输入训练样本包括由具备不同农药布撒状况的土壤反射、透射和/或辐射的入射光；所述输出训练样本包括相应的农药布撒状况；
和/或，
所述输入训练样本包括由具备不同微量元素含量状况的土壤反射、透射和/或辐射的入射光；所述输出训练样本包括相应的微量元素含量状况；
和/或，
所述输入训练样本包括由具备不同抗药性的土壤反射、透射和/或辐射的入射光；所述输出训练样本包括相应的抗药性；
和/或，
所述输入训练样本包括由具备不同生长状况的农作物反射、透射和/或辐射的入射光；所述输出训练样本包括相应的农作物生长状况。
进一步地，在对包含不同的光调制结构、图像传感器和具有不同的全连接参数和非线性激活参数的处理器的光人工神经网络智能农业精准控制芯片进行训练时，所述不同的光调制结构通过采用计算机光学仿真设计 的方式设计实现。
进一步地，所述光滤波器层中的光调制结构包含规则结构和/或不规则结构；和/或，所述光滤波器层中的光调制结构包含离散型结构和/或连续型结构。
进一步地，所述光滤波器层为单层结构或多层结构。
进一步地，所述光滤波器层中的光调制结构包含由多个微纳单元组成的单元阵列，每个微纳单元对应图像传感器上的一个或多个像素点；各个微纳单元的结构相同或不同。
进一步地，所述微纳单元包含规则结构和/或不规则结构；和/或，所述微纳单元包含离散型结构和/或连续型结构。
进一步地，所述微纳单元包含有多组微纳结构阵列，各组微纳结构阵列的结构相同或不同。
进一步地，各组微纳结构阵列具有宽带滤波或窄带滤波的作用。
进一步地，各组微纳结构阵列为周期结构阵列或非周期结构阵列。
进一步地，所述微纳单元包含的多组微纳结构阵列中有一组或多组空结构。
进一步地，所述微纳单元具有偏振无关特性。
进一步地，所述微纳单元具有四重旋转对称性。
进一步地，所述光滤波器层由一层或多层滤波器层构成；
所述滤波器层是由半导体材料、金属材料、液晶、量子点材料、钙钛矿材料中的一种或多种制备；和/或，所述滤波器层是由光子晶体、超表面、随机结构、纳米结构、金属表面等离激元SPP微纳结构、可调法布里-珀罗谐振腔中的一种或多种制备的滤波器层。
进一步地，所述半导体材料包括硅、氧化硅、氮化硅、氧化钛、按照预设比例混合的复合材料以及直接带隙化合物半导体材料中的一种或多种；和/或，所述纳米结构包括纳米点二维材料、纳米柱二维材料和纳米线二维材料中的一种或多种。
进一步地，所述光滤波器层的厚度为0.1λ～10λ，其中λ表示入射光的中心波长。
本申请实施例还提供了一种智能农业控制设备，包括如上所述的光人 工神经网络智能农业精准控制芯片。
本申请实施例还提供了一种如上面所述的光人工神经网络智能农业精准控制芯片的制备方法，包括：
在所述图像传感器的感光区域的表面制备包含有光调制结构的光滤波器层；
生成具备对信号进行全连接处理与非线性激活处理功能的处理器；
连接所述图像传感器和所述处理器；
其中，所述光滤波器层用于通过所述光调制结构对进入至所述光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息；所述入射光携带信息包括光强度分布信息、光谱信息、所述入射光的角度信息以及所述入射光的相位信息；
所述图像传感器用于将与不同位置点经光滤波器层调制后对应的入射光携带信息转换为与不同位置点对应的电信号，并将与不同位置点对应的电信号发送给所述处理器；所述处理器用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到农业精准控制处理结果；所述电信号为经光滤波器层调制后的图像信号，所述入射光包括农业对象的反射光、透射光和/或辐射光；所述农业对象包括农作物和/或土壤。
进一步地，所述光人工神经网络智能农业精准控制芯片的制备方法，还包括：对所述光人工神经网络智能农业精准控制芯片的训练过程，具体包括：
利用与农业精准控制智能处理任务对应的输入训练样本和输出训练样本，对包含不同的光调制结构、图像传感器和具有不同的全连接参数以及不同的非线性激活参数的处理器的光人工神经网络智能农业精准控制芯片进行训练得到满足训练收敛条件的光调制结构、图像传感器和处理器，并将满足训练收敛条件的光调制结构、图像传感器和处理器作为训练好的光调制结构、图像传感器和处理器。
进一步地，在对包含不同的光调制结构、图像传感器和具有不同的全连接参数和非线性激活参数的处理器的光人工神经网络智能农业精准控制芯片进行训练时，所述不同的光调制结构通过采用计算机光学仿真设计 的方式设计实现。
本申请实施例还提供了一种光人工神经网络冶炼终点监测芯片，用于冶炼终点监测任务，包括：光滤波器层、图像传感器和处理器；所述光滤波器层对应人工神经网络的输入层以及输入层到线性层的连接权重，所述图像传感器对应人工神经网络的线性层；所述处理器对应人工神经网络的非线性层以及输出层；
所述光滤波器层设置于所述图像传感器的感光区域的表面，所述光滤波器层包含有光调制结构，所述光滤波器层用于通过所述光调制结构对进入至所述光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息；所述入射光携带信息包括光强度分布信息、光谱信息、所述入射光的角度信息以及所述入射光的相位信息；所述入射光包括炼钢炉口的反射光、透射光和/或辐射光；
所述图像传感器用于将与不同位置点经光滤波器层调制后对应的入射光携带信息转换为与不同位置点对应的电信号，并将与不同位置点对应的电信号发送给所述处理器；所述电信号为经光滤波器层调制后的图像信号；
所述处理器用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到冶炼终点监测结果；
其中，所述冶炼终点监测任务包括识别冶炼终点，所述冶炼终点监测结果包括冶炼终点识别结果。
进一步地，所述冶炼终点监测任务还包括识别冶炼过程中的碳含量和/钢水温度，所述冶炼终点监测结果包括冶炼过程中的碳含量和/钢水温度识别结果。
进一步地，所述光人工神经网络冶炼终点监测芯片包括训练好的光调制结构、图像传感器和处理器；
所述训练好的光调制结构、图像传感器和处理器是指利用与冶炼终点监测任务对应的输入训练样本以及输出训练样本，对包含不同的光调制结构、图像传感器和具有不同的全连接参数以及不同的非线性激活参数的处理器的光人工神经网络冶炼终点监测芯片进行训练得到的满足训练收敛 条件的光调制结构、图像传感器和处理器；
所述输入训练样本包括由冶炼至终点以及未冶炼至终点的炼钢炉口反射、透射和/或辐射的入射光；所述输出训练样本包括是否冶炼至终点的判定结果。
进一步地，当所述冶炼终点监测任务还包括识别冶炼过程中的碳含量和/钢水温度时，相应地，所述输入训练样本还包括由冶炼至不同碳含量和/钢水温度的炼钢炉口反射、透射和/或辐射的入射光，所述输出训练样本还包括相应的碳含量和/钢水温度。
进一步地，在对包含不同的光调制结构、图像传感器和具有不同的全连接参数和非线性激活参数的处理器的光人工神经网络冶炼终点监测芯片进行训练时，所述不同的光调制结构通过采用计算机光学仿真设计的方式设计实现。
进一步地，所述光滤波器层中的光调制结构包含规则结构和/或不规则结构；和/或，所述光滤波器层中的光调制结构包含离散型结构和/或连续型结构。
进一步地，所述光滤波器层为单层结构或多层结构。
进一步地，所述光滤波器层中的光调制结构包含由多个微纳单元组成的单元阵列，每个微纳单元对应图像传感器上的一个或多个像素点；各个微纳单元的结构相同或不同。
进一步地，所述微纳单元包含规则结构和/或不规则结构；和/或，所述微纳单元包含离散型结构和/或连续型结构。
进一步地，所述微纳单元包含有多组微纳结构阵列，各组微纳结构阵列的结构相同或不同。
进一步地，各组微纳结构阵列具有宽带滤波或窄带滤波的作用。
进一步地，各组微纳结构阵列为周期结构阵列或非周期结构阵列。
进一步地，所述微纳单元包含的多组微纳结构阵列中有一组或多组空结构。
进一步地，所述微纳单元具有偏振无关特性。
进一步地，所述微纳单元具有四重旋转对称性。
进一步地，所述光滤波器层由一层或多层滤波器层构成；
所述滤波器层是由半导体材料、金属材料、液晶、量子点材料、钙钛矿材料中的一种或多种制备；和/或，所述滤波器层是由光子晶体、超表面、随机结构、纳米结构、金属表面等离激元SPP微纳结构、可调法布里-珀罗谐振腔中的一种或多种制备的滤波器层。
进一步地，所述半导体材料包括硅、氧化硅、氮化硅、氧化钛、按照预设比例混合的复合材料以及直接带隙化合物半导体材料中的一种或多种；和/或，所述纳米结构包括纳米点二维材料、纳米柱二维材料和纳米线二维材料中的一种或多种。
进一步地，所述光滤波器层的厚度为0.1λ～10λ，其中λ表示入射光的中心波长。
本申请实施例还提供了智能冶炼控制设备，其特征在于，包括如上所述的光人工神经网络冶炼终点监测芯片。
本申请实施例还提供了一种如上面所述的光人工神经网络冶炼终点监测的制备方法，包括：
在所述图像传感器的感光区域的表面制备包含有光调制结构的光滤波器层；
生成具备对信号进行全连接处理与非线性激活处理功能的处理器；
连接所述图像传感器和所述处理器；
其中，所述光滤波器层用于通过所述光调制结构对进入至所述光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息；所述入射光携带信息包括光强度分布信息、光谱信息、所述入射光的角度信息以及所述入射光的相位信息；
所述图像传感器用于将与不同位置点经光滤波器层调制后对应的入射光携带信息转换为与不同位置点对应的电信号，并将与不同位置点对应的电信号发送给所述处理器；所述处理器用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到冶炼终点监测结果；所述电信号为经光滤波器层调制后的图像信号，所述入射光包括炼钢炉口的反射光、透射光和/或辐射光。
进一步地，所述光人工神经网络冶炼终点监测芯片的制备方法，还包 括：对所述光人工神经网络冶炼终点监测芯片的训练过程，具体包括：
利用与冶炼终点监测任务对应的输入训练样本和输出训练样本，对包含不同的光调制结构、图像传感器和具有不同的全连接参数以及不同的非线性激活参数的处理器的光人工神经网络冶炼终点监测芯片进行训练得到满足训练收敛条件的光调制结构、图像传感器和处理器，并将满足训练收敛条件的光调制结构、图像传感器和处理器作为训练好的光调制结构、图像传感器和处理器。
进一步地，在对包含不同的光调制结构、图像传感器和具有不同的全连接参数和非线性激活参数的处理器的光人工神经网络冶炼终点监测芯片进行训练时，所述不同的光调制结构通过采用计算机光学仿真设计的方式设计实现。
本申请实施例提供的光人工神经网络智能芯片、智能处理设备及制备方法，实现了一种能够实现人工神经网络功能的全新智能芯片，在该智能芯片中，光滤波器层设置于图像传感器的感光区域的表面，光滤波器层包含有光调制结构，光滤波器层用于通过光调制结构对进入至光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息，相应地，图像传感器用于将与不同位置点对应的入射光携带信息转换为与不同位置点对应的电信号，与此同时，与图像传感器连接的处理器用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到人工神经网络的输出信号，由此可见，在该智能芯片中，光滤波器层作为人工神经网络的输入层，图像传感器作为人工神经网络的线性层，同时，光滤波器层对进入光滤波器层的入射光的滤波作用对应输入层到线性层的连接权重，也即该智能芯片中的光滤波器层和图像传感器实现了人工神经网络中输入层和线性层的相关功能，也即本申请实施例将现有技术中采用软件实现的人工神经网络中的输入层和线性层进行了剥离，利用硬件的方式实现了人工神经网络中的输入层和线性层这两层结构，从而使得后续在使用该智能芯片进行人工神经网络智能处理时不需要再进行与输入层和线性层对应的复杂的信号处理和算法处理，只需由智能芯片中的处理器进行与电信号全连接与非线性激活的相关处理即可，这样可以大幅降低人工神经网络处理时的功耗和延时。 由此可见，本申请实施例将光滤波器层作为人工神经网络的输入层，图像传感器作为人工神经网络的线性层，将光滤波器层对进入光滤波器层的入射光的滤波作用作为输入层到线性层的连接权重，利用光滤波器层和图像传感器将目标对象的入射光携带信息投影成电信号，然后在处理器中实现电信号的全连接处理与非线性激活处理，由此可见，本申请实施例不但能够省去现有技术中与输入层和线性层对应的复杂的信号处理和算法处理，而且本申请实施例实际上同时利用了目标对象的图像信息、光谱信息、入射光的角度和入射光的相位信息，即目标对象空间不同点处的光携带信息，由此可见，由于目标对象空间不同点处的入射光携带信息涵盖了目标对象的图像、成分、形状、三维深度、结构等信息，从而在依据目标对象空间不同点处的入射光携带信息进行识别处理时，可以涵盖目标对象的图像、成分、形状、三维深度、结构等多维度的信息，从而可以解决背景技术部分所提到的采用目标对象的二维图像信息难以保证识别的准确性例如难以区分是真实人物还是图片的问题，由此可见，本申请实施例提供的光人工神经网络芯片，不但能够实现低功耗和低延时的效果，还能够实现高准确率的效果，从而可以为应用于智能感知、识别和/或决策等智能处理任务做好准备。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请第一个实施例提供的光人工神经网络智能芯片的结构示意图；
图2是本申请一实施例提供的光人工神经网络智能芯片识别原理示意图；
图3是本申请一实施例提供的光人工神经网络智能芯片拆解示意图；
图4是本申请一实施例提供的目标对象识别过程示意图；
图5是本申请一实施例提供的一种光滤波器层的俯视图；
图6是本申请一实施例提供的另一种光滤波器层的俯视图；
图7是本申请一实施例提供的又一种光滤波器层的俯视图；
图8是本申请一实施例提供的另又一种光滤波器层的俯视图；
图9是本申请一实施例提供的再又一种光滤波器层的俯视图；
图10是本申请一实施例提供的还又一种光滤波器层的俯视图；
图11是本申请一实施例提供的微纳结构宽带滤波效果示意图；
图12是本申请一实施例提供的微纳结构窄带滤波效果示意图；
图13是本申请一实施例提供的前照式的图像传感器结构示意图；
图14是本申请一实施例提供的后照式的图像传感器结构示意图；
图15是本申请第三个实施例提供的光人工神经网络智能芯片的制备方法的流程示意图；
图16是本申请一实施例提供的人脸识别过程示意图；
图17是本申请一实施例提供的手指血糖检测过程示意图；
图18是本申请一实施例提供的手腕血糖检测过程示意图；
图19是本申请一实施例提供的对农业对象进行识别的过程示意图；
图20是本申请一实施例提供的对农作物和/或土壤进行识别或定性分析的立体示意图；
图21是本申请一实施例提供的对冶炼过程中炉口进行识别以确定冶炼终点的示意图。
具体实施方式
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
现有的智能识别技术，通常需要先对人或物体成像，再经过图像的预处理、特征提取、特征匹配等步骤，实现对人或物体的识别。然而，只利 用人或物体的二维图像信息难以保证识别的准确性，例如难以区分真实的人脸和人脸照片，并且成像过程需要将光学信息转换为数字电子信号，再传输到计算机中进行后续的算法处理，大量数据的传输和处理造成了较大的功耗和延时。基于此，本申请实施例提供一种光人工神经网络智能芯片，该智能芯片中的光滤波器层对应人工神经网络的输入层，图像传感器对应人工神经网络的线性层，光滤波器层对进入光滤波器层的入射光的滤波作用对应输入层到线性层的连接权重，本申请实施例利用光滤波器层和图像传感器将目标对象的空间光谱信息投影成电信号，然后在处理器中实现电信号的全连接处理与非线性激活处理，由此可见，本申请实施例不但能够省去现有技术中与输入层和线性层对应的复杂的信号处理和算法处理，而且本申请实施例实际上同时利用了目标对象光强度分布信息(即图像信息)、光谱信息、入射光的角度和入射光的相位信息，即目标对象空间不同点处的光携带信息，从而使得可以从空间图像、光谱、角度、相位信息，从而可以提高智能处理(如智能识别)的准确性，由此可见，本申请实施例提供的光人工神经网络芯片，不但能够实现低功耗和低延时的效果，还能够提高智能处理的准确率，从而可以较好地应用在智能感知、识别和/或决策等智能处理领域。下面将通过具体实施例对本申请提供的内容进行详细解释和说明。
如图1所示，本申请第一个实施例提供的光人工神经网络智能芯片，包括：光滤波器层1、图像传感器2和处理器3；所述光滤波器层1对应人工神经网络的输入层以及输入层到线性层的连接权重，所述图像传感器2对应人工神经网络的线性层；所述处理器3对应人工神经网络的非线性层以及输出层；
所述光滤波器层1设置于所述图像传感器的感光区域的表面，所述光滤波器层1包含有光调制结构，所述光滤波器层1用于通过所述光调制结构对进入至所述光调制结构不同位置点处的入射光分别进行随波长变化强度调制的频谱调制，即对不同波长的入射光进行不同的强度调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息；所述入射光携带信息包括光强度分布信息、光谱信息、所述入射光的角度信息以及所述入射光的相位信息；
所述图像传感器2用于将与不同位置点经光滤波器层1调制后对应的入射光携带信息转换为与不同位置点对应的电信号，并将与不同位置点对应的电信号发送给所述处理器3；所述电信号为经光滤波器层调制后的图像信号；
所述处理器3用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到人工神经网络的输出信号。
在本实施例中，光滤波器层1设置于图像传感器的感光区域的表面，光滤波器层1包含有光调制结构，光滤波器1层用于通过光调制结构对进入至光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的调制后的入射光携带信息，相应地，图像传感器2用于将与不同位置点对应的入射光携带信息转换为与不同位置点对应的电信号，即经光滤波器层调制后的图像信号，与此同时，处理器3用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到人工神经网络的输出信号。
在本实施例中，光滤波器层1包含有光调制结构，通过光调制结构对进入至光调制结构不同位置点处的入射光(例如待识别目标的反射光、透射光、辐射光等相关作用光)进行不同强度的频谱调制，以在图像传感器2感光区域的表面得到与不同位置点对应的入射光携带信息。
在本实施例中，可以理解的是，调制强度与光调制结构的具体结构形式有关，例如，可以通过设计不同的光调制结构(如改变光调制结构的形状和/或尺寸参数)来实现不同的调制强度。
在本实施例中，可以理解的是，光滤波器层1上不同位置处的光调制结构对入射光具有不同的频谱调制作用，光调制结构对入射光不同波长成分的调制强度对应于人工神经网络的连接强度，也即对应输入层以及输入层到线性层的连接权重。需要说明的是，光滤波器层1是由多个光滤波器单元组成的，每个光滤波器单元内不同位置处的光调制结构是不同的，因此对入射光具有不同的频谱调制作用；光滤波器单元之间不同位置处的光调制结构可以相同或不同，因此对入射光具有相同或不同的频谱调制作用。
在本实施例中，图像传感器2将与不同位置点对应的入射光携带信息 转换为与不同位置点对应的电信号，并将与不同位置点对应的电信号发送给处理器3，图像传感器2对应神经网络的线性层。
在本实施例中，处理器3将不同位置点的电信号进行全连接处理与非线性激活处理，进而得到人工神经网络的输出信号。
可以理解的是，处理器3对应神经网络的非线性层以及输出层，也可以理解成对应神经网络中除输入层和线性层以外的剩余层(其他所有层)。
此外，需要补充说明的是，处理器3可以设置在所述智能芯片内，也即所述处理器3可以和所述滤波器层1以及图像传感器2一起设置在智能芯片内，也可以单独地设置在智能芯片外，并通过数据线或连接器件与智能芯片内中的图像传感器2连接，本实施例对此不作限定。
此外，需要说明的是，所述处理器3可以采用计算机实现，也可以采用具有一定运算能力的ARM或FPGA电路板实现，还可以采用微处理器实现，本实施例对此不做限定。此外，正如前面所述，所述处理器3可以集成在所述智能芯片内，也可以独立于所述智能芯片外设置。当所述处理器3独立于所述智能芯片外设置时，可以通过信号读出电路将图像传感器2中的电信号读出至处理器3中，进而由处理器3对读出的电信号进行全连接处理与非线性激活处理。
在本实施例中，可以理解的是，处理器3在进行非线性激活处理时，可以采用非线性激活函数实现，例如可以采用Sigmoid函数、Tanh函数、ReLU函数等，本实施例对此不作限定。
在本实施例中，光滤波器层1对应人工神经网络的输入层以及输入层到线性层的连接权重，图像传感器2对应于人工神经网的线性层，将空间不同位置点的入射光携带信息转化为电信号，处理器3对应人工神经网络的非线性层以及输出层，将不同位置的电信号进行全连接，经由非线性激活函数得到人工神经网络的输出信号，实现对特定目标的智能感知、识别和/或决策。
如图2左侧所示，光人工神经网络智能芯片包括光滤波器层1、图像传感器2和处理器3，在图2中，处理器3采用信号读出电路和计算机来实现。如图2右侧所示，光人工神经网络智能芯片中的光滤波器层1对应人工神经网络的输入层，图像传感器2对应人工神经网络的线性层，处理 器3对应人工神经网络的非线性层和输出层，光滤波器层1对进入光滤波器层1的入射光的滤波作用对应输入层到线性层的连接权重，由此可见，本实施例提供的智能芯片中的光滤波器层和图像传感器通过硬件的方式实现了人工神经网络中输入层和线性层的相关功能，从而使得后续在使用该智能芯片进行智能处理时不需要再进行与输入层和线性层对应的复杂的信号处理和算法处理，这样可以大幅降低人工神经网络处理时的功耗和延时。此外，由于本实施例同时利用了目标对象的图像信息和空间不同点处的图像信息、光谱信息，入射光的角度信息、入射光的相位信息等，从而可以更加准确地实现对目标对象的智能处理。
如图2右侧所示，将光滤波器层1不同位置处的入射光频谱P_λ投影/连接到图像传感器的光电流响应I_N上，处理器3包括信号读出电路和计算机，处理器3中的信号读出电路将光电流响应读出至传输到计算机中，由计算机进行电信号的全连接处理与非线性激活处理，最后输出结果。
如图3所示，光滤波器层1上的光调制结构集成在图像传感器2上方，对入射光进行调制，将入射光的频谱信息投影/连接到图像传感器2的不同像素点上，得到包含入射光频谱信息和图像信息的电信号，也即入射光经过光滤波器层1后，由图像传感器2转换成电信号，形成包含入射光的频谱信息的图像，最后由与图像传感器2连接的处理器3对包含入射光的频谱信息和图像信息的电信号进行处理。由此可见，本实施例提供的光人工神经网络芯片实际上同时利用了目标对象的图像信息、光谱信息、入射光的角度和入射光的相位信息，即目标对象空间不同点处的光携带信息，由此可见，由于目标对象空间不同点处的入射光携带信息涵盖了目标对象的图像、成分、形状、三维深度、结构等信息，从而在依据目标对象空间不同点处的入射光携带信息进行识别处理时，可以涵盖目标对象的图像、成分、形状、三维深度、结构等多维度的信息，从而可以解决背景技术部分所提到的采用目标对象的二维图像信息难以保证识别的准确性例如难以区分是真实人物还是图片的问题，实现面向不同应用领域的智能感知、识别和/或决策功能，并且实现了低功耗、低延时和高准确率的频谱光人工神经网智能芯片。
本申请实施例提供的光人工神经网络智能芯片，包括光滤波器层、图 像传感器和处理器，光滤波器层设置于图像传感器的感光区域的表面，光滤波器层包含有光调制结构，光滤波器层用于通过光调制结构对进入至光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息，相应地，图像传感器用于将与不同位置点对应的入射光携带信息转换为与不同位置点对应的电信号，与此同时，与图像传感器连接的处理器用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到人工神经网络的输出信号，由此可见，在该智能芯片中，光滤波器层作为人工神经网络的输入层，图像传感器作为人工神经网络的线性层，同时，光滤波器层对进入光滤波器层的入射光的滤波作用对应输入层到线性层的连接权重，也即该智能芯片中的光滤波器层和图像传感器实现了人工神经网络中输入层和线性层的相关功能，也即本申请实施例将现有技术中采用软件实现的人工神经网络中的输入层和线性层进行了剥离，利用硬件的方式实现了人工神经网络中的输入层和线性层这两层结构，从而使得后续在使用该智能芯片进行人工神经网络智能处理时不需要再进行与输入层和线性层对应的复杂的信号处理和算法处理，只需由智能芯片中的处理器进行与电信号全连接与非线性激活的相关处理即可，这样可以大幅降低人工神经网络处理时的功耗和延时。由此可见，本申请实施例将光滤波器层作为人工神经网络的输入层，图像传感器作为人工神经网络的线性层，将光滤波器层对进入光滤波器层的入射光的滤波作用作为输入层到线性层的连接权重，利用光滤波器层和图像传感器将目标对象的入射光携带信息投影成电信号，然后在处理器中实现电信号的全连接处理与非线性激活处理，由此可见，本申请实施例不但能够省去现有技术中与输入层和线性层对应的复杂的信号处理和算法处理，而且本申请实施例实际上同时利用了目标对象的图像信息、光谱信息、入射光的角度信息和入射光的相位信息，即目标对象空间不同点处的入射光携带信息，由此可见，由于目标对象空间不同点处的入射光携带信息涵盖了目标对象的图像、成分、形状、三维深度、结构等信息，从而在依据目标对象空间不同点处的入射光携带信息进行识别处理时，可以涵盖目标对象的图像、成分、形状、三维深度、结构等多维度的信息，从而可以解决背景技术部分所提到的采用目标对象的二维图像信息 难以保证识别的准确性例如难以区分是真实人物还是图片的问题，由此可见，本申请实施例提供的光人工神经网络芯片，不但能够实现低功耗和低延时的效果，还能够实现高准确率的效果，从而可以为应用于智能感知、识别和/或决策等智能处理任务做好准备。
基于上述实施例的内容，在本实施例中，所述光人工神经网络智能芯片用于目标对象的智能处理任务；所述智能处理任务至少包括智能感知、智能识别和智能决策任务中的一种或多种；
目标对象的反射光、透射光和/或辐射光进入至训练好的光人工神经网络智能芯片中，得到所述目标对象的智能处理结果；所述智能处理结果至少包括智能感知结果、智能识别结果和/或智能决策结果中的一种或多种；
训练好的光人工神经网络智能芯片是指包括训练好的光调制结构、图像传感器和处理器的光人工神经网络智能芯片；所述训练好的光调制结构、图像传感器和处理器是指利用与所述智能处理任务对应的输入训练样本和输出训练样本，对包含不同的光调制结构、图像传感器和具有不同的全连接参数与非线性激活参数的处理器的光人工神经网络智能芯片进行训练得到的满足训练收敛条件的光调制结构、图像传感器和处理器。
在本实施例中，光人工神经网络智能芯片可以用于目标对象的智能处理任务，例如，包括智能感知、智能识别和智能决策任务中的一种或多种任务。
在本实施例中，可以理解的是，智能感知是指将物理世界的信号通过摄像头、麦克风或者其他传感器的硬件设备，借助语音识别、图像识别等前沿技术，映射到数字世界，再将这些数字信息进一步提升至可认知的层次，比如记忆、理解、规划、决策等等。智能识别是指利用计算机对图像进行处理、分析和理解，以识别各种不同模式的目标和对象的技术，现阶段智能识别技术一般分为人脸识别与商品识别，人脸识别主要运用在安全检查、身份核验与移动支付中，商品识别主要运用在商品流通过程中，特别是无人货架、智能零售柜等无人零售领域。智能决策是指解决由计算机自动组织和协调多模型运行，对大量数据库中数据的存取和处理，进行相应数据处理和数值计算。
在本实施例中，目标对象的反射光、透射光和/或辐射光进入至训练好 的光人工神经网络智能芯片中，得到目标对象的智能处理结果。
在本实施例中，以目标对象的识别任务为例进行说明，可以理解的是，在利用该智能芯片进行识别任务时，首先需要对光人工神经网络智能芯片进行训练，这里对光人工神经网络智能芯片进行训练是指通过训练确定适用于当前识别任务的光调制结构，以及，适用于当前识别任务的全连接参数与非线性激活参数。
可以理解的是，由于光滤波器层对进入光滤波器层的入射光的滤波作用对应人工神经网络输入层到线性层的连接权重，因此，在训练时，改变光滤波器层中的光调制结构相当于改变人工神经网络输入层到线性层的连接权重，通过训练收敛条件，确定出适用于当前识别任务的光调制结构，以及，适用于当前识别任务的全连接参数与非线性激活参数，从而完成对智能芯片的训练。
可以理解的是，在对智能芯片训练后，就可以使用该智能芯片执行识别任务。具体地，携带有目标对象图像信息以及空间光谱信息的入射光进入训练好的智能芯片的光滤波器层1后，光滤波器层1中的光调制结构会对入射光进行调制，调制后的光信号强度由图像传感器2探测并转换成电信号，再由处理器3进行全连接处理与非线性激活处理，就能得到目标对象的识别结果。
如图4所示，对于目标对象识别的完整流程为：宽谱光源100照射到目标对象200上，然后目标对象的反射光或透射光由光人工神经网络智能芯片300采集，或者目标对象直接向外辐射的光由光人工神经网络智能芯片300采集，由智能芯片中的光滤波器层、图像传感器和处理器进行处理后，即可得到识别结果。
其中，训练好的光人工神经网络智能芯片是指包括训练好的光调制结构、图像传感器和处理器的光人工神经网络智能芯片；所述训练好的光调制结构、图像传感器和处理器是指利用与所述智能处理任务对应的输入训练样本和输出训练样本，对包含不同的光调制结构、图像传感器和具有不同的全连接参数与非线性激活参数的处理器的光人工神经网络智能芯片进行训练得到的满足训练收敛条件的光调制结构、图像传感器和处理器。
举例来说，对于智能识别任务来说，与智能识别任务对应的输入训练 样本为识别对象样本，与智能识别任务对应的输出训练样本为所述识别对象样本的识别结果。可以理解的是，对于识别任务来说，由于本实施例提供的智能芯片的优势还在于能够获取到识别对象的空间光谱信息，因此，为充分利用该优势，对于作为输入训练样本的识别对象样本优先采用真实的识别对象，而不是识别对象的二维图像。当然，这并不代表不可以将二维图像作为识别对象样本。
此外，在本实施例提供的光人工神经网络智能芯片还可以用于目标对象的其他智能处理任务，如智能感知、智能决策等任务。
在本实施例中，光滤波器层1作为神经网络的输入层，图像传感器2作为神经网络的线性层，为了使神经网络的损失函数最小，将光滤波器层中的光调制结构对目标对象的入射光中不同波长成分的调制强度作为神经网络的输入层到线性层的连接权重，通过调整滤波器的结构可以调整目标对象的入射光中不同波长成分的调制强度，从而实现对输入层到线性层连接权重的调整，进而优化神经网络的训练。
因此，本实施例中光调制结构是基于神经网络训练得到的，通过计算机对训练样本进行光学仿真，获取训练样本中光调制结构对智能处理任务中目标对象的入射光不同波长成分的样本调制强度，将样本调制强度作为神经网络的输入层到线性层的连接权重，进行非线性激活，并利用与智能处理任务对应的训练样本进行神经网络训练，直至神经网络收敛时将对应的训练样本光调制结构作为对应智能处理任务的光滤波器层。
由此可见，本实施例通过在物理层实现神经网络的输入层(光滤波器层)和线性层(图像传感器)，不但能够省去现有技术中与输入层和线性层对应的复杂的信号处理和算法处理，而且本申请实施例实际上同时利用了目标对象的图像信息、光谱信息、入射光的角度信息和入射光的相位信息，即目标对象空间不同点处的入射光携带信息，由此可见，由于目标对象空间不同点处的入射光携带信息涵盖了目标对象的图像、成分、形状、三维深度、结构等信息，从而在依据目标对象空间不同点处的入射光携带信息进行识别处理时，可以涵盖目标对象的图像、成分、形状、三维深度、结构等多维度的信息，从而可以解决背景技术部分所提到的采用目标对象的二维图像信息难以保证识别的准确性例如难以区分是真实人物还是图 片的问题，由此可见，本申请实施例提供的光人工神经网络芯片，不但能够实现低功耗和低延时的效果，还能够实现高准确率的效果，从而可以为应用于智能感知、识别和/或决策等智能处理任务做好准备。
基于上述实施例的内容，在本实施例中，在对包含不同的光调制结构、图像传感器和具有不同的全连接参数和非线性激活参数的处理器的光人工神经网络智能芯片进行训练时，所述不同的光调制结构通过采用计算机光学仿真设计的方式设计实现。
在本实施例中，光学仿真能让用户在制作物理原型之前就通过数字环境体验产品，例如对于汽车来说，因为光照及反射光会干扰驾驶员注意力，尤其是在夜间行驶的情况下，合适的光学仿真解决方案不仅能有效帮助用户提高设计效率，还能对光线与材料的交互进行仿真，以便了解产品在真实条件下的展示效果。因此，本实施例通过计算机光学仿真对光调制结构进行设计，通过光学仿真调整光调制结构，直至神经网络收敛时确定对应的光调制结构为最终需要制作的光调制结构尺寸，节省原型制作时间和成本，提高产品效率，轻松解决复杂的光学问题。
例如，可以通过FDTD软件来对光调制结构进行仿真设计，在光学仿真中改变光调制结构，从而可以准确地预测光调制结构对不同入射光的调制强度，并将其作为神经网络输入层与线性层的连接权重，对光人工神经网络智能芯片进行训练，准确获取光调制结构。
由此可见，本实施例通过采用计算机光学仿真设计的方式设计光调制结构，节省了光调制结构原型制作时间和成本，提高产品效率。
基于上述实施例的内容，在本实施例中，所述光滤波器层中的光调制结构包含规则结构和/或不规则结构；和/或，所述光滤波器层中的光调制结构包含离散型结构和/或连续型结构。
在本实施例中，所述光滤波器层中的光调制结构可以只包括规则结构，也可以只包括不规则结构，还可以既包括规则结构，又包括不规则结构。
在本实施例中，这里光调制结构包括规则结构可以指：光调制结构包含的最小调制单元是规则的结构，如最小调制单元可以长方形、正方形和圆形等规则的图形。此外，这里光调制结构包括规则结构还可以指：光调 制结构包含的最小调制单元的排布方式是规则的，如排布方式可以是规则的阵列形式、圆形形式、梯形形式、多边形形式等。此外，这里光调制结构包括规则结构还可以指：光调制结构包含的最小调制单元是规则的结构，同时最小调制单元的排布方式也是规则的等。
在本实施例中，这里的光调制结构包括不规则结构可以指：光调制结构包含的最小调制单元是不规则的结构，如最小调制单元可以不规则多边形、随机形状等不规则的图形。此外，这里光调制结构包括不规则结构还可以指：光调制结构包含的最小调制单元的排布方式是不规则的，如排布方式可以是不规则的多边形形式、随机排列形式等。此外，这里光调制结构包括不规则结构还可以指：光调制结构包含的最小调制单元是不规则的结构，同时最小调制单元的排布方式也是不规则的等。
在本实施例中，所述光滤波器层中的光调制结构可以包括离散型结构，也可以包括连续型结构，还可以既包括离散型结构，又包括连续型结构。
在本实施例中，这里光调制结构包括连续型结构可以指：光调制结构是由连续的调制图案构成的；这里光调制结构包括离散型结构可以指：光调制结构是由离散的调制图案构成的。
可以理解的是，这里连续的调制图案可以指直线型图案、波浪线型图案、折线型图案等等。
可以理解的是，这里离散的调制图案可以指由离散的图形(如离散的点、离散的三角形、离散的星形等)形成的调制图案。
在本实施例中，需要说明的是，光调制结构对不同波长的光具有不同的调制作用，具体的调制方式包括但不限于散射、吸收、干涉、表面等离激元、谐振增强等。通过设计不同的滤波器结构，使得光通过不同组的滤波器结构后，对应的透射谱不同。
基于上述实施例的内容，在本实施例中，所述光滤波器层为单层结构或多层结构。
在本实施例中，需要说明的是，所述光滤波器层可以为单层滤波器结构，也可以是多层滤波器结构，例如可以是两层、三层、四层等多层结构。
在本实施例中，如图1所示，所述光滤波器层1为单层结构，光滤波 器层1的厚度与目标波长范围相关，对于波长400nm～10μm，光栅结构的厚度可以为50nm～5μm。
可以理解的是，由于光滤波器层1的作用是对入射光进行频谱调制，因此，优选折射率高、损耗小的材料制备，例如可以选择硅、锗、锗硅材料、硅的化合物、锗的化合物、III-V族材料等进行制备，其中硅的化合物包括但不限于氮化硅、二氧化硅、碳化硅等。
此外，需要说明的是，为在输入层和线性层之间形成更多或更复杂的连接权重，优选地，可以将所述光滤波器层1设置为多层结构，各层对应的光调制结构可以设置为不同的结构，从而增加光滤波器层对入射光的频谱调制能力，从而可以在输入层和线性层之间形成更多或更为复杂的连接权重，进而提高智能芯片在处理智能任务时的准确度。
此外，需要说明的是，对于包含多层结构的滤波器层来说，每层结构的材料可以相同，也可以不同，举例来说，对于有二层的光滤波器层1，第一层可以为硅层，第二层可以为氮化硅层。
需要说明的是，光滤波器层1厚度与目标波长范围相关，对于波长400nm～10μm，多层结构总的厚度可以为50nm～5μm。
基于上述实施例的内容，在本实施例中，所述光滤波器层中的光调制结构包含由多个微纳单元组成的单元阵列，每个微纳单元对应图像传感器上的一个或多个像素点；各个微纳单元的结构相同或不同。
在本实施例中，为能够获得呈阵列分布的连接权重(用于连接输入层与线性层之间的连接权重)以便于处理器进行后续的全连接与非线性激活处理，优选地，在本实施例中，光调制结构为阵列结构形式，具体地，光调制结构包括由多个微纳单元组成的单元阵列，每个微纳单元对应图像传感器上的一个或多个像素点。需要说明的是，各个微纳单元的结构可以相同，也可以不同。此外，需要说明的是，各个微纳单元的结构可以是周期的，也可以是非周期的。此外，需要说明的是，各个微纳单元还可以进一步包含有多组微纳结构阵列，各组微纳结构阵列的结构相同或不同等。
下面结合图5～图9进行举例说明，在本实施例中，如图5所示，光滤波器层1包含多个重复的连续或离散的微纳单元，如11、22、33、44、55、66，每个微纳单元结构相同(且每个微纳单元为非周期结构)，每个 微纳单元对应图像传感器2上的一个或多个像素点；如图6所示，光滤波器层1包含多个重复的微纳单元，如11、22、33、44、55、66，每个微纳单元结构相同(与图5的区别在于图6中每个微纳单元为周期结构)，每个微纳单元对应图像传感器2上的一个或多个像素点；如图7所示，光滤波器层1包含多个重复的微纳单元，如11、22、33、44、55、66，每个微纳单元结构相同(且每个微纳单元为周期结构)，每个微纳单元对应图像传感器2上的一个或多个像素点，与图6的区别在于图7中的各微纳单元内周期阵列的单元形状具有四重旋转对称性；如图8所示，光滤波器层1包含多个微纳单元，如11、22、33、44、55、66，与图6的区别在于每个微纳单元结构互不相同，每个微纳单元对应图像传感器2上的一个或多个像素点，本实施例中光滤波器层1包含多个互不相同的微纳单元，也即智能芯片上不同区域对入射光的调制作用不同，从而提高了设计的自由度，进而也可以提升识别的准确率。如图9所示，光滤波器层1包含多个重复的微纳单元，如11、22、33、44、55、66，每个微纳单元结构相同，与图5的区别在于各微纳单元是由分立的非周期阵列结构组成的，每个微纳单元对应图像传感器2上的一个或多个像素点。
在本实施例中，微纳单元对不同波长的光具有不同的调制作用，具体的调制方式包括但不限于散射、吸收、干涉、表面等离激元、谐振增强等。通过设计不同的滤波器结构，使得光通过不同组的滤波器结构后，对应的透射谱不同。
基于上述实施例的内容，在本实施例中，所述微纳单元包含规则结构和/或不规则结构；和/或，所述微纳单元包含离散型结构和/或连续型结构。
在本实施例中，所述微纳单元可以只包括规则结构，也可以只包括不规则结构，还可以既包括规则结构，又包括不规则结构。
在本实施例中，这里微纳单元包括规则结构可以指：微纳单元包含的最小调制单元是规则的结构，如最小调制单元可以长方形、正方形和圆形等规则的图形。此外，这里微纳单元包括规则结构还可以指：微纳单元包含的最小调制单元的排布方式是规则的，如排布方式可以是规则的阵列形式、圆形形式、梯形形式、多边形形式等。此外，这里微纳单元包括规则结构还可以指：微纳单元包含的最小调制单元是规则的结构，同时最小调 制单元的排布方式也是规则的等。
在本实施例中，这里的微纳单元包括不规则结构可以指：微纳单元包含的最小调制单元是不规则的结构，如最小调制单元可以不规则多边形、随机形状等不规则的图形。此外，这里微纳单元包括不规则结构还可以指：微纳单元包含的最小调制单元的排布方式是不规则的，如排布方式可以是不规则的多边形形式、随机排列形式等。此外，这里微纳单元包括不规则结构还可以指：微纳单元包含的最小调制单元是不规则的结构，同时最小调制单元的排布方式也是不规则的等。
在本实施例中，所述光滤波器层中的微纳单元可以包括离散型结构，也可以包括连续型结构，还可以既包括离散型结构，又包括连续型结构。
在本实施例中，这里微纳单元包括连续型结构可以指：微纳单元是由连续的调制图案构成的；这里微纳单元包括离散型结构可以指：微纳单元是由离散的调制图案构成的。
可以理解的是，这里连续的调制图案可以指直线型图案、波浪线型图案、折线型图案等等。
可以理解的是，这里离散的调制图案可以指由离散的图形(如离散的点、离散的三角形、离散的星形等)形成的调制图案。
在本实施例中，需要说明的是，不同微纳单元对不同波长的光具有不同的调制作用，具体的调制方式包括但不限于散射、吸收、干涉、表面等离激元、谐振增强等。通过设计不同的微纳单元，使得光通过不同组的微纳单元后，对应的透射谱不同。
基于上述实施例的内容，在本实施例中，所述微纳单元包含有多组微纳结构阵列，各组微纳结构阵列的结构相同或不同。
在本实施例中，如图5所示，光滤波器层1包含多个重复的微纳单元，如11、22、33、44、55、66，每个微纳单元包含有多组微纳结构阵列，如微纳单元11包含4个不同的微纳结构阵列110、111、112和113，滤波单元44包含4个不同的微纳结构阵列440、441、442和443。如图10所示，光滤波器层1包含多个微纳单元，如11、22、33、44、55、66，每个微纳单元包含有多组微纳结构阵列，如微纳单元11包含4个相同的微纳结构阵列110、111、112和113。
需要说明的是，这里只是以包括四组微纳结构阵列的微纳单元进行举例说明，并不起到限制作用，在实际应用中，还可以根据需要设置包括六组、八组或其他数量组微纳结构阵列的微纳单元。
在本实施例中，微纳单元内的每组微纳结构阵列对不同波长的光具有不同的调制作用，并且各组滤波结构之间对输入光的调制作用也不同，具体的调制方式包括但不限于散射、吸收、干涉、表面等离激元、谐振增强等。通过设计不同的微纳结构阵列，使得光通过不同组的微纳结构阵列后，对应的透射谱不同。
基于上述实施例的内容，在本实施例中，各组微纳结构阵列具有宽带滤波或窄带滤波的作用。
在本实施例中，为了能够获取目标对象的入射光不同波长成分的调制强度作为神经网络输入层和线性层的连接权重，通过采用不同的微纳结构阵列实现宽带滤波和窄带滤波，因此本实施例中微纳结构阵列通过对目标对象的入射光进行宽带滤波或窄带滤波，获取目标对象的入射光不同波长成分的调制强度。如图11和图12所示，光滤波器层中各组微纳结构阵列具有宽带滤波或窄带滤波的作用。
可以理解的是，对于各组微纳结构阵列来说，可以都具备宽带滤波作用，也可以都具备窄带滤波作用，也可以部分具备宽带滤波作用，部分具备窄带滤波作用。此外，各组微纳结构阵列的宽带滤波范围和窄带滤波范围也可以相同或不同。举例来说，通过设计微纳单元内各组微纳结构的周期、占空比、半径、边长等尺寸参数，使其具有窄带滤波作用，即只有一个(或较少个)波长的光可以通过。又如，通过设计微纳单元内各组微纳结构的周期、占空比、半径、边长等尺寸参数，使其具有宽带滤波作用，即允许较多波长或所有波长的光可以通过。
可以理解的是，在具体使用时，可以根据应用场景进行宽带滤波、窄带滤波或其组合的方式确定各组微纳结构阵列的滤波状态。
基于上述实施例的内容，在本实施例中，各组微纳结构阵列为周期结构阵列或非周期结构阵列。
在本实施例中，各组微纳结构阵列可以均为周期结构阵列，也可以均为非周期结构阵列，也可以部分为周期结构阵列，部分为非周期结构阵列。 其中，周期结构阵列易于进行光学仿真设计，非周期结构阵列可以实现更复杂的调制作用。
在本实施例中，如图5所示，光滤波器层1包含多个重复的微纳单元，如11、22、33、44、55、66，每个微纳单元由多组微纳结构阵列组成，各微纳结构阵列结构互不相同，且微纳结构阵列为非周期性结构。其中，非周期性结构指微纳结构阵列上的调制孔形状按照非周期排列方式进行排布。如图5所示，微纳单元11包含4个不同的非周期结构阵列110、111、112和113，微纳单元44包含4个不同的非周期结构阵列440、441、442和443，非周期性结构的微纳结构阵列是通过前期针对智能处理任务，由神经网络数据训练设计得到的，通常是不规则形状的结构。如图6所示，光滤波器层1包含多个重复的微纳单元，如11、22、33、44、55、66，每个微纳单元由多组微纳结构阵列组成，各微纳结构阵列结构互不相同，与图5所不同的是，微纳结构阵列为周期性结构。其中，周期性结构指微纳结构阵列上的调制孔形状按照周期排列方式进行排布，周期的大小通常为20nm～50μm。如图6所示，微纳单元11包含4个不同的周期结构阵列110、111、112和113，微纳单元44包含4个不同的周期结构阵列440、441、442和443，周期性结构的滤波器结构是通过前期针对智能处理任务，由神经网络数据训练设计得到的，通常是不规则形状的结构。如图7所示，光滤波器层1包含多个互不相同的微纳单元，如11、22、33、44、55、66，每个微纳单元由多组微纳结构阵列组成，各微纳结构阵列结构互不相同，且微纳结构阵列为周期性结构。其中，周期性结构指滤波器结构上的形状按照周期排列方式进行排布，周期的大小通常为20nm～50μm。如图7所示，微纳单元11与微纳单元12的微纳结构阵列互不相同，微纳单元11包含4个不同的周期结构阵列110、111、112和113，微纳单元44包含4个不同的周期结构阵列440、441、442和443，周期性结构的微纳结构阵列是通过前期针对智能处理任务，由神经网络数据训练设计得到的，通常是不规则形状的结构。
需要说明的是，图5～图9每个微纳单元均包括四组微纳结构阵列，四组微纳结构阵列分别采用四种不同形状的调制孔形成，四组微纳结构阵列用于对入射光具有不同的调制作用。需要说明的是，这里只是以包括四 组微纳结构阵列的微纳单元进行举例说明，并不起到限制作用，在实际应用中，还可以根据需要设置包括六组、八组或其他数量组微纳结构阵列的微纳单元。在本实施例中，四种不同形状可以为圆形、十字形、正多边形和矩形(不限于此)。
在本实施例中，微纳单元内的每组微纳结构阵列对不同波长的光具有不同的调制作用，并且各组微纳结构阵列之间对输入光的调制作用也不同，具体的调制方式包括但不限于散射、吸收、干涉、表面等离激元、谐振增强等。通过设计不同的微纳结构阵列，使得光通过不同组的微纳结构阵列后，对应的透射谱不同。
基于上述实施例的内容，在本实施例中，所述微纳单元包含的多组微纳结构阵列中有一组或多组空结构。
下面结合图9所示例子进行举例说明，在本实施例中，如图9所示，光滤波器层1包含多个重复的微纳单元，如11、22、33、44、55、66，每个微纳单元由多组微纳结构阵列组成，多组微纳结构阵列对应的结构互不相同，微纳结构阵列为周期性结构，与上述实施例所不同的是，对于任一微纳单元包含一组或多组空结构，所述空结构用于直通入射光。可以理解的是，当多组微纳结构阵列中包含有一组或多组空结构时，可以形成更为丰富的频谱调制效果，从而满足特定场景下的频谱调制需求(或满足特定场景下输入层与线性层之间的特定连接权重需求)。
如图9所示，每个微纳单元均包括一组微纳结构阵列和三组空结构，微纳单元11包含1个非周期结构阵列111，微纳单元22包含1个非周期结构阵列221，微纳单元33包含1个非周期结构阵列331，微纳单元44包含1个非周期结构阵列441，微纳单元55包含1个非周期结构阵列551，微纳单元66包含1个非周期结构阵列661，其中微纳结构阵列用于对入射光进行不同的调制。需要说明的是，这里只是以包括一组微纳结构阵列和三组空结构进行举例说明，并不起到限制作用，在实际应用中，还可以根据需要设置包括一组微纳结构阵列和五组空结构或其他数量组微纳结构阵列的微纳单元。在本实施例中，微纳结构阵列可以采用圆形、十字形、正多边形和矩形的调制孔制成(不限于此)。
需要说明的是，微纳单元包含的多组微纳结构阵列中也可以均不包含 空结构，即多组微纳结构阵列可以是非周期结构阵列，也可以是周期结构阵列。
基于上述实施例的内容，在本实施例中，所述微纳单元具有偏振无关特性。
在本实施例中，由于微纳单元具有偏振无关特性，因此使得光滤波器层对入射光的偏振不敏感，从而实现了对入射角、偏振均不敏感的光人工神经网络智能芯片。本申请实施例提供的光人工神经网络智能芯片对入射光的入射角以及偏振特性不敏感，即测量结果不会受到入射光的入射角度和偏振特性的影响，从而可以保证光谱测量性能的稳定性，进而可以保证智能处理的稳定性，如智能感知的稳定性、智能识别的稳定性、智能决策的稳定性等等。需要说明的是，微纳单元也可以具有偏振相关特性。
基于上述实施例的内容，在本实施例中，所述微纳单元具有四重旋转对称性。
在本实施例中，需要说明的是，四重旋转对称性属于偏振无关特性中的一种具体情况，通过将微纳单元设计为具有四重旋转对称性的结构，可以满足偏振无关特性的要求。
下面结合图7所示例子进行举例说明，在本实施例中，如图7所示，光滤波器层1包含多个重复的微纳单元，如11、22、33、44、55、66，每个微纳单元由多组微纳结构阵列组成，多组微纳结构阵列对应的结构互不相同，微纳结构阵列为周期性结构，与上述实施例所不同的是，每组微纳结构阵列对应的结构可为圆、十字、正多边形、矩形等具有四重旋转对称性的结构，即结构旋转90°、180°、270°后，与原来的结构重合，从而使得结构具有偏振无关的特性，使得在不同偏振光入射都能取得相同的智能识别效果。
基于上述实施例的内容，在本实施例中，所述光滤波器层由一层或多层滤波器层构成；
所述滤波器层是由半导体材料、金属材料、液晶、量子点材料、钙钛矿材料中的一种或多种制备；和/或，所述滤波器层是由光子晶体、超表面、随机结构、纳米结构、金属表面等离激元SPP(Surface Plasmon Polaritons，SPP)微纳结构、可调法布里-珀罗谐振腔(Fabry-perot Cavity，FP腔)中 的一种或多种制备的滤波器层。
所述半导体材料包括硅、氧化硅、氮化硅、氧化钛、按照预设比例混合的复合材料以及直接带隙化合物半导体材料中的一种或多种；和/或，所述纳米结构包括纳米点二维材料、纳米柱二维材料和纳米线二维材料中的一种或多种。
其中，光子晶体，以及超表面与随机结构组合可采用CMOS工艺兼容，能够有较好的调制效果，微纳调制结构微孔中也可以填充其他材料做表面的平滑化处理；量子点与钙钛矿可以利用材料本身的谱调制特性使得单个调制结构的体积最小；SPP体积较小，可实现偏振相关的光调制；液晶可以用电压动态调控，提升空间分辨率；可调法布里-珀罗谐振腔可以动态调控，提升空间分辨率。
基于上述实施例的内容，在本实施例中，所述光滤波器层的厚度为0.1λ～10λ，其中λ表示入射光的中心波长。
在本实施例中，需要说明的是，若光滤波器层的厚度远小于入射光的中心波长，就不能起到有效的频谱调制作用；若光滤波器层的厚度远大于入射光的中心波长，工艺上难以制备，并且会引入较大的光学损耗。因此，本实施例中为了减少光学损耗易于制备，并且保证有效的频谱调制作用，光滤波器层1中每个微纳单元的整体尺寸(面积)通常为λ 2～105λ 2，厚度通常为0.1λ～10λ(λ表示目标对象的入射光的中心波长)。如图5所示，每个微纳单元的整体尺寸为0.5μm 2～40000μm 2，光滤波器层1中的介质材料为多晶硅，厚度为50nm～2μm。
基于上述实施例的内容，在本实施例中，所述图像传感器为下述中的任意一项或多项：
CMOS图像传感器(Contact Image Sensor，CIS)、电荷耦合元件(Charge Coupled Device，CCD)、单光子雪崩二极管(Single Photon Avalanche Diode，SPAD)阵列和焦平面光电探测器阵列。
在本实施例中，需要说明的是，采用晶圆级别的CMOS图像传感器CIS，在晶圆级别实现单片集成，可以最大程度减小图像传感器与光滤波器层之间的距离，有利用于缩小单元的尺寸，降低器件体积和封装成本，SPAD可以用于弱光探测，CCD可以用于强光探测。
在本实施例中，光滤波器层和图像传感器可以由互补金属氧化物半导体(Complementary Metal Oxide Semiconductor，CMOS)集成工艺制造，有利于降低器件失效率，提高器件的成品率并降低成本。例如，可以通过在图像传感器上直接生长一层或多层介质材料，再进行刻蚀，在除去用于刻蚀的牺牲层之前，沉积金属材料，最后再移除牺牲层，进行制备得到光滤波器层。
基于上述实施例的内容，在本实施例中，所述人工神经网络的类型包括：前馈神经网络。
在本实施例中，前馈神经网络(Feedforward Neural Network，FNN)，又称作深度前馈网络(Deep Feedforward Network，DFN)、多层感知机(Multi-Layer Perceptron，MLP)，是一种最简单的神经网络，各神经元分层排列。每个神经元只与前一层的神经元相连。接收前一层的输出，并输出给下一层，各层间没有反馈。前馈神经网络结构简单，易于在硬件上实现，应用广泛，能够以任意精度逼近任意连续函数及平方可积函数.而且可以精确实现任意有限训练样本集。前馈网络是一种静态非线性映射。通过简单非线性处理单元的复合映射，可获得复杂的非线性处理能力。
基于上述实施例的内容，在本实施例中，所述光滤波器层与所述图像传感器之前设置有透光介质层。
在本实施例中，需要说明的是，在所述光滤波器层与所述图像传感器之间设置透光介质层，可以有效将光滤波器层与图像传感器层分开，避免两者相互干扰。
基于上述实施例的内容，在本实施例中，所述图像传感器为前照式，包括：自上而下设置的金属线层和光探测层，所述光滤波器层集成在所述金属线层远离所述光探测层的一面；或，
所述图像传感器为背照式，包括：自上而下设置的光探测层和金属线层，所述光滤波器层集成在所述光探测层远离所述金属线层的一面。
在本实施例中，如图13所示为前照式的图像传感器，硅探测层21在金属线层22下方，光滤波器层1直接集成到金属线层22上。
在本实施例中，与图13不同的是，图14所示为背照式的图像传感器，硅探测层21在金属线层22上方，光滤波器层1直接集成到硅探测层21 上。
需要说明的是，对于背照式的图像传感器，硅探测层21在金属线层22上方，可以减少金属线层对入射光的影响，从而可以提高器件的量子效率。
根据上面的内容可知，本实施例将光滤波器层作为人工神经网络的输入层，将图像传感器作为人工神经网络的线性层，将光滤波器层对进入光滤波器层的入射光的滤波作用作为输入层到线性层的连接权重，本实施例提供的光人工神经网络智能芯片中的光滤波器层和图像传感器通过硬件的方式实现了人工神经网络中输入层和线性层的相关功能，从而使得后续在使用该智能芯片进行智能处理时不需要再进行与输入层和线性层对应的复杂的信号处理和算法处理，这样可以大幅降低人工神经网络处理时的功耗和延时。此外，本实施例由于同时利用了目标对象的图像信息和空间不同点处的光谱信息，从而可以更加准确地实现对目标对象的智能处理。
基于相同的发明构思，本申请另一实施例提供了一种智能处理设备，包括：如上面实施例所述的光人工神经网络智能芯片。所述智能处理设备包括智能手机、智能电脑、智能识别设备、智能感知设备和智能决策设备中的一种或多种。
由于本实施例提供的智能处理设备包括上述实施例所述的光人工神经网络智能芯片，因此，本实施例提供的智能处理设备具备上述实施例所述的光人工神经网络智能芯片的全部有益效果，由于上述实施例已经对此进行了较为详尽的描述，因此本实施例不再赘述。
基于相同的发明构思，本申请另一实施例提供了一种如上面所述实施例的光人工神经网络智能芯片的制备方法，如图15所示，具体包括如下步骤：
步骤1510、在所述图像传感器的感光区域的表面制备包含有光调制结构的光滤波器层；
步骤1520、生成具备对信号进行全连接处理与非线性激活处理功能的处理器；
步骤1530、连接所述图像传感器和所述处理器；
其中，所述光滤波器层用于通过所述光调制结构对进入至所述光调制 结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息；所述入射光携带信息包括光强度分布信息、光谱信息、所述入射光的角度信息以及所述入射光的相位信息；
所述图像传感器用于将与不同位置点经光滤波器层调制后对应的入射光携带信息转换为与不同位置点对应的电信号，并将与不同位置点对应的电信号发送给所述处理器；所述处理器用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到人工神经网络的输出信号。
在所述图像传感器的感光区域的表面制备包含有光调制结构的光滤波器层，包括：
在所述图像传感器的感光区域的表面生长一层或多层预设材料；
对所述一层或多层预设材料进行光调制结构图案的干法刻蚀，得到包含有光调制结构的光滤波器层；
或对所述一层或多层预设材料进行压印转移，得到包含有光调制结构的光滤波器层；
或通过对所述一层或多层预设材料进行外加动态调控，得到包含有光调制结构的光滤波器层；
或对所述一层或多层预设材料进行分区打印，得到包含有光调制结构的光滤波器层；
或对所述一层或多层预设材料进行分区材料生长，得到包含有光调制结构的光滤波器层；
或对所述一层或多层预设材料进行量子点转移，得到包含有光调制结构的光滤波器层。
当所述光人工神经网络智能芯片用于目标对象的智能处理任务时，利用与所述智能处理任务对应的输入训练样本和输出训练样本，对包含不同的光调制结构、图像传感器和具有不同的全连接参数和非线性激活参数的处理器的光人工神经网络智能芯片进行训练，得到满足训练收敛条件的光调制结构、图像传感器和处理器。
在本实施例中，需要说明的是，如图1所示，光滤波器层1可以通过在图像传感器2上直接生长一层或多层介质材料，再进行刻蚀，在除去用 于刻蚀的牺牲层之前，沉积金属材料，最后再移除牺牲层，进行制备得到。通过设计光调制结构的尺寸参数，各个单元能够对目标范围内不同波长的光有不同的调制作用，并且该调制作用对入射角度、偏振均不敏感。光滤波器层1中的每个单元对应图像传感器2上一个或多个像素。1是直接在2上制备的。
在本实施例中，需要说明的是，如图14所示，假设图像传感器2为背照式结构，则光滤波器层1可以在背照式的图像传感器的硅探测器层21上直接刻蚀，然后再沉积金属进行制备得到。
此外，需要说明的是，所述光滤波器层上的光调制结构可以通过对一层或多层预设材料进行光调制结构图案的干法刻蚀，干法刻蚀就是直接将图像传感器感光区域表面的一层或多层预设材料中不需要的部分去除，得到包含有光调制结构的光滤波器层；或对一层或多层预设材料进行压印转移，压印转移是在其它衬底上通过刻蚀制备所需的结构，再通过PDMS等材料将结构转移到图像传感器的感光区域，得到包含有光调制结构的光滤波器层；或通过对一层或多层预设材料进行外加动态调控，外加动态调控是采用有源材料，然后外加电极通过改变电压来调控相应区域的光调制特性，得到包含有光调制结构的光滤波器层；或对一层或多层预设材料进行分区打印，分区打印是分区采用打印的技术，得到包含有光调制结构的光滤波器层；或对一层或多层预设材料进行分区材料生长，得到包含有光调制结构的光滤波器层；或对一层或多层预设材料进行量子点转移，得到包含有光调制结构的光滤波器层。
此外，需要说明的是，由于本实施例提供的制备方法是上述实施例中的光人工神经网络智能芯片的制备方法，因此，关于一些原理和结构等方面的详细内容，可以参见上述实施例的介绍，本实施例对此不再赘述。
可以理解的是，人脸识别技术是一种生物特征识别技术，广泛应用于门禁考勤系统、刑侦系统、电子商务等领域。人脸识别的主要过程包括人脸图像的采集、预处理、特征提取、匹配与识别。然而，只利用人脸的图像信息难以保证识别的准确性，例如难以区分真实的人脸和人脸照片；即使结合深度信息，也难以区分人脸模型和真实人脸。因此，挖掘更多的人脸信息，实现更高准确率、安全可靠的快速人脸识别具有重要意义。
为此，基于前述实施例介绍的光人工神经网络智能芯片，本实施例提供一种用于人脸准确识别的新型光电芯片，该芯片由光滤波器层构成光人工神经网络的输入层以及输入层到线性层的连接权重，由图像传感器构成光人工神经网络的线性层，通过采集待识别人脸的图像信息和光谱信息，可以实现快速准确、安全可靠的人脸识别。下面具体对本实施例的内容进行详细解释和说明。
本申请实施例还提供了一种光人工神经网络人脸识别芯片，用于人脸识别处理任务，包括：光滤波器层、图像传感器和处理器；所述光滤波器层对应人工神经网络的输入层以及输入层到线性层的连接权重，所述图像传感器对应人工神经网络的线性层；所述处理器对应人工神经网络的非线性层以及输出层；
所述光滤波器层设置于所述图像传感器的感光区域的表面，所述光滤波器层包含有光调制结构，所述光滤波器层用于通过所述光调制结构对进入至所述光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息；所述入射光携带信息包括光强度分布信息、光谱信息、所述入射光的角度信息以及所述入射光的相位信息；所述入射光包括用户人脸的反射光；
所述图像传感器用于将与不同位置点经光滤波器层调制后对应的入射光携带信息转换为与不同位置点对应的电信号，并将与不同位置点对应的电信号发送给所述处理器；所述电信号为经光滤波器层调制后的图像信号；
所述处理器用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到人脸识别处理结果。
由此可见，本实施例实现了一种能够实现人工神经网络功能的全新光人工神经网络人脸识别芯片，用于人脸识别任务，本申请实施例在硬件芯片上嵌入了人工神经网络，将硬件芯片上的光滤波器层作为人工神经网络的输入层以及输入层到线性层的连接权重，将硬件芯片上的图像传感器作为人工神经网络的线性层，本申请实施例将人脸的空间光谱信息入射到预先训练好的硬件芯片中，通过硬件芯片对人脸的空间光谱信息进行人工神经网络分析进而得出人脸识别结果，需要说明的是，本申请实施例实现了 低功耗、安全可靠的快速准确人脸识别。
可以理解的是，在该光人工神经网络人脸识别芯片中，其上的硬件结构-光滤波器层对应人工神经网络的输入层以及输入层到线性层的连接权重，其上的硬件结构-图像传感器对应人工神经网络的线性层；所述处理器对应人工神经网络的非线性层以及输出层。具体地，光滤波器层设置于图像传感器的感光区域的表面，光滤波器层包含有光调制结构，光滤波器层用于通过光调制结构对进入至光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息，相应地，图像传感器用于将与不同位置点对应的入射光携带信息转换为与不同位置点对应的电信号，与此同时，与图像传感器连接的处理器用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到人工神经网络的输出信号，由此可见，在该人脸识别芯片中，光滤波器层作为人工神经网络的输入层，图像传感器作为人工神经网络的线性层，同时，光滤波器层对进入光滤波器层的入射光的滤波作用对应输入层到线性层的连接权重，也即该人脸识别芯片中的光滤波器层和图像传感器实现了人工神经网络中输入层和线性层的相关功能，也即本申请实施例将现有技术中采用软件实现的人工神经网络中的输入层和线性层进行了剥离，利用硬件的方式实现了人工神经网络中的输入层和线性层这两层结构，从而使得后续在使用该人脸识别芯片进行人工神经网络人脸识别处理时不需要再进行与输入层和线性层对应的复杂的信号处理和算法处理，只需由人脸识别芯片中的处理器进行与电信号全连接与非线性激活的相关处理即可，这样可以大幅降低人工神经网络人脸识别时的功耗和延时。由此可见，本申请实施例将光滤波器层作为人工神经网络的输入层，图像传感器作为人工神经网络的线性层，将光滤波器层对进入光滤波器层的入射光的滤波作用作为输入层到线性层的连接权重，利用光滤波器层和图像传感器将人脸的空间光谱信息投影成电信号，然后在处理器中实现电信号的全连接处理与非线性激活处理，由此可见，本申请实施例不但能够省去现有技术中与输入层和线性层对应的复杂的信号处理和算法处理，而且本申请实施例实际上同时利用了人脸的图像信息、光谱信息、入射光的角度信息以及入射光的相位信息，即人脸空间不同点处的入射光携带信息，由 此可见，由于人脸空间不同点处的入射光携带信息涵盖了人脸的图像、成分、形状、三维深度、结构等信息，从而在依据人脸空间不同点处的入射光携带信息进行识别处理时，可以涵盖人脸的图像、成分、形状、三维深度、结构等多维度的信息，从而可以准确地进行人脸识别。
进一步地，所述光人工神经网络人脸识别芯片包括训练好的光调制结构、图像传感器和处理器；
所述训练好的光调制结构、图像传感器和处理器是指利用与人脸识别任务对应的输入训练样本以及输出训练样本，对包含不同的光调制结构、图像传感器和具有不同的全连接参数以及不同的非线性激活参数的处理器的光人工神经网络人脸识别芯片进行训练得到的满足训练收敛条件的光调制结构、图像传感器和处理器；
所述输入训练样本包括由不同人脸反射的入射光；所述输出训练样本包括相应的人脸识别结果。
进一步地，在对包含不同的光调制结构、图像传感器和具有不同的全连接参数和非线性激活参数的处理器的光人工神经网络人脸识别芯片进行训练时，所述不同的光调制结构通过采用计算机光学仿真设计的方式设计实现。
在本实施例中，由于增加了光的频谱调制，可以实现输入为物体图像及其频谱的人工神经网光电芯片，可以实现快速准确、安全可靠的活体人脸识别。
在本实施例中，对于人脸识别，可以先对大量人群的人脸进行采集，通过数据训练得到输入层到线性层的权重，即光滤波器层的系统函数，便可以逆向设计出所需的光滤波器层，将其集成在图像传感器上方。在实际训练时，利用待识别的人脸样本，利用制作完成的光滤波器层的输出，对电信号全连接层的权重进一步训练并优化，便可实现高准确率的光人工神经网络，完成对用户人脸的快速准确识别。
可以理解的是，光滤波器层上的调制结构的具体调制图案是通过前期采集大量人群的人脸，由人工神经网络数据训练设计得到的，通常是不规则形状的结构，当然也有可能是规则形状的结构。
如图16所示，对于人脸识别的完整流程为：环境光或其他光源照射 到用户人脸上，然后反射光由芯片采集，经内部处理后得到识别结果。
可以理解的是，该芯片实际上同时利用了人脸的图像信息、光谱信息、入射光的角度信息和入射光的相位信息，提高了人脸识别的准确性和安全性，尤其对于非活体的人脸模型也能够准确地将其排除在外。同时该芯片在硬件上部分实现了人工神经网络，提高了人脸识别的速度。此外，该芯片方案可以利用现有的CMOS工艺实现量产，降低了器件的体积、功耗和成本。
进一步地，所述光滤波器层中的光调制结构包含规则结构和/或不规则结构；和/或，所述光滤波器层中的光调制结构包含离散型结构和/或连续型结构。
进一步地，所述光滤波器层为单层结构或多层结构。
进一步地，所述光滤波器层中的光调制结构包含由多个微纳单元组成的单元阵列，每个微纳单元对应图像传感器上的一个或多个像素点；各个微纳单元的结构相同或不同。
进一步地，所述微纳单元包含规则结构和/或不规则结构；和/或，所述微纳单元包含离散型结构和/或连续型结构。
进一步地，所述微纳单元包含有多组微纳结构阵列，各组微纳结构阵列的结构相同或不同。
进一步地，各组微纳结构阵列具有宽带滤波或窄带滤波的作用。
进一步地，各组微纳结构阵列为周期结构阵列或非周期结构阵列。
进一步地，所述微纳单元包含的多组微纳结构阵列中有一组或多组空结构。
进一步地，所述微纳单元具有偏振无关特性。
进一步地，所述微纳单元具有四重旋转对称性。
进一步地，所述光滤波器层由一层或多层滤波器层构成；
所述滤波器层是由半导体材料、金属材料、液晶、量子点材料、钙钛矿材料中的一种或多种制备；和/或，所述滤波器层是由光子晶体、超表面、随机结构、纳米结构、金属表面等离激元SPP微纳结构、可调法布里-珀罗谐振腔中的一种或多种制备的滤波器层。
进一步地，所述半导体材料包括硅、氧化硅、氮化硅、氧化钛、按照 预设比例混合的复合材料以及直接带隙化合物半导体材料中的一种或多种；和/或，所述纳米结构包括纳米点二维材料、纳米柱二维材料和纳米线二维材料中的一种或多种。
进一步地，所述光滤波器层的厚度为0.1λ～10λ，其中λ表示入射光的中心波长。
本申请实施例还提供了一种人脸识别设备，包括上面实施例所述的光人工神经网络人脸识别芯片。该人脸识别设备可以为便携式的人脸识别设备，也可以是安装在固定位置的人脸识别设备。由于该人脸识别设备具有和上述光人工神经网络人脸识别芯片类似的有益效果，故此处不再赘述。
本申请实施例还提供了一种如上面所述的光人工神经网络人脸识别芯片的制备方法，包括：
在所述图像传感器的感光区域的表面制备包含有光调制结构的光滤波器层；
生成具备对信号进行全连接处理与非线性激活处理功能的处理器；
连接所述图像传感器和所述处理器；
其中，所述光滤波器层用于通过所述光调制结构对进入至所述光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息；所述入射光携带信息包括光强度分布信息、光谱信息、所述入射光的角度信息以及所述入射光的相位信息；
所述图像传感器用于将与不同位置点经光滤波器层调制后对应的入射光携带信息转换为与不同位置点对应的电信号，并将与不同位置点对应的电信号发送给所述处理器；所述处理器用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到人脸识别处理结果；所述电信号为经光滤波器层调制后的图像信号，所述入射光包括人脸的反射光。
进一步地，所述光人工神经网络人脸识别芯片的制备方法，还包括：对所述光人工神经网络人脸识别芯片的训练过程，具体包括：
利用与人脸识别任务对应的输入训练样本和输出训练样本，对包含不同的光调制结构、图像传感器和具有不同的全连接参数以及不同的非线性激活参数的处理器的光人工神经网络人脸识别芯片进行训练得到满足训 练收敛条件的光调制结构、图像传感器和处理器，并将满足训练收敛条件的光调制结构、图像传感器和处理器作为训练好的光调制结构、图像传感器和处理器。
进一步地，在对包含不同的光调制结构、图像传感器和具有不同的全连接参数和非线性激活参数的处理器的光人工神经网络人脸识别芯片进行训练时，所述不同的光调制结构通过采用计算机光学仿真设计的方式设计实现。
需要说明的是，本实施例提供的基于微纳调制结构和图像传感器的光人工神经网络人脸识别芯片有以下效果：A、将人工神经网络部分嵌入包含各种光滤波器层的图像传感器中，实现安全可靠、快速准确的人脸识别。B、可以通过CMOS工艺一次流片完成对该芯片的制备，有利于降低器件失效率，提高器件的成品良率，并降低成本。C、在晶圆级别实现单片集成，可以最大程度减小传感器与光滤波器层之间的距离，有利于缩小单元的尺寸，降低器件体积和封装成本。
需要说明的是，关于本实施例提供的光人工神经网络人脸识别芯片的详细结构类描述，可参见前述实施例中对于光人工神经网络芯片的介绍，为避免赘述，此处不再介绍。此外，关于光人工神经网络人脸识别芯片制备方法的详细介绍，也可以参见前述实施例中对于光人工神经网络芯片制备方法的介绍，此处不再赘述。
目前无创血糖检测的方法主要有光学和辐射方法、反向离子电渗分析法、电磁波法、超声波法以及组织液提取法等。例如，近红外光谱检测法主要利用血糖浓度与其近红外光谱吸收之间的关系，用近红外光照射皮肤，并从反射光强度变化来反映出血糖浓度。这种方法具有测量快速、无需化学试剂及消耗品等优点，但是由于被测对象的个体差异大，且取得的信号又非常微弱，因此在测量部位选择、测量条件选取、重叠光谱中提取微弱化学信息的方法等关键性技术方面还有待进一步解决。且信号处理系统体积大，无法随身携带。还有皮下组织液检测法，通过测量皮下渗出的组织液的葡萄糖浓度来反映血糖浓度。根据此原理可制成检测葡萄糖的手表，并可实时连续监控血糖浓度，然而这种方法准确性较差，且反应速度慢，因此很难替代现有的有创血糖仪。因此，无创伤血糖检测成为治疗糖 尿病的迫切需求。
为此，基于前述实施例介绍的光人工神经网络智能芯片，本实施例提供一种用于血糖检测的新型光电芯片，该芯片由光滤波器层构成光人工神经网络的输入层以及输入层到线性层的连接权重，由图像传感器构成光人工神经网络的线性层，通过采集人体待测部位的图像信息和光谱信息，可以实现快速准确、安全可靠的无创血糖检测。下面具体对本实施例的内容进行详细解释和说明。
本申请实施例还提供了一种光人工神经网络血糖检测芯片，用于血糖检测任务，包括：光滤波器层、图像传感器和处理器；所述光滤波器层对应人工神经网络的输入层以及输入层到线性层的连接权重，所述图像传感器对应人工神经网络的线性层；所述处理器对应人工神经网络的非线性层以及输出层；
所述光滤波器层设置于所述图像传感器的感光区域的表面，所述光滤波器层包含有光调制结构，所述光滤波器层用于通过所述光调制结构对进入至所述光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息；所述入射光携带信息包括光强度分布信息、光谱信息、所述入射光的角度信息以及所述入射光的相位信息；所述入射光包括人体待测部位的反射光和/或透射光；
所述图像传感器用于将与不同位置点经光滤波器层调制后对应的入射光携带信息转换为与不同位置点对应的电信号，并将与不同位置点对应的电信号发送给所述处理器；所述电信号为经光滤波器层调制后的图像信号；
所述处理器用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到血糖检测结果。
由此可见，本实施例实现了一种能够实现人工神经网络功能的全新光人工神经网络血糖检测芯片，用于血糖检测任务，本申请实施例在硬件芯片上嵌入了人工神经网络，将硬件芯片上的光滤波器层作为人工神经网络的输入层以及输入层到线性层的连接权重，将硬件芯片上的图像传感器作为人工神经网络的线性层，本申请实施例将人体待测部位的空间光谱信息 入射到预先训练好的硬件芯片中，通过硬件芯片对人体待测部位的空间光谱信息进行人工神经网络分析进而得出血糖检测结果，需要说明的是，本申请实施例实现了低功耗、安全可靠的快速准确无创血糖检测。
可以理解的是，在该光人工神经网络血糖检测芯片中，其上的硬件结构-光滤波器层对应人工神经网络的输入层以及输入层到线性层的连接权重，其上的硬件结构-图像传感器对应人工神经网络的线性层；所述处理器对应人工神经网络的非线性层以及输出层。具体地，光滤波器层设置于图像传感器的感光区域的表面，光滤波器层包含有光调制结构，光滤波器层用于通过光调制结构对进入至光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息，相应地，图像传感器用于将与不同位置点对应的入射光携带信息转换为与不同位置点对应的电信号，与此同时，与图像传感器连接的处理器用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到人工神经网络的输出信号，由此可见，在该血糖检测芯片中，光滤波器层作为人工神经网络的输入层，图像传感器作为人工神经网络的线性层，同时，光滤波器层对进入光滤波器层的入射光的滤波作用对应输入层到线性层的连接权重，也即该血糖检测芯片中的光滤波器层和图像传感器实现了人工神经网络中输入层和线性层的相关功能，也即本申请实施例将现有技术中采用软件实现的人工神经网络中的输入层和线性层进行了剥离，利用硬件的方式实现了人工神经网络中的输入层和线性层这两层结构，从而使得后续在使用该血糖检测芯片进行人工神经网络血糖检测处理时不需要再进行与输入层和线性层对应的复杂的信号处理和算法处理，只需由人脸识别芯片中的处理器进行与电信号全连接与非线性激活的相关处理即可，这样可以大幅降低人工神经网络人脸识别时的功耗和延时。由此可见，本申请实施例将光滤波器层作为人工神经网络的输入层，图像传感器作为人工神经网络的线性层，将光滤波器层对进入光滤波器层的入射光的滤波作用作为输入层到线性层的连接权重，利用光滤波器层和图像传感器将人体待测部位的空间光谱信息投影成电信号，然后在处理器中实现电信号的全连接处理与非线性激活处理，由此可见，本申请实施例不但能够省去现有技术中与输入层和线性层对应的复杂的信号处理和算法处 理，而且本申请实施例实际上同时利用了人体待测部位的血液的图像信息、光谱信息、入射光的角度信息以及入射光的相位信息，即人体待测部位血液空间不同点处的入射光携带信息，由此可见，由于人体待测部位血液空间不同点处的入射光携带信息涵盖了人体待测部位血液的图像、成分、形状、三维深度、结构等信息，从而在依据人体待测部位空间血液不同点处的入射光携带信息进行识别处理时，可以涵盖人体待测部位血液的图像、成分、形状、三维深度、结构等多维度的信息，从而可以准确地进行血糖检测。
进一步地，所述光人工神经网络血糖检测芯片包括训练好的光调制结构、图像传感器和处理器；
所述训练好的光调制结构、图像传感器和处理器是指利用与血糖检测任务对应的输入训练样本以及输出训练样本，对包含不同的光调制结构、图像传感器和具有不同的全连接参数以及不同的非线性激活参数的处理器的光人工神经网络血糖检测芯片进行训练得到的满足训练收敛条件的光调制结构、图像传感器和处理器；
所述输入训练样本包括由具有不同血糖值的人体待测部位反射、透射和/或辐射的入射光；所述输出训练样本包括相应的血糖值。
进一步地，在对包含不同的光调制结构、图像传感器和具有不同的全连接参数和非线性激活参数的处理器的光人工神经网络血糖检测芯片进行训练时，所述不同的光调制结构通过采用计算机光学仿真设计的方式设计实现。
在本实施例中，由于增加了光的频谱调制，可以实现输入为人体待测部位图像及其频谱的人工神经网光电芯片，可以实现快速准确、安全可靠的无创血糖检测。
在本实施例中，对于无创血糖检测，可以先对大量具备血糖信息的人体部位对应的光谱信号数据进行采集，并用计算机模拟入射光经过微纳调制结构的响应，通过数据训练，可以逆向设计出所需的微纳调制结构，将其集成在图像传感器上方。在用户检测血糖的过程中，通过对不同波长的入射光调制得到的电信号进行算法还原，便可实现对该用户血糖值的快速准确检测。
可以理解的是，光滤波器层上的调制结构的具体调制图案是通过前期采集大量具备血糖信息的人体部位对应的光谱信号数据，由人工神经网络数据训练设计得到的，通常是不规则形状的结构，当然也有可能是规则形状的结构。
如图17所示，对于血糖检测的完整流程为：环境光或其他光源照射到人体待测部位(手指)上，然后反射光由芯片采集，经内部处理后得到血糖检测结果。如图18所示，对于血糖检测的完整流程为：环境光或其他光源照射到人体待测部位(手腕)上，然后反射光由芯片采集，经内部处理后得到血糖检测结果。
可以理解的是，该芯片实际上同时利用了人体待测部位的图像信息和光谱信息，提高了血糖检测的准确性和安全性。同时该芯片在硬件上部分实现了人工神经网络，提高了血糖检测的速度。此外，该芯片方案可以利用现有的CMOS工艺实现量产，降低了器件的体积、功耗和成本。
进一步地，所述光滤波器层中的光调制结构包含规则结构和/或不规则结构；和/或，所述光滤波器层中的光调制结构包含离散型结构和/或连续型结构。
进一步地，所述光滤波器层为单层结构或多层结构。
进一步地，所述光滤波器层中的光调制结构包含由多个微纳单元组成的单元阵列，每个微纳单元对应图像传感器上的一个或多个像素点；各个微纳单元的结构相同或不同。
进一步地，所述微纳单元包含规则结构和/或不规则结构；和/或，所述微纳单元包含离散型结构和/或连续型结构。
进一步地，所述微纳单元包含有多组微纳结构阵列，各组微纳结构阵列的结构相同或不同。
进一步地，各组微纳结构阵列具有宽带滤波或窄带滤波的作用。
进一步地，各组微纳结构阵列为周期结构阵列或非周期结构阵列。
进一步地，所述微纳单元包含的多组微纳结构阵列中有一组或多组空结构。
进一步地，所述微纳单元具有偏振无关特性。
进一步地，所述微纳单元具有四重旋转对称性。
进一步地，所述光滤波器层由一层或多层滤波器层构成；
所述滤波器层是由半导体材料、金属材料、液晶、量子点材料、钙钛矿材料中的一种或多种制备；和/或，所述滤波器层是由光子晶体、超表面、随机结构、纳米结构、金属表面等离激元SPP微纳结构、可调法布里-珀罗谐振腔中的一种或多种制备的滤波器层。
进一步地，所述半导体材料包括硅、氧化硅、氮化硅、氧化钛、按照预设比例混合的复合材料以及直接带隙化合物半导体材料中的一种或多种；和/或，所述纳米结构包括纳米点二维材料、纳米柱二维材料和纳米线二维材料中的一种或多种。
进一步地，所述光滤波器层的厚度为0.1λ～10λ，其中λ表示入射光的中心波长。
本申请实施例还提供了一种智能血糖检测仪，包括上面实施例所述的光人工神经网络血糖检测芯片，该血糖检测仪可以为可穿戴式血糖检测仪。由于该智能血糖检测仪具有和上述光人工神经网络血糖检测芯片类似的有益效果，故此处不再赘述。
本申请实施例还提供了一种如上面所述的光人工神经网络血糖检测芯片的制备方法，包括：
在所述图像传感器的感光区域的表面制备包含有光调制结构的光滤波器层；
生成具备对信号进行全连接处理与非线性激活处理功能的处理器；
连接所述图像传感器和所述处理器；
其中，所述光滤波器层用于通过所述光调制结构对进入至所述光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息；所述入射光携带信息包括光强度分布信息、光谱信息、所述入射光的角度信息以及所述入射光的相位信息；
所述图像传感器用于将与不同位置点经光滤波器层调制后对应的入射光携带信息转换为与不同位置点对应的电信号，并将与不同位置点对应的电信号发送给所述处理器；所述处理器用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到血糖检测结果；所述电信号为 经光滤波器层调制后的图像信号，所述入射光包括人体待测部位的反射光和/或透射光。
进一步地，所述光人工神经网络血糖检测芯片的制备方法，还包括：对所述光人工神经网络血糖检测芯片的训练过程，具体包括：
利用与血糖检测任务对应的输入训练样本和输出训练样本，对包含不同的光调制结构、图像传感器和具有不同的全连接参数以及不同的非线性激活参数的处理器的光人工神经网络血糖检测芯片进行训练得到满足训练收敛条件的光调制结构、图像传感器和处理器，并将满足训练收敛条件的光调制结构、图像传感器和处理器作为训练好的光调制结构、图像传感器和处理器。
进一步地，在对包含不同的光调制结构、图像传感器和具有不同的全连接参数和非线性激活参数的处理器的光人工神经网络血糖检测芯片进行训练时，所述不同的光调制结构通过采用计算机光学仿真设计的方式设计实现。
需要说明的是，本实施例提供的基于微纳调制结构和图像传感器的光人工神经网络血糖检测芯片有以下效果：A、将人工神经网络部分嵌入包含各种光滤波器层的图像传感器中，实现安全可靠、快速准确的非侵入式血糖检测。B、可以通过CMOS工艺一次流片完成对该芯片的制备，有利于降低器件失效率，提高器件的成品良率，并降低成本。C、在晶圆级别实现单片集成，可以最大程度减小传感器与光滤波器层之间的距离，有利于缩小单元的尺寸，降低器件体积和封装成本。
需要说明的是，关于本实施例提供的光人工神经网络血糖检测芯片的详细结构类描述，可参见前述实施例中对于光人工神经网络芯片的介绍，为避免赘述，此处不再介绍。此外，关于光人工神经网络血糖检测芯片制备方法的详细介绍，也可以参见前述实施例中对于光人工神经网络智能芯片制备方法的介绍，此处不再赘述。
精准农业是由信息技术支持的根据空间变异，定位、定时、定量地实施一整套现代化农事操作技术与管理的系统，其基本涵义是根据作物生长的土壤性状，调节对作物的投入，判断农田内部的土壤性状与生产力空间变异，另一方面确定农作物的生产目标，进行定位的“系统诊断、优化配 方、技术组装、科学管理”，调动土壤生产力，以最少的或最节省的投入达到同等收入或更高的收入，并改善环境，高效地利用各类农业资源，取得经济效益和环境效益。
目前在进行精准农业控制时，存在处理速度慢，识别不准确的问题。例如，对于土壤肥力的识别，需要进行图像采集、图像预处理、特征提取、特征匹配等步骤，以实现对土壤肥力的识别。然而，目前只利用土壤的二维图像信息难以保证识别的准确性，例如二维图像存在失真因素等，并且，成像过程需要将光学信息转换为数字电子信号，再传输到计算机中进行后续的算法处理，大量数据的传输和处理造成了较大的功耗和延时。此外，对于农作物生长状况的识别也存在类似的问题。
精准农业的技术原理是根据土壤肥力和农作物生长状况的空间差异，调节对作物的投入，当对土壤肥力和农作物生长状况识别不准确以及存在较大的功耗和时延时，将会严重影响精准农业的发展。
为此，基于前述实施例介绍的光人工神经网络智能芯片，本实施例提供一种用于农业精准控制智能处理任务的新型光电芯片，该芯片由光滤波器层构成光人工神经网络的输入层以及输入层到线性层的连接权重，由图像传感器构成光人工神经网络的线性层，通过采集农田土壤及作物的图像信息和光谱信息，可以实现快速准确、安全可靠的土壤肥力、农药布撒情况、微量元素含量以及作物生长状况等的识别和定性分析。下面具体对本实施例的内容进行详细解释和说明。
本申请实施例还提供了一种光人工神经网络智能农业精准控制芯片，用于农业精准控制智能处理任务，包括：光滤波器层、图像传感器和处理器；所述光滤波器层对应人工神经网络的输入层以及输入层到线性层的连接权重，所述图像传感器对应人工神经网络的线性层；所述处理器对应人工神经网络的非线性层以及输出层；
所述光滤波器层设置于所述图像传感器的感光区域的表面，所述光滤波器层包含有光调制结构，所述光滤波器层用于通过所述光调制结构对进入至所述光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息；所述入射光携带信息包括光强度分布信息、光谱信息、所述入射光的角度信息以 及所述入射光的相位信息；所述入射光包括农业对象的反射光、透射光和/或辐射光；所述农业对象包括农作物和/或土壤；
所述图像传感器用于将与不同位置点经光滤波器层调制后对应的入射光携带信息转换为与不同位置点对应的电信号，并将与不同位置点对应的电信号发送给所述处理器；所述电信号为经光滤波器层调制后的图像信号；
所述处理器用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到农业精准控制处理结果；
其中，所述农业精准控制智能处理任务包括：土壤肥力检测、农药布撒检测、微量元素含量检测、抗药性检测以及农作物生长状况检测中的一种或多种；所述农业精准控制处理结果包括：土壤肥力检测结果、农药布撒检测结果、微量元素含量检测结果、抗药性检测结果以及农作物生长状况检测结果中的一种或多种。
由此可见，本实施例实现了一种能够实现人工神经网络功能的全新光人工神经网络智能农业精准控制芯片，用于农业精准控制智能处理任务，本申请实施例在硬件芯片上嵌入了人工神经网络，将硬件芯片上的光滤波器层作为人工神经网络的输入层以及输入层到线性层的连接权重，将硬件芯片上的图像传感器作为人工神经网络的线性层，本申请实施例将农田土壤及作物的空间光谱信息入射到预先训练好的硬件芯片中，通过硬件芯片对农田土壤及作物的空间光谱信息进行人工神经网络分析进而得出农业精准控制处理结果，需要说明的是，本申请实施例实现了低功耗、安全可靠的快速准确的土壤肥力、农药布撒情况、微量元素含量以及作物生长状况等的识别和定性分析。
可以理解的是，在该光人工神经网络智能农业精准控制芯片中，其上的硬件结构-光滤波器层对应人工神经网络的输入层以及输入层到线性层的连接权重，其上的硬件结构-图像传感器对应人工神经网络的线性层；所述处理器对应人工神经网络的非线性层以及输出层。具体地，光滤波器层设置于图像传感器的感光区域的表面，光滤波器层包含有光调制结构，光滤波器层用于通过光调制结构对进入至光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点 对应的入射光携带信息，相应地，图像传感器用于将与不同位置点对应的入射光携带信息转换为与不同位置点对应的电信号，与此同时，与图像传感器连接的处理器用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到人工神经网络的输出信号，由此可见，在该智能农业精准控制芯片中，光滤波器层作为人工神经网络的输入层，图像传感器作为人工神经网络的线性层，同时，光滤波器层对进入光滤波器层的入射光的滤波作用对应输入层到线性层的连接权重，也即该智能农业精准控制芯片中的光滤波器层和图像传感器实现了人工神经网络中输入层和线性层的相关功能，也即本申请实施例将现有技术中采用软件实现的人工神经网络中的输入层和线性层进行了剥离，利用硬件的方式实现了人工神经网络中的输入层和线性层这两层结构，从而使得后续在使用该智能农业精准控制芯片进行人工神经网络农业精准控制智能处理时不需要再进行与输入层和线性层对应的复杂的信号处理和算法处理，只需由智能农业精准控制芯片中的处理器进行与电信号全连接与非线性激活的相关处理即可，这样可以大幅降低人工神经网络智能农业精准控制时的功耗和延时。由此可见，本申请实施例将光滤波器层作为人工神经网络的输入层，图像传感器作为人工神经网络的线性层，将光滤波器层对进入光滤波器层的入射光的滤波作用作为输入层到线性层的连接权重，利用光滤波器层和图像传感器将人脸的空间光谱信息投影成电信号，然后在处理器中实现电信号的全连接处理与非线性激活处理，由此可见，本申请实施例不但能够省去现有技术中与输入层和线性层对应的复杂的信号处理和算法处理，而且本申请实施例实际上同时利用了农田土壤及作物的图像信息、光谱信息、入射光的角度信息以及入射光的相位信息，即农田土壤及作物空间不同点处的入射光携带信息，由此可见，由于农田土壤及作物空间不同点处的入射光携带信息涵盖了农田土壤及作物的图像、成分、形状、三维深度、结构等信息，从而在依据农田土壤及作物空间不同点处的入射光携带信息进行识别处理时，可以涵盖农田土壤及作物的图像、成分、形状、三维深度、结构等多维度的信息，从而可以准确地进行土壤肥力、农药布撒情况、微量元素含量以及作物生长状况等的识别和定性分析。
进一步地，所述光人工神经网络智能农业精准控制芯片包括训练好的 光调制结构、图像传感器和处理器；
所述训练好的光调制结构、图像传感器和处理器是指利用与所述农业精准控制智能处理任务对应的输入训练样本以及输出训练样本，对包含不同的光调制结构、图像传感器和具有不同的全连接参数以及不同的非线性激活参数的处理器的光人工神经网络智能农业精准控制芯片进行训练得到的满足训练收敛条件的光调制结构、图像传感器和处理器；
其中，所述输入训练样本包括由具备不同肥力的土壤反射、透射和/或辐射的入射光；所述输出训练样本包括相应的土壤肥力；
和/或，
所述输入训练样本包括由具备不同农药布撒状况的土壤反射、透射和/或辐射的入射光；所述输出训练样本包括相应的农药布撒状况；
和/或，
所述输入训练样本包括由具备不同微量元素含量状况的土壤反射、透射和/或辐射的入射光；所述输出训练样本包括相应的微量元素含量状况；
和/或，
所述输入训练样本包括由具备不同抗药性的土壤反射、透射和/或辐射的入射光；所述输出训练样本包括相应的抗药性；
和/或，
所述输入训练样本包括由具备不同生长状况的农作物反射、透射和/或辐射的入射光；所述输出训练样本包括相应的农作物生长状况。
进一步地，在对包含不同的光调制结构、图像传感器和具有不同的全连接参数和非线性激活参数的处理器的光人工神经网络智能农业精准控制芯片进行训练时，所述不同的光调制结构通过采用计算机光学仿真设计的方式设计实现。
在本实施例中，对于农业精准控制智能处理任务，可以先对大量不同状态、位置的土壤样本、农作物样本进行采集，通过数据训练得到输入层到线性层的权重，即光滤波器层的系统函数，便可以逆向设计出所需的光滤波器层，将其集成在图像传感器上方。在实际训练时，利用土壤样本、农作物样本，利用制作完成的光滤波器层的输出，对电信号全连接层的权重进一步训练并优化，便可实现高准确率的光人工神经网络，完成对土壤 肥力、农药布撒情况、微量元素含量以及作物生长状况等的识别和定性分析。
可以理解的是，光滤波器层上的调制结构的具体调制图案是通过前期采集大量不同状态、位置的土壤样本、农作物样本，由人工神经网络数据训练设计得到的，通常是不规则形状的结构，当然也有可能是规则形状的结构。
如图19所示，对于农业对象识别的完整流程为：环境光或其他光源照射到农业对象上，然后反射光由芯片采集，经内部处理后得到识别结果。如图20所示，由光谱、图像采集器采集农田土壤、农作物的光谱数据及图像，然后反射光由精准农业控制芯片采集，再经处理器的内部算法处理，即可得到识别结果。
可以理解的是，该芯片实际上同时利用了农田土壤及作物的图像信息和光谱信息，实现安全可靠、快速准确的精准农业控制。此外，该芯片方案可以利用现有的CMOS工艺实现量产，降低了器件的体积、功耗和成本。
进一步地，所述光滤波器层中的光调制结构包含规则结构和/或不规则结构；和/或，所述光滤波器层中的光调制结构包含离散型结构和/或连续型结构。
进一步地，所述光滤波器层为单层结构或多层结构。
进一步地，所述光滤波器层中的光调制结构包含由多个微纳单元组成的单元阵列，每个微纳单元对应图像传感器上的一个或多个像素点；各个微纳单元的结构相同或不同。
进一步地，所述微纳单元包含规则结构和/或不规则结构；和/或，所述微纳单元包含离散型结构和/或连续型结构。
进一步地，所述微纳单元包含有多组微纳结构阵列，各组微纳结构阵列的结构相同或不同。
进一步地，各组微纳结构阵列具有宽带滤波或窄带滤波的作用。
进一步地，各组微纳结构阵列为周期结构阵列或非周期结构阵列。
进一步地，所述微纳单元包含的多组微纳结构阵列中有一组或多组空结构。
进一步地，所述微纳单元具有偏振无关特性。
进一步地，所述微纳单元具有四重旋转对称性。
进一步地，所述光滤波器层由一层或多层滤波器层构成；
所述滤波器层是由半导体材料、金属材料、液晶、量子点材料、钙钛矿材料中的一种或多种制备；和/或，所述滤波器层是由光子晶体、超表面、随机结构、纳米结构、金属表面等离激元SPP微纳结构、可调法布里-珀罗谐振腔中的一种或多种制备的滤波器层。
进一步地，所述半导体材料包括硅、氧化硅、氮化硅、氧化钛、按照预设比例混合的复合材料以及直接带隙化合物半导体材料中的一种或多种；和/或，所述纳米结构包括纳米点二维材料、纳米柱二维材料和纳米线二维材料中的一种或多种。
进一步地，所述光滤波器层的厚度为0.1λ～10λ，其中λ表示入射光的中心波长。
本申请实施例还提供了一种智能农业控制设备，包括：如上面实施例所述的光人工神经网络智能农业精准控制芯片。所述智能农业控制设备可以包括施肥机设备、撒药机设备、抗药性分析设备、无人机设备、农业智能机器人设备、农作物健康分析设备、农作物生长状监测设备等。
本申请另一实施例提供了一种如上面所述实施例的光人工神经网络智能农业精准控制芯片的制备方法，包括：
在所述图像传感器的感光区域的表面制备包含有光调制结构的光滤波器层；
生成具备对信号进行全连接处理与非线性激活处理功能的处理器；
连接所述图像传感器和所述处理器；
其中，所述光滤波器层用于通过所述光调制结构对进入至所述光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息；所述入射光携带信息包括光强度分布信息、光谱信息、所述入射光的角度信息以及所述入射光的相位信息；
所述图像传感器用于将与不同位置点经光滤波器层调制后对应的入射光携带信息转换为与不同位置点对应的电信号，并将与不同位置点对应的电信号发送给所述处理器；所述处理器用于将与不同位置点对应的电信 号进行全连接处理与非线性激活处理，得到农业精准控制处理结果；所述电信号为经光滤波器层调制后的图像信号，所述入射光包括农业对象的反射光、透射光和/或辐射光；所述农业对象包括农作物和/或土壤。
进一步地，所述光人工神经网络智能农业精准控制芯片的制备方法，还包括：对所述光人工神经网络智能农业精准控制芯片的训练过程，具体包括：
利用与农业精准控制智能处理任务对应的输入训练样本和输出训练样本，对包含不同的光调制结构、图像传感器和具有不同的全连接参数以及不同的非线性激活参数的处理器的光人工神经网络智能农业精准控制芯片进行训练得到满足训练收敛条件的光调制结构、图像传感器和处理器，并将满足训练收敛条件的光调制结构、图像传感器和处理器作为训练好的光调制结构、图像传感器和处理器。
进一步地，在对包含不同的光调制结构、图像传感器和具有不同的全连接参数和非线性激活参数的处理器的光人工神经网络智能农业精准控制芯片进行训练时，所述不同的光调制结构通过采用计算机光学仿真设计的方式设计实现。
需要说明的是，本实施例提供的基于微纳调制结构和图像传感器的光人工神经网络智能农业精准控制芯片有以下效果：A、将人工神经网络部分嵌入包含各种光滤波器层的图像传感器中，实现安全可靠、快速准确的精准农业控制。B、可对土壤、农作物等引入人工神经网训练识别，便于后续集成到无人机、智能机器人等工业智能控制系统实现大区域的精准农业控制，且识别准确性高、定性分析精准。C、可以通过CMOS工艺一次流片完成对该芯片的制备，有利于降低器件失效率，提高器件的成品良率，并降低成本。D、在晶圆级别实现单片集成，可以最大程度减小传感器与光滤波器层之间的距离，有利于缩小单元的尺寸，降低器件体积和封装成本。
需要说明的是，关于本实施例提供的光人工神经网络智能农业精准控制芯片的详细结构类描述，可参见前述实施例中对于光人工神经网络智能芯片的介绍，为避免赘述，此处不再介绍。此外，关于光人工神经网络智能农业精准控制芯片制备方法的详细介绍，也可以参见前述实施例中对于 光人工神经网络智能芯片制备方法的介绍，此处不再赘述。
转炉炼钢是目前世界上应用最广泛、最高效的炼钢方法，冶炼终点控制是转炉生产中的关键技术之一，终点的准确判断在提高钢水质量、缩短冶炼周期方面具有重要的意义。但由于入炉原料不稳定、复杂的化学反应和所炼钢种的严格等原因，冶炼终点的准确控制仍是难点。对终点碳含量、钢水温度进行准确地在线检测，一直是全世界冶金行业亟待解决的难题。目前行业内用于终点控制主要依赖人工经验、或复杂的大型仪器设备测炉口温度及炉渣残留物定性测量，精度低且成本较高。
为此，基于前述实施例介绍的光人工神经网络智能芯片，本实施例提供一种用于冶炼终点监测的新型光电芯片，该芯片由光滤波器层构成光人工神经网络的输入层以及输入层到线性层的连接权重，由图像传感器构成光人工神经网络的线性层，通过采集炼钢炉口的图像信息和光谱信息，可以实现快速准确、安全可靠的识别冶炼终点。下面具体对本实施例的内容进行详细解释和说明。
本申请实施例还提供了一种光人工神经网络冶炼终点监测芯片，用于冶炼终点监测任务，包括：光滤波器层、图像传感器和处理器；所述光滤波器层对应人工神经网络的输入层以及输入层到线性层的连接权重，所述图像传感器对应人工神经网络的线性层；所述处理器对应人工神经网络的非线性层以及输出层；
所述光滤波器层设置于所述图像传感器的感光区域的表面，所述光滤波器层包含有光调制结构，所述光滤波器层用于通过所述光调制结构对进入至所述光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息；所述入射光携带信息包括光强度分布信息、光谱信息、所述入射光的角度信息以及所述入射光的相位信息；所述入射光包括炼钢炉口的反射光、透射光和/或辐射光；
所述图像传感器用于将与不同位置点经光滤波器层调制后对应的入射光携带信息转换为与不同位置点对应的电信号，并将与不同位置点对应的电信号发送给所述处理器；所述电信号为经光滤波器层调制后的图像信号；
所述处理器用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到冶炼终点监测结果；
其中，所述冶炼终点监测任务包括识别冶炼终点，所述冶炼终点监测结果包括冶炼终点识别结果。
由此可见，本实施例实现了一种能够实现人工神经网络功能的全新光人工神经网络冶炼终点监测芯片，用于冶炼终点监测任务，本申请实施例在硬件芯片上嵌入了人工神经网络，将硬件芯片上的光滤波器层作为人工神经网络的输入层以及输入层到线性层的连接权重，将硬件芯片上的图像传感器作为人工神经网络的线性层，本申请实施例将炼钢炉口的空间光谱信息入射到预先训练好的硬件芯片中，通过硬件芯片对炼钢炉口的空间光谱信息进行人工神经网络分析进而得出冶炼终点识别结果，需要说明的是，本申请实施例实现了低功耗、安全可靠的快速准确冶炼终点识别。
可以理解的是，在该光人工神经网络冶炼终点监测芯片中，其上的硬件结构-光滤波器层对应人工神经网络的输入层以及输入层到线性层的连接权重，其上的硬件结构-图像传感器对应人工神经网络的线性层；所述处理器对应人工神经网络的非线性层以及输出层。具体地，光滤波器层设置于图像传感器的感光区域的表面，光滤波器层包含有光调制结构，光滤波器层用于通过光调制结构对进入至光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息，相应地，图像传感器用于将与不同位置点对应的入射光携带信息转换为与不同位置点对应的电信号，与此同时，与图像传感器连接的处理器用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到人工神经网络的输出信号，由此可见，在该冶炼终点监测芯片中，光滤波器层作为人工神经网络的输入层，图像传感器作为人工神经网络的线性层，同时，光滤波器层对进入光滤波器层的入射光的滤波作用对应输入层到线性层的连接权重，也即该冶炼终点监测芯片中的光滤波器层和图像传感器实现了人工神经网络中输入层和线性层的相关功能，也即本申请实施例将现有技术中采用软件实现的人工神经网络中的输入层和线性层进行了剥离，利用硬件的方式实现了人工神经网络中的输入层和线性层这两层结构，从而使得后续在使用该冶炼终点监测芯片进行人工 神经网络冶炼终点识别处理时不需要再进行与输入层和线性层对应的复杂的信号处理和算法处理，只需由冶炼终点监测芯片中的处理器进行与电信号全连接与非线性激活的相关处理即可，这样可以大幅降低人工神经网络冶炼终点监测时的功耗和延时。由此可见，本申请实施例将光滤波器层作为人工神经网络的输入层，图像传感器作为人工神经网络的线性层，将光滤波器层对进入光滤波器层的入射光的滤波作用作为输入层到线性层的连接权重，利用光滤波器层和图像传感器将炼钢炉口的空间光谱信息投影成电信号，然后在处理器中实现电信号的全连接处理与非线性激活处理，由此可见，本申请实施例不但能够省去现有技术中与输入层和线性层对应的复杂的信号处理和算法处理，而且本申请实施例实际上同时利用了炼钢炉口的图像信息、光谱信息、入射光的角度信息以及入射光的相位信息，即炼钢炉口空间不同点处的入射光携带信息，由此可见，由于炼钢炉口空间不同点处的入射光携带信息涵盖了炼钢炉口的图像、成分、形状、三维深度、结构等信息，从而在依据炼钢炉口空间不同点处的入射光携带信息进行识别处理时，可以涵盖炼钢炉口的图像、成分、形状、三维深度、结构等多维度的信息，从而可以准确地进行冶炼终点识别。
进一步地，所述冶炼终点监测任务还包括识别冶炼过程中的碳含量和/钢水温度，所述冶炼终点监测结果包括冶炼过程中的碳含量和/钢水温度识别结果。
进一步地，所述光人工神经网络冶炼终点监测芯片包括训练好的光调制结构、图像传感器和处理器；
所述训练好的光调制结构、图像传感器和处理器是指利用与冶炼终点监测任务对应的输入训练样本以及输出训练样本，对包含不同的光调制结构、图像传感器和具有不同的全连接参数以及不同的非线性激活参数的处理器的光人工神经网络冶炼终点监测芯片进行训练得到的满足训练收敛条件的光调制结构、图像传感器和处理器；
所述输入训练样本包括由冶炼至终点以及未冶炼至终点的炼钢炉口反射、透射和/或辐射的入射光；所述输出训练样本包括是否冶炼至终点的判定结果。
进一步地，当所述冶炼终点监测任务还包括识别冶炼过程中的碳含量 和/钢水温度时，相应地，所述输入训练样本还包括由冶炼至不同碳含量和/钢水温度的炼钢炉口反射、透射和/或辐射的入射光，所述输出训练样本还包括相应的碳含量和/钢水温度。
进一步地，在对包含不同的光调制结构、图像传感器和具有不同的全连接参数和非线性激活参数的处理器的光人工神经网络冶炼终点监测芯片进行训练时，所述不同的光调制结构通过采用计算机光学仿真设计的方式设计实现。
在本实施例中，对于冶炼终点识别，可以先对大量终点时刻的转炉炉口图像和光谱信息进行采集，通过数据训练得到输入层到线性层的权重，即光滤波器层的系统函数，便可以逆向设计出所需的光滤波器层，将其集成在图像传感器上方。在实际训练时，利用待识别的炼钢炉口样本，利用制作完成的光滤波器层的输出，对电信号全连接层的权重进一步训练并优化，便可实现高准确率的光人工神经网络，完成对冶炼终点的快速准确识别。
可以理解的是，光滤波器层上的调制结构的具体调制图案是通过前期采集大量终点时刻的转炉炉口图像和光谱信息，由人工神经网络数据训练设计得到的，通常是不规则形状的结构，当然也有可能是规则形状的结构。
如图21所示，对于炼钢炉口进行识别以确定是否为冶炼终点的完整流程为：环境光或其他光源照射到炼钢炉口上，然后反射光由芯片采集，经内部处理后得到识别结果。
可以理解的是，该芯片实际上同时利用了炼钢炉口的图像信息和光谱信息，提高了冶炼终点识别的准确性。同时该芯片在硬件上部分实现了人工神经网络，提高了冶炼终点识别的速度。此外，该芯片方案可以利用现有的CMOS工艺实现量产，降低了器件的体积、功耗和成本。
进一步地，所述光滤波器层中的光调制结构包含规则结构和/或不规则结构；和/或，所述光滤波器层中的光调制结构包含离散型结构和/或连续型结构。
进一步地，所述光滤波器层为单层结构或多层结构。
进一步地，所述光滤波器层中的光调制结构包含由多个微纳单元组成的单元阵列，每个微纳单元对应图像传感器上的一个或多个像素点；各个 微纳单元的结构相同或不同。
进一步地，所述微纳单元包含规则结构和/或不规则结构；和/或，所述微纳单元包含离散型结构和/或连续型结构。
进一步地，所述微纳单元包含有多组微纳结构阵列，各组微纳结构阵列的结构相同或不同。
进一步地，各组微纳结构阵列具有宽带滤波或窄带滤波的作用。
进一步地，各组微纳结构阵列为周期结构阵列或非周期结构阵列。
进一步地，所述微纳单元包含的多组微纳结构阵列中有一组或多组空结构。
进一步地，所述微纳单元具有偏振无关特性。
进一步地，所述微纳单元具有四重旋转对称性。
进一步地，所述光滤波器层由一层或多层滤波器层构成；
所述滤波器层是由半导体材料、金属材料、液晶、量子点材料、钙钛矿材料中的一种或多种制备；和/或，所述滤波器层是由光子晶体、超表面、随机结构、纳米结构、金属表面等离激元SPP微纳结构、可调法布里-珀罗谐振腔中的一种或多种制备的滤波器层。
进一步地，所述半导体材料包括硅、氧化硅、氮化硅、氧化钛、按照预设比例混合的复合材料以及直接带隙化合物半导体材料中的一种或多种；和/或，所述纳米结构包括纳米点二维材料、纳米柱二维材料和纳米线二维材料中的一种或多种。
进一步地，所述光滤波器层的厚度为0.1λ～10λ，其中λ表示入射光的中心波长。
本申请实施例还提供了一种智能冶炼控制设备，包括：如上面实施例所述的光人工神经网络冶炼终点监测芯片。所述智能冶炼控制设备可以包括各种与冶炼过程控制有关的设备，本实施例对此不作限定。本实施例提供的智能冶炼控制设备具备上述实施例所述的光人工神经网络冶炼终点监测芯片的全部有益效果，由于上述实施例已经对此进行了较为详尽的描述，因此本实施例不再赘述。
本申请实施例还提供了一种如上面所述的光人工神经网络冶炼终点监测的制备方法，包括：
在所述图像传感器的感光区域的表面制备包含有光调制结构的光滤波器层；
生成具备对信号进行全连接处理与非线性激活处理功能的处理器；
连接所述图像传感器和所述处理器；
其中，所述光滤波器层用于通过所述光调制结构对进入至所述光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息；所述入射光携带信息包括光强度分布信息、光谱信息、所述入射光的角度信息以及所述入射光的相位信息；
所述图像传感器用于将与不同位置点经光滤波器层调制后对应的入射光携带信息转换为与不同位置点对应的电信号，并将与不同位置点对应的电信号发送给所述处理器；所述处理器用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到冶炼终点监测结果；所述电信号为经光滤波器层调制后的图像信号，所述入射光包括炼钢炉口的反射光、透射光和/或辐射光。
进一步地，所述光人工神经网络冶炼终点监测芯片的制备方法，还包括：对所述光人工神经网络冶炼终点监测芯片的训练过程，具体包括：
利用与冶炼终点监测任务对应的输入训练样本和输出训练样本，对包含不同的光调制结构、图像传感器和具有不同的全连接参数以及不同的非线性激活参数的处理器的光人工神经网络冶炼终点监测芯片进行训练得到满足训练收敛条件的光调制结构、图像传感器和处理器，并将满足训练收敛条件的光调制结构、图像传感器和处理器作为训练好的光调制结构、图像传感器和处理器。
进一步地，在对包含不同的光调制结构、图像传感器和具有不同的全连接参数和非线性激活参数的处理器的光人工神经网络冶炼终点监测芯片进行训练时，所述不同的光调制结构通过采用计算机光学仿真设计的方式设计实现。
需要说明的是，本实施例提供的基于微纳调制结构和图像传感器的光人工神经网络冶炼终点监测芯片有以下效果：A、将人工神经网络部分嵌入包含各种光滤波器层的图像传感器中，实现安全可靠、快速准确的冶炼 终点控制。B、可检测样本包括但不限于转炉炼钢的终点控制，引入人工神经网训练检测冶炼炉口温度和物质元素，极易于后端工业控制系统的集成、且识别准确性高、定性分析精准。C、可以通过CMOS工艺一次流片完成对该芯片的制备，有利于降低器件失效率，提高器件的成品良率，并降低成本。D、在晶圆级别实现单片集成，可以最大程度减小传感器与光滤波器层之间的距离，有利于缩小单元的尺寸，降低器件体积和封装成本。
需要说明的是，关于本实施例提供的光人工神经网络冶炼终点监测芯片的详细结构类描述，可参见前述实施例中对于光人工神经网络智能芯片的介绍，为避免赘述，此处不再介绍。此外，关于光人工神经网络冶炼终点监测芯片制备方法的详细介绍，也可以参见前述实施例中对于光人工神经网络智能芯片制备方法的介绍，此处不再赘述。
最后应说明的是：以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围。

Claims (25)

  1. 一种光人工神经网络智能芯片，其特征在于，包括：光滤波器层、图像传感器和处理器；所述光滤波器层对应人工神经网络的输入层以及输入层到线性层的连接权重，所述图像传感器对应人工神经网络的线性层；所述处理器对应人工神经网络的非线性层以及输出层；
    所述光滤波器层设置于所述图像传感器的感光区域的表面，所述光滤波器层包含有光调制结构，所述光滤波器层用于通过所述光调制结构对进入至所述光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息；所述入射光携带信息包括光强度分布信息、光谱信息、所述入射光的角度信息以及所述入射光的相位信息；
    所述图像传感器用于将与不同位置点经光滤波器层调制后对应的入射光携带信息转换为与不同位置点对应的电信号，并将与不同位置点对应的电信号发送给所述处理器；所述电信号为经光滤波器层调制后的图像信号；
    所述处理器用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到人工神经网络的输出信号。
  2. 根据权利要求1所述的光人工神经网络智能芯片，其特征在于，所述光人工神经网络智能芯片用于目标对象的智能处理任务；所述智能处理任务至少包括智能感知、智能识别和智能决策任务中的一种或多种；
    目标对象的反射光、透射光和/或辐射光进入至训练好的光人工神经网络智能芯片中，得到所述目标对象的智能处理结果；所述智能处理结果至少包括智能感知结果、智能识别结果和/或智能决策结果中的一种或多种；
    其中，训练好的光人工神经网络智能芯片是指包括训练好的光调制结构、图像传感器和处理器的光人工神经网络智能芯片；所述训练好的光调制结构、图像传感器和处理器是指利用与所述智能处理任务对应的输入训练样本和输出训练样本，对包含不同的光调制结构、图像传感器和具有不同的全连接参数与非线性激活参数的处理器的光人工神经网络智能芯片进行训练得到的满足训练收敛条件的光调制结构、图像传感器和处理器。
  3. 根据权利要求2所述的光人工神经网络智能芯片，其特征在于， 在对包含不同的光调制结构、图像传感器和具有不同的全连接参数和非线性激活参数的处理器的光人工神经网络智能芯片进行训练时，所述不同的光调制结构通过采用计算机光学仿真设计的方式设计实现。
  4. 根据权利要求1～3任一项所述的光人工神经网络智能芯片，其特征在于，所述光滤波器层中的光调制结构包含规则结构和/或不规则结构；和/或，所述光滤波器层中的光调制结构包含离散型结构和/或连续型结构。
  5. 根据权利要求1～3任一项所述的光人工神经网络智能芯片，其特征在于，所述光滤波器层为单层结构或多层结构。
  6. 根据权利要求1～3任一项所述的光人工神经网络智能芯片，其特征在于，所述光滤波器层中的光调制结构包含由多个微纳单元组成的单元阵列，每个微纳单元对应图像传感器上的一个或多个像素点；各个微纳单元的结构相同或不同。
  7. 根据权利要求6所述的光人工神经网络智能芯片，其特征在于，所述微纳单元包含规则结构和/或不规则结构；和/或，所述微纳单元包含离散型结构和/或连续型结构。
  8. 根据权利要求6所述的光人工神经网络智能芯片，其特征在于，所述微纳单元包含有多组微纳结构阵列，各组微纳结构阵列的结构相同或不同。
  9. 根据权利要求8所述的光人工神经网络智能芯片，其特征在于，各组微纳结构阵列具有宽带滤波或窄带滤波的作用。
  10. 根据权利要求8所述的光人工神经网络智能芯片，其特征在于，各组微纳结构阵列为周期结构阵列或非周期结构阵列。
  11. 根据权利要求8所述的光人工神经网络智能芯片，其特征在于，所述微纳单元包含的多组微纳结构阵列中有一组或多组空结构。
  12. 根据权利要求8所述的光人工神经网络智能芯片，其特征在于，所述微纳单元具有偏振无关特性。
  13. 根据权利要求12所述的光人工神经网络智能芯片，其特征在于，所述微纳单元具有四重旋转对称性。
  14. 根据权利要求1所述的光人工神经网络智能芯片，其特征在于，所述光滤波器层由一层或多层滤波器层构成；
    所述滤波器层是由半导体材料、金属材料、液晶、量子点材料、钙钛矿材料中的一种或多种制备；和/或，所述滤波器层是由光子晶体、超表面、随机结构、纳米结构、金属表面等离激元SPP微纳结构、可调法布里-珀罗谐振腔中的一种或多种制备的滤波器层。
  15. 根据权利要求14所述的光人工神经网络智能芯片，其特征在于，所述半导体材料包括硅、氧化硅、氮化硅、氧化钛、按照预设比例混合的复合材料以及直接带隙化合物半导体材料中的一种或多种；和/或，所述纳米结构包括纳米点二维材料、纳米柱二维材料和纳米线二维材料中的一种或多种。
  16. 根据权利要求1所述的光人工神经网络智能芯片，其特征在于，所述光滤波器层的厚度为0.1λ～10λ，其中λ表示入射光的中心波长。
  17. 根据权利要求1所述的光人工神经网络智能芯片，其特征在于，所述图像传感器为下述中的任意一项或多项：
    CMOS图像传感器CIS、电荷耦合元件CCD、单光子雪崩二极管SPAD阵列和焦平面光电探测器阵列。
  18. 根据权利要求1所述的光人工神经网络智能芯片，其特征在于，所述人工神经网络的类型包括：前馈神经网络。
  19. 根据权利要求1所述的光人工神经网络智能芯片，其特征在于，所述光滤波器层与所述图像传感器之前设置有透光介质层。
  20. 根据权利要求1所述的光人工神经网络智能芯片，其特征在于，所述图像传感器为前照式，包括：自上而下设置的金属线层和光探测层，所述光滤波器层集成在所述金属线层远离所述光探测层的一面；或，
    所述图像传感器为背照式，包括：自上而下设置的光探测层和金属线层，所述光滤波器层集成在所述光探测层远离所述金属线层的一面。
  21. 一种智能处理设备，其特征在于，包括：如权利要求1～19任一项所述的光人工神经网络智能芯片。
  22. 根据权利要求21所述的智能处理设备，其特征在于，所述智能处理设备包括智能手机、智能电脑、智能识别设备、智能感知设备和智能决策设备中的一种或多种。
  23. 一种如权利要求1～20任一项所述的光人工神经网络智能芯片的 制备方法，其特征在于，包括：
    在所述图像传感器的感光区域的表面制备包含有光调制结构的光滤波器层；
    生成具备对信号进行全连接处理与非线性激活处理功能的处理器；
    连接所述图像传感器和所述处理器；
    其中，所述光滤波器层用于通过所述光调制结构对进入至所述光调制结构不同位置点处的入射光分别进行不同的频谱调制，以在所述感光区域的表面得到与不同位置点对应的入射光携带信息；所述入射光携带信息包括光强度分布信息、光谱信息、所述入射光的角度信息以及所述入射光的相位信息；
    所述图像传感器用于将与不同位置点经光滤波器层调制后对应的入射光携带信息转换为与不同位置点对应的电信号，并将与不同位置点对应的电信号发送给所述处理器；所述处理器用于将与不同位置点对应的电信号进行全连接处理与非线性激活处理，得到人工神经网络的输出信号；所述电信号为经光滤波器层调制后的图像信号。
  24. 根据权利要求23所述的光人工神经网络智能芯片的制备方法，其特征在于，在所述图像传感器的感光区域的表面制备包含有光调制结构的光滤波器层，包括：
    在所述图像传感器的感光区域的表面生长一层或多层预设材料；
    对所述一层或多层预设材料进行光调制结构图案的刻蚀，得到包含有光调制结构的光滤波器层；
    或对所述一层或多层预设材料进行压印转移，得到包含有光调制结构的光滤波器层；
    或通过对所述一层或多层预设材料进行外加动态调制，得到包含有光调制结构的光滤波器层；
    或对所述一层或多层预设材料进行分区打印，得到包含有光调制结构的光滤波器层；
    或对所述一层或多层预设材料进行分区生长，得到包含有光调制结构的光滤波器层；
    或对所述一层或多层预设材料进行量子点转移，得到包含有光调制结 构的光滤波器层。
  25. 根据权利要求23所述的光人工神经网络智能芯片的制备方法，其特征在于，当所述光人工神经网络智能芯片用于目标对象的智能处理任务时，利用与所述智能处理任务对应的输入训练样本和输出训练样本，对包含不同的光调制结构、图像传感器和具有不同的全连接参数和非线性激活参数的处理器的光人工神经网络智能芯片进行训练，得到满足训练收敛条件的光调制结构、图像传感器和处理器。
PCT/CN2021/115965 2021-02-08 2021-09-01 光人工神经网络智能芯片、智能处理设备及制备方法 WO2022166188A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110172841.0 2021-02-08
CN202110172841.0A CN114912599A (zh) 2021-02-08 2021-02-08 光人工神经网络智能芯片、智能处理设备及制备方法

Publications (1)

Publication Number Publication Date
WO2022166188A1 true WO2022166188A1 (zh) 2022-08-11

Family

ID=82704630

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/115965 WO2022166188A1 (zh) 2021-02-08 2021-09-01 光人工神经网络智能芯片、智能处理设备及制备方法

Country Status (3)

Country Link
US (1) US20220253686A1 (zh)
CN (1) CN114912599A (zh)
WO (1) WO2022166188A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117764020A (zh) * 2024-02-22 2024-03-26 沐曦集成电路(上海)有限公司 芯片设计调整方法、电子设备和介质

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116224475B (zh) * 2023-03-10 2024-03-08 天津大学 光转换器及光转换系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110133782A (zh) * 2019-05-31 2019-08-16 苏州麦田光电技术有限公司 一种可见光波段光学神经网络元件
WO2019200289A1 (en) * 2018-04-13 2019-10-17 The Regents Of The University Of California Devices and methods employing optical-based machine learning using diffractive deep neural networks
CN110866601A (zh) * 2019-10-16 2020-03-06 复旦大学 一种基于光电神经网络的复合采集处理系统
CN111458777A (zh) * 2020-04-22 2020-07-28 中国计量大学 一种光学芯片及制作方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019200289A1 (en) * 2018-04-13 2019-10-17 The Regents Of The University Of California Devices and methods employing optical-based machine learning using diffractive deep neural networks
CN110133782A (zh) * 2019-05-31 2019-08-16 苏州麦田光电技术有限公司 一种可见光波段光学神经网络元件
CN110866601A (zh) * 2019-10-16 2020-03-06 复旦大学 一种基于光电神经网络的复合采集处理系统
CN111458777A (zh) * 2020-04-22 2020-07-28 中国计量大学 一种光学芯片及制作方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117764020A (zh) * 2024-02-22 2024-03-26 沐曦集成电路(上海)有限公司 芯片设计调整方法、电子设备和介质
CN117764020B (zh) * 2024-02-22 2024-04-26 沐曦集成电路(上海)有限公司 芯片设计调整方法、电子设备和介质

Also Published As

Publication number Publication date
CN114912599A (zh) 2022-08-16
US20220253686A1 (en) 2022-08-11

Similar Documents

Publication Publication Date Title
WO2022166188A1 (zh) 光人工神经网络智能芯片、智能处理设备及制备方法
WO2022166189A1 (zh) 光人工神经网络智能芯片及制备方法
Chen et al. Artificial intelligence in meta-optics
Weng et al. Deep learning networks for the recognition and quantitation of surface-enhanced Raman spectroscopy
CN103155544B (zh) 基于角度敏感像素（asp）的图像处理系统、处理方法和应用
CN111490060A (zh) 光谱成像芯片及光谱识别设备
TWI790639B (zh) 一種圖像採集晶片、物體成像識別設備及物體成像識別方法
CN101784227A (zh) 激光散斑成像系统和方法
US11620849B2 (en) Spectral imaging chip and apparatus, information processing method, fingerprint living body identification device and fingerprint module
US6422998B1 (en) Fractal analysis of time varying data
CN110680282A (zh) 一种大脑暂时性异常态的检测方法、装置和系统
Sarwar et al. Miniaturizing a chip-scale spectrometer using local strain engineering and total-variation regularized reconstruction
CN211828773U (zh) 光谱成像芯片及光谱识别设备
Zheng et al. Processing IMU action recognition based on brain-inspired computing with microfabricated MEMS resonators
Li Optimal Modeling of College Students' Mental Health Based on Brain-Computer Interface and Imaging Sensing
Lensu et al. Color-sensitive biosensors for imaging applications
CN114913111A (zh) 血糖检测芯片、智能血糖检测仪及制备方法
CN210112118U (zh) 一种图像采集芯片及物体成像识别设备
CN114913944A (zh) 光人工神经网络血糖检测芯片及制备方法
Lyu et al. Tactile recognition technology based on Multi-channel fiber optical sensing system
CN114913553A (zh) 光人工神经网络指纹识别芯片、指纹识别装置及制备方法
CN115128791A (zh) 光谱成像天文望远镜及天文望远镜光谱成像方法
CN114912597A (zh) 农业控制芯片、智能农业控制设备及制备方法
CN114912600A (zh) 光人工神经网络智能农业精准控制芯片及制备方法
Tamee et al. Muscle sensor model using small scale optical device for pattern recognitions

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21924187

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21924187

Country of ref document: EP

Kind code of ref document: A1