CN114912602A - Optical artificial neural network smelting end point monitoring chip and preparation method thereof - Google Patents

Optical artificial neural network smelting end point monitoring chip and preparation method thereof Download PDF

Info

Publication number
CN114912602A
CN114912602A CN202110172852.9A CN202110172852A CN114912602A CN 114912602 A CN114912602 A CN 114912602A CN 202110172852 A CN202110172852 A CN 202110172852A CN 114912602 A CN114912602 A CN 114912602A
Authority
CN
China
Prior art keywords
optical
neural network
artificial neural
smelting
different
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110172852.9A
Other languages
Chinese (zh)
Inventor
崔开宇
刘仿
黄翊东
张巍
冯雪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tsinghua University
Original Assignee
Tsinghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tsinghua University filed Critical Tsinghua University
Priority to CN202110172852.9A priority Critical patent/CN114912602A/en
Publication of CN114912602A publication Critical patent/CN114912602A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/067—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using optical means
    • G06N3/0675—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using optical means using electro-optical, acousto-optical or opto-electronic means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/10—Interfaces, programming languages or software development kits, e.g. for simulating neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P10/00—Technologies related to metal processing
    • Y02P10/20—Recycling

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Neurology (AREA)
  • Medicines Containing Antibodies Or Antigens For Use As Internal Diagnostic Agents (AREA)
  • Investigating Or Analyzing Materials Using Thermal Means (AREA)

Abstract

The invention provides an optical artificial neural network smelting end point monitoring chip and a preparation method thereof, which are used for a smelting end point monitoring task.

Description

Optical artificial neural network smelting end point monitoring chip and preparation method thereof
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to an optical artificial neural network smelting end point monitoring chip and a preparation method thereof.
Background
Converter steelmaking is the most widely and efficiently applied steelmaking method in the world at present, smelting end point control is one of key technologies in converter production, and accurate judgment of an end point has important significance in improving molten steel quality and shortening smelting period. But the accurate control of the smelting end point is still difficult due to the reasons of unstable raw materials entering the furnace, complex chemical reaction, strict steel grade and the like. Accurate online detection of the end point carbon content and the molten steel temperature is an urgent problem to be solved in the worldwide metallurgical industry. The existing industrial end point control method for the large-scale instrument and equipment mainly depends on manual experience or complex to measure the temperature of a furnace mouth and qualitatively measure slag residues, and is low in precision and high in cost.
Disclosure of Invention
Aiming at the problems in the prior art, the embodiment of the invention provides an optical artificial neural network smelting end point monitoring chip and a preparation method thereof.
Specifically, the embodiment of the invention provides the following technical scheme:
in a first aspect, an embodiment of the present invention provides an optical artificial neural network smelting endpoint monitoring chip, which is used for a smelting endpoint monitoring task, and includes: an optical filter layer, an image sensor, and a processor; the optical filter layer corresponds to an input layer and a linear layer of the artificial neural network and the connection weight of the input layer to the linear layer, and the square detection response of the image sensor corresponds to a first nonlinear activation function in the nonlinear layer of the artificial neural network; the processor corresponds to the full connection and output layer of the artificial neural network, or the processor corresponds to the second nonlinear activation function and output layer in the full connection and nonlinear layer of the artificial neural network;
the optical filter layer is arranged on the surface of the image sensor and comprises an optical modulation structure, and the optical modulation structure is used for respectively carrying out different spectrum modulation on incident light entering different position points of the optical modulation structure so as to obtain incident light carrying information corresponding to different position points on the surface of the image sensor; the incident light comprises reflected light, transmitted light and/or radiated light of a steelmaking furnace mouth;
the image sensor carries out first nonlinear activation processing on incident light carrying information corresponding to different position points after being modulated by the optical filter layer through square detection response, converts the incident light carrying information into electric signals corresponding to the different position points, and sends the electric signals corresponding to the different position points to the processor;
the processor performs full connection processing on the electric signals corresponding to different position points, or performs full connection processing and secondary nonlinear activation processing on the electric signals corresponding to different position points to obtain a smelting end point monitoring result;
the smelting end point monitoring task comprises the step of identifying a smelting end point, and the smelting end point monitoring result comprises a smelting end point identification result.
Further, the incident light carrying information includes at least one of light intensity distribution information, spectrum information, angle information of the incident light, and phase information of the incident light.
Further, the smelting end point monitoring task also comprises the step of identifying the carbon content and/or the molten steel temperature in the smelting process, and the smelting end point monitoring result comprises the carbon content and/or the molten steel temperature identification result in the smelting process.
Further, the optical artificial neural network smelting end point monitoring chip comprises a trained optical modulation structure, an image sensor and a processor;
the trained optical modulation structure, the image sensor and the processor are the optical modulation structure, the image sensor and the processor which meet the training convergence condition and are obtained by training an optical artificial neural network smelting endpoint monitoring chip comprising different optical modulation structures, image sensors and processors with different full connection parameters by using an input training sample and an output training sample corresponding to the smelting endpoint monitoring task; or the trained optical modulation structure, image sensor and processor are optical modulation structure, image sensor and processor which meet the training convergence condition and are obtained by training an optical artificial neural network smelting endpoint monitoring chip which comprises different optical modulation structures, image sensors and processors with different full connection parameters and different second nonlinear activation parameters by using input training samples and output training samples corresponding to the smelting endpoint monitoring task;
wherein the input training samples comprise incident light reflected, transmitted, and/or radiated from smelt-to-end and unmelted-to-end steelmaking tuyeres; and the output training sample comprises a judgment result of whether smelting is completed to the end point.
Further, when the smelting endpoint monitoring task further comprises the identification of carbon content and/or molten steel temperature in the smelting process, correspondingly, the input training samples further comprise incident light reflected, transmitted and/or radiated by a steelmaking converter mouth smelted to different carbon content and/or molten steel temperature, and the output training samples further comprise corresponding carbon content and/or molten steel temperature.
Further, when training an optical artificial neural network smelting end point monitoring chip comprising different optical modulation structures, image sensors and processors with different full-connection parameters, or training an optical artificial neural network smelting end point monitoring chip comprising different optical modulation structures, image sensors and processors with different full-connection parameters and different second nonlinear activation parameters, the different optical modulation structures are designed and realized by adopting a computer optical simulation design mode.
Further, the light modulating structures in the optical filter layer comprise regular structures and/or irregular structures; and/or, the light modulating structures in the optical filter layer comprise discrete type structures and/or continuous type structures.
Furthermore, the light modulation structure in the optical filter layer comprises a unit array consisting of a plurality of micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of the micro-nano units are the same or different.
Further, the micro-nano unit comprises a regular structure and/or an irregular structure; and/or the micro-nano unit comprises a discrete structure and/or a continuous structure.
Further, the micro-nano unit comprises a plurality of groups of micro-nano structure arrays, and the structures of the micro-nano structure arrays are the same or different.
Furthermore, each group of micro-nano structure array has the function of broadband filtering or narrow-band filtering.
Furthermore, each group of micro-nano structure array is a periodic structure array or a non-periodic structure array.
Furthermore, one or more groups of hollow structures are arranged in a plurality of groups of micro-nano structure arrays contained in the micro-nano unit.
Further, the micro-nano unit has quadruple rotational symmetry.
Further, the optical filter layer is composed of one or more layers of structures;
each layer of structure is prepared by one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials and perovskite materials; and/or the filter layer is prepared from one or more of a photonic crystal, a super surface, a random structure, a nano structure, a metal Surface Plasmon Polariton (SPP) micro-nano structure and an adjustable Fabry-Perot resonant cavity.
Further, the semiconductor material comprises one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed according to a preset proportion and direct band gap compound semiconductor materials; and/or the nanostructure comprises one or more of a nanodot two-dimensional material, a nanorod two-dimensional material, and a nanowire two-dimensional material.
Further, the optical filter layer has a thickness of 0.1 λ to 10 λ, where λ represents a center wavelength of incident light.
In a second aspect, the embodiment of the invention provides an intelligent smelting control device, which comprises the optical artificial neural network smelting endpoint monitoring chip.
In a third aspect, an embodiment of the present invention provides a method for preparing the above optical artificial neural network smelting endpoint monitoring chip, including:
preparing an optical filter layer containing an optical modulation structure on the surface of the image sensor;
generating a processor with a function of performing full connection processing on the signal or generating a processor with a function of performing full connection processing and secondary nonlinear activation processing on the signal;
connecting the image sensor and the processor;
the optical filter layer is used for respectively carrying out different spectrum modulation on incident light entering different position points of the optical modulation structure through the optical modulation structure so as to obtain light field distribution signals corresponding to the different position points on the surface of the image sensor;
the image sensor carries out first nonlinear activation processing on the light field distribution signals corresponding to the different position points after being modulated by the optical filter layer through square detection response, then converts the light field distribution signals into electric signals corresponding to the different position points, and sends the electric signals corresponding to the different position points to the processor;
and the processor performs full connection processing on the electric signals corresponding to the different position points, or performs full connection processing and secondary nonlinear activation processing on the electric signals corresponding to the different position points to obtain a smelting end point monitoring result.
Further, still include: the training process of the optical artificial neural network smelting end point monitoring chip specifically comprises the following steps:
training an optical artificial neural network smelting endpoint monitoring chip comprising different optical modulation structures, image sensors and processors with different full-connection parameters by using input training samples and output training samples corresponding to the smelting endpoint monitoring task to obtain the optical modulation structures, the image sensors and the processors meeting the training convergence conditions, and taking the optical modulation structures, the image sensors and the processors meeting the training convergence conditions as the trained optical modulation structures, the image sensors and the processors;
or training an optical artificial neural network smelting endpoint monitoring chip comprising different optical modulation structures, image sensors and processors with different full-connection parameters and different second nonlinear activation parameters by using input training samples and output training samples corresponding to the smelting endpoint monitoring task to obtain the optical modulation structures, the image sensors and the processors meeting the training convergence conditions, and taking the optical modulation structures, the image sensors and the processors meeting the training convergence conditions as the trained optical modulation structures, the trained image sensors and the trained processors.
Further, preparing an optical filter layer containing a light modulation structure on the surface of the image sensor includes:
growing one or more layers of preset materials on the surface of the image sensor;
etching the light modulation structure pattern on the one or more layers of preset materials to obtain an optical filter layer containing a light modulation structure;
or carrying out impression transfer on the one or more layers of preset materials to obtain an optical filter layer containing an optical modulation structure;
or the one or more layers of preset materials are subjected to additional dynamic modulation to obtain an optical filter layer containing an optical modulation structure;
or printing the one or more layers of preset materials in a partition mode to obtain an optical filter layer containing an optical modulation structure;
or carrying out partition growth on the one or more layers of preset materials to obtain an optical filter layer containing an optical modulation structure;
or quantum dot transfer is carried out on the one or more layers of preset materials to obtain the optical filter layer containing the optical modulation structure.
The embodiment of the invention realizes a novel optical artificial neural network smelting end point monitoring chip capable of realizing the function of an artificial neural network, which is used for a smelting end point monitoring task, wherein in the optical artificial neural network smelting end point monitoring chip, an optical filter layer corresponds to an input layer and a linear layer of the artificial neural network, and an image sensor corresponds to a part of a nonlinear layer of the artificial neural network; the processor corresponds to another portion of the non-linear layer of the artificial neural network and the output layer. Specifically, the optical filter layer is arranged on the surface of the image sensor and comprises optical modulation structures, and the optical modulation structures are used for respectively carrying out different spectrum modulation on incident light entering different position points of the optical modulation structures so as to obtain information carried by the incident light corresponding to the different position points on the surface of the image sensor. Meanwhile, in the embodiment of the invention, the image sensor carries out the first nonlinear activation processing on the incident light carrying information corresponding to different position points after being modulated by the optical filter layer through square detection response and then converts the incident light carrying information into electric signals corresponding to different position points, and sends the electric signals corresponding to different position points to the processor, the processor carries out full connection processing on the electric signals corresponding to different position points, or the processor carries out full connection processing and secondary nonlinear activation processing on the electric signals corresponding to different position points to obtain the output signal of the artificial neural network, so that, in the chip for monitoring the smelting end point of the optical artificial neural network, the optical filter layer corresponds to an input layer and a linear layer of the artificial neural network and the connection weight from the input layer to the linear layer, the square detection response of the image sensor corresponds to a first-time nonlinear activation function in a nonlinear layer of the artificial neural network; the processor corresponds to the full connection and output layer of the artificial neural network, or the processor corresponds to the full connection of the artificial neural network, the second nonlinear activation function and output layer in the nonlinear layer, that is, the optical filter layer and the image sensor in the optical artificial neural network smelting endpoint monitoring chip realize the related functions of the input layer, the linear layer and part of the nonlinear activation function in the artificial neural network, that is, the embodiment of the invention strips the input layer, the linear layer and part or all of the nonlinear activation function in the artificial neural network realized by software in the prior art, and realizes the structures of the input layer, the linear layer and part or all of the nonlinear activation function in the artificial neural network by using a hardware mode, so that the optical artificial neural network smelting endpoint monitoring chip does not need to carry out the following processes of the artificial neural network intelligent processing with the input layer, The linear layer and a part or all of the complex signal processing and algorithm processing corresponding to the nonlinear activation function are only required to be carried out by the processor in the optical artificial neural network smelting end point monitoring chip, and the processor is in full connection with the electric signal or full connection with the electric signal and carries out the second nonlinear activation processing, so that the power consumption and the time delay during the artificial neural network processing can be greatly reduced.
Therefore, the embodiment of the invention provides the novel photoelectric chip for quickly and accurately identifying the smelting end point, and the chip partially embeds the artificial neural network into hardware equipment to realize safe, reliable, quick and accurate smelting end point control.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic structural diagram of an optical artificial neural network smelting endpoint monitoring chip according to a first embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating the recognition principle of an optical artificial neural network smelting endpoint monitoring chip according to an embodiment of the present invention;
FIG. 3 is a schematic view of a disassembled optical artificial neural network smelting endpoint monitoring chip according to an embodiment of the present invention;
FIG. 4 is a schematic view of a furnace mouth identification for determining a smelting endpoint in a smelting process according to an embodiment of the present invention;
FIG. 5 is a top view of an optical filter layer according to an embodiment of the present invention;
FIG. 6 is a top view of another optical filter layer provided in accordance with an embodiment of the present invention;
FIG. 7 is a top view of yet another optical filter layer provided in accordance with an embodiment of the present invention;
FIG. 8 is a top view of another optical filter layer according to an embodiment of the present invention;
FIG. 9 is a top view of yet another optical filter layer provided in accordance with an embodiment of the present invention;
FIG. 10 is a top view of yet another optical filter layer provided in accordance with an embodiment of the present invention;
fig. 11 is a schematic diagram of a micro-nano structure broadband filtering effect according to an embodiment of the present invention;
fig. 12 is a schematic diagram of a narrow-band filtering effect of a micro-nano structure according to an embodiment of the present invention;
FIG. 13 is a schematic diagram of a front-illuminated image sensor according to an embodiment of the present invention;
FIG. 14 is a schematic diagram of a back-illuminated image sensor according to an embodiment of the present invention;
fig. 15 is a schematic flowchart of a method for manufacturing an optical artificial neural network smelting endpoint monitoring chip according to a third embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Converter steelmaking is the most widely and efficiently applied steelmaking method in the world at present, smelting end point control is one of key technologies in converter production, and accurate judgment of an end point has important significance in improving molten steel quality and shortening smelting period. But the accurate control of the smelting end point is still difficult due to the reasons of unstable raw materials entering the furnace, complex chemical reaction, strict steel grade and the like. Accurate online detection of the end point carbon content and the molten steel temperature is an urgent problem to be solved in the metallurgical industry all over the world. The existing industrial terminal control method for the furnace mouth temperature measurement and the qualitative measurement of slag residues mainly depends on manual experience or complex large-scale instruments and equipment, and is low in precision and high in cost. Therefore, the realization of the online smelting terminal control chip which has micro volume, low cost, safety, reliability and easy integration of a control system at the later stage has important significance on the development of the steel smelting industry and the automation of the industry, and is really 4.0 of the industry. Specifically, the embodiment of the invention provides an optical artificial neural network smelting endpoint monitoring chip, which is used for a smelting endpoint monitoring task, namely the chip can accurately and contactlessly identify a smelting endpoint, and the chip has the following structure: the optical filter layer in the chip corresponds to an input layer and a linear layer of the artificial neural network and the connection weight of the input layer to the linear layer, and the square detection response of the image sensor in the chip corresponds to a first-time nonlinear activation function in the nonlinear layer of the artificial neural network; the processor in the chip corresponds to the full connection and output layer of the artificial neural network, or the processor corresponds to the full connection of the artificial neural network, the second nonlinear activation function in the nonlinear layer and the output layer. According to the embodiment of the invention, complex signal processing and algorithm processing corresponding to an input layer, a linear layer and a part or all of nonlinear activation functions in the prior art can be omitted, so that the time delay can be greatly reduced. In addition, the embodiment of the invention can also utilize one or more information of image information, spectral information, incident light angle and incident light phase information of the steelmaking furnace mouth, namely incident light carrying information at different points of the steelmaking furnace mouth space, so that the embodiment of the invention can be seen in that the incident light carrying information at different points of the steelmaking furnace mouth space covers information of the steelmaking furnace mouth, such as image, composition, shape, three-dimensional depth, structure and the like, so that when identification processing is carried out according to the incident light carrying information at different points of the steelmaking furnace mouth space, the embodiment of the invention can cover multi-dimensional information of the steelmaking furnace mouth, such as image, composition, shape, three-dimensional depth, structure and the like, thereby improving the accuracy of smelting end point identification, and therefore, the invention provides the optical artificial neural network smelting end point monitoring chip which well solves the worldwide problem that the smelting end point is difficult to monitor, meanwhile, due to the adoption of the optical artificial neural network chip, the complex signal processing and algorithm processing corresponding to the relevant input layer, linear layer and partial or all nonlinear activation functions in the optical artificial neural network can be replaced by pre-prepared hardware (an optical filter layer and an image sensor), so that the processing official function and the time delay are greatly reduced, and the rapid, accurate, safe and reliable smelting end point detection can be realized. The invention will now be explained and illustrated in detail by means of specific examples.
As shown in fig. 1, the optical artificial neural network smelting endpoint monitoring chip provided in the first embodiment of the present invention is used for a smelting endpoint monitoring task, and includes: an optical filter layer 1, an image sensor 2 and a processor 3; the optical filter layer 1 corresponds to an input layer and a linear layer of the artificial neural network and the connection weight of the input layer to the linear layer, and the square detection response of the image sensor 2 corresponds to a first nonlinear activation function in the nonlinear layer of the artificial neural network; the processor 3 corresponds to the full connection and output layer of the artificial neural network, or the processor corresponds to the second nonlinear activation function and output layer in the full connection and nonlinear layer of the artificial neural network;
the optical filter layer 1 is arranged on the surface of the image sensor or the surface of a photosensitive area of the image sensor, the optical filter layer 1 comprises an optical modulation structure, and the optical filter layer 1 is used for respectively performing spectrum modulation with intensity modulation along with wavelength variation on incident light entering different positions of the optical modulation structure through the optical modulation structure, namely performing different intensity modulation on the incident light with different wavelengths so as to obtain incident light carrying information corresponding to different positions on the surface of the image sensor;
in this embodiment, the square detection response of the image sensor 2 means that the image sensor detects intensity information of an incident light field, and the intensity information of the incident light field is a square of a light field signal modulus, that is, the image sensor 2 performs first nonlinear activation processing on incident light carrying information corresponding to different position points after being modulated by the optical filter layer 1 through the square detection response, converts the incident light carrying information into electric signals corresponding to the different position points, and sends the electric signals corresponding to the different position points to the processor 3; the electric signal is an image signal modulated by the optical filter layer; wherein the incident light comprises reflected light, transmitted light and/or radiated light of a steelmaking furnace mouth;
the processor 3 is used for carrying out full connection processing on the electric signals corresponding to the different position points, or carrying out full connection processing and secondary nonlinear activation processing on the electric signals corresponding to the different position points by the processor to obtain an output signal of the artificial neural network, so as to obtain a smelting end point monitoring result.
In this embodiment, the incident light carrying information includes image information and/or various optical spatial information of a target object to be processed by the optical artificial neural network intelligent chip, for example, the incident light carrying information includes at least one of light intensity distribution information, spectrum information, angle information of the incident light, and phase information of the incident light.
In this embodiment, the optical filter layer 1 is disposed on the surface of the image sensor, the optical filter layer 1 includes an optical modulation structure, and the optical filter layer 1 is configured to perform different spectrum modulations on incident light entering different position points of the optical modulation structure through the optical modulation structure, so as to obtain modulated light field distribution signals corresponding to different position points on the surface of the image sensor. It follows that in an embodiment, the modulating effect of the light modulating structures on the optical filter layer on the incident light can be seen as the coupling weights of the input layer to said linear layer;
in this embodiment, when the image sensor 2 performs photoelectric conversion on the modulated incident light carrying information, since the image sensor 2 can only detect the intensity information of light, the electrical signal obtained by processing the light field distribution signal is proportional to the square of the mode of the light field distribution signal, and therefore, the image sensor 2 has a square detection response, so that the image sensor 2 can be regarded as a part of the nonlinear layer of the artificial neural network, that is, the square detection response of the image sensor 2 can be regarded as the first nonlinear activation function of the artificial neural network.
In this embodiment, the image sensor 2 performs a first nonlinear activation process on the incident light carrying information corresponding to different position points modulated by the optical filter layer 1 through a square detection response, and then converts the incident light carrying information into electrical signals corresponding to the different position points, that is, image signals modulated by the optical filter layer, and meanwhile, the processor 3 connected to the image sensor 2 is configured to perform a full connection process or a full connection and a second nonlinear activation process on the electrical signals corresponding to the different position points, so as to obtain an output signal of the artificial neural network.
In this embodiment, the optical filter layer 1 includes an optical modulation structure, and performs spectrum modulation with different intensities on incident light (for example, related action light such as reflected light, transmitted light, and radiated light of an object to be identified) entering different position points of the optical modulation structure through the optical modulation structure, so as to obtain incident light carrying information corresponding to the different position points on the surface of the image sensor 2.
In this embodiment, it is understood that the modulation intensity is related to the specific structural form of the light modulation structure, for example, different modulation intensities can be realized by designing different light modulation structures (for example, changing the shape and/or size parameters of the light modulation structure).
In this embodiment, it can be understood that the light modulation structures at different positions on the optical filter layer 1 have different spectrum modulation effects on the incident light, and the modulation intensities of the light modulation structures on different wavelength components of the incident light correspond to the connection intensities of the linear layers of the artificial neural network, that is, correspond to the input layers and the connection weights of the input layers to the linear layers. It should be noted that the optical filter layer 1 is composed of a plurality of optical filter units, and the optical modulation structures at different positions in each optical filter unit are different, so that the optical filter layer has different spectrum modulation effects on incident light; the light modulating structures at different locations between the optical filter cells may be the same or different and thus have the same or different spectral modulation effect on the incident light.
In this embodiment, the image sensor 2 performs a first nonlinear activation process on the incident light carrying information corresponding to different position points modulated by the optical filter layer 1 through a square detection response, and then converts the incident light carrying information into electrical signals corresponding to the different position points, and sends the electrical signals corresponding to the different position points to the processor 3, where the image sensor 2 corresponds to a part of a nonlinear layer of a neural network.
In this embodiment, the processor 3 performs full-connection processing on the electrical signals at different position points, or the processor 3 performs full-connection processing and second nonlinear activation processing on the electrical signals at different position points, so as to obtain an output signal of the artificial neural network.
It is understood that, in the present embodiment, the image sensor 2 corresponds to a part of the nonlinear layer of the neural network, and the processor 3 corresponds to another part of the nonlinear layer of the neural network and the output layer, and also may be understood to correspond to the remaining layers (all other layers) of the neural network except the first nonlinear activation function in the input layer, the linear layer and the nonlinear layer.
In this embodiment, the square detection response of the image sensor 2 corresponds to the first nonlinear activation function in the nonlinear layer of the neural network, and in this case, only the full-connection processing may be performed in the processor, and the second nonlinear activation processing may not be performed, or both the full-connection processing and the second nonlinear activation processing may be performed in the processor. The method may be determined according to an actual application scenario of the chip, which is not limited in this embodiment.
In addition, it should be added that the processor 3 may be disposed in the optical artificial neural network smelting endpoint monitoring chip, that is, the processor 3 may be disposed in the optical artificial neural network smelting endpoint monitoring chip together with the filter layer 1 and the image sensor 2, or may be separately disposed outside the optical artificial neural network smelting endpoint monitoring chip and connected to the image sensor 2 in the optical artificial neural network smelting endpoint monitoring chip through a data line or a connection device, which is not limited in this embodiment.
In addition, it should be noted that the processor 3 may be implemented by a computer, may also be implemented by an ARM or FPGA circuit board having a certain operation capability, and may also be implemented by a microprocessor, which is not limited in this embodiment. In addition, as mentioned above, the processor 3 may be integrated into the optical artificial neural network smelting endpoint monitoring chip, or may be disposed outside the optical artificial neural network smelting endpoint monitoring chip independently. When the processor 3 is arranged outside the optical artificial neural network smelting endpoint monitoring chip independently, the electrical signals in the image sensor 2 can be read into the processor 3 through the signal reading circuit, and then the processor 3 performs full connection processing and nonlinear activation processing on the read electrical signals.
In this embodiment, it is understood that, when performing the second nonlinear activation processing, the processor 3 may be implemented by using a nonlinear activation function, for example, a Sigmoid function, a Tanh function, a ReLU function, and the like, which is not limited in this embodiment.
In this embodiment, the optical filter layer 1 corresponds to an input layer, a linear layer, and a connection weight from the input layer to the linear layer of the artificial neural network, the image sensor 2 corresponds to a part of a nonlinear layer of the artificial neural network, that is, a square detection response of the image sensor 2 corresponds to a first nonlinear activation function of the artificial neural network, the image sensor 2 is configured to perform nonlinear activation processing on incident light carrying information at different spatial location points through the square detection response and further convert the incident light carrying information into an electrical signal, and the processor 3 corresponds to the remaining layers of the artificial neural network and fully connects the electrical signals at different spatial locations, or further obtains an output signal of the artificial neural network through a second nonlinear activation function.
As shown in the left side of fig. 2, the optical artificial neural network smelting endpoint monitoring chip includes an optical filter layer 1, an image sensor 2 and a processor 3, and in fig. 2, the processor 3 is implemented by using a signal readout circuit and a computer. As shown in the right side of fig. 2, an optical filter layer 1 in the chip for monitoring smelting endpoint of an optical artificial neural network corresponds to an input layer and a linear layer of the artificial neural network, an image sensor 2 corresponds to a part of a nonlinear layer of the artificial neural network, a processor 3 corresponds to another part of the nonlinear layer and an output layer of the artificial neural network, the filtering effect of the optical filter layer 1 on incident light entering the optical filter layer 1 corresponds to the connection weight from the input layer to the linear layer, and the square detection response of the image sensor 2 corresponds to a first nonlinear activation function of the artificial neural network, so that it can be seen that the optical filter layer and the image sensor in the chip for monitoring smelting endpoint of an optical artificial neural network provided by the embodiment realize related functions of the input layer, the linear layer and a part or all of the nonlinear activation function in the artificial neural network in a hardware manner, thereby enabling the subsequent intelligent processing using the chip for monitoring smelting endpoint of an optical artificial neural network to be unnecessary By performing complicated signal processing and arithmetic processing corresponding to the input layer and the linear layer (for example, by omitting calculations such as the weight of connection from the input layer to the linear layer), power consumption and delay in the artificial neural network processing can be significantly reduced. In addition, because the embodiment simultaneously utilizes the image information, the spectrum information, the angle information of the incident light and the phase information of the incident light at different points of the steelmaking furnace mouth space, the intelligent processing of the steelmaking furnace mouth can be more accurately realized.
As shown in the right side of FIG. 2, the optical filter layer 1 has different broadband spectrum modulation effects on the incident light, and will correspond to the incident light spectrum P at the cell location λ Projecting/connecting to the emergent light field E N C, removing; the square detection response of the image sensor 2 corresponds to a part of the nonlinear activation of the optical artificial neural networkFunction of the emergent light field E of the optical filter layer 1 N Conversion to photocurrent response of image sensor I N The above. The processor 3 comprises a signal readout circuit and a computer, the signal readout circuit in the processor 3 reads out the photocurrent response I N The signal is transmitted to a computer, the computer carries out full connection processing of the electric signal or carries out nonlinear activation processing again, and finally, the result is output.
As shown in fig. 3, the optical modulation structure on the optical filter layer 1 is integrated on the surface of the image sensor 2, modulates the incident light, projects/connects the spectrum information of the incident light onto different pixels of the image sensor 2 to obtain an electrical signal containing the spectrum information and the image information of the incident light, that is, after the incident light passes through the optical filter layer 1, the square detection response of the image sensor 2 is nonlinearly activated and then converted into an electrical signal to form an image containing the spectrum information of the incident light, and finally, the electrical signal containing the image information, the spectrum information, the angle information of the incident light and the phase information of the incident light at different points in space is processed by the processor 3 connected to the image sensor 2 to obtain an output result.
In this embodiment, the incident light carrying information may include one or more (including two) of light intensity distribution information, spectrum information, angle information of the incident light, and phase information of the incident light.
For example, in one implementation, the incident light carrying information may include light intensity distribution information, and in other implementations, multiple information of image information, spectrum information, angle of incident light, and phase information of incident light of the target object may be simultaneously utilized to identify the target object, so that intelligent identification of the target object may be more accurately implemented.
Therefore, the optical artificial neural network smelting endpoint monitoring chip provided by the embodiment of the invention can actually utilize one or more information of image information, spectrum information, incident light angle and incident light phase information of a steelmaking furnace mouth, namely, incident light carrying information at different spatial points, and the artificial neural network is embedded in hardware, so that the information of material components, image shapes, three-dimensional depths and the like can be further extracted from the spatial image, the spectrum, the angle and the phase information, thereby solving the problem that the smelting endpoint is difficult to accurately identify in the background art, and simultaneously, the embodiment of the invention can save complex signal processing and algorithm processing corresponding to an input layer, a linear layer and a part of nonlinear activation function in the prior art, thereby being capable of realizing low power consumption and low time delay, thus, the optical artificial neural network smelting endpoint monitoring chip provided by the embodiment of the invention, the effects of low power consumption, low time delay and high recognition rate can be simultaneously met, so that the smelting end point can be quickly and accurately recognized.
The invention provides an optical artificial neural network smelting end point monitoring chip and a preparation method thereof, which realize a novel optical artificial neural network smelting end point monitoring chip capable of realizing the function of an artificial neural network and are used for a smelting end point monitoring task, wherein in the optical artificial neural network smelting end point monitoring chip, an optical filter layer corresponds to an input layer and a linear layer of the artificial neural network, and an image sensor corresponds to a part of a nonlinear layer of the artificial neural network; the processor corresponds to another portion of the non-linear layer of the artificial neural network and the output layer. Specifically, the optical filter layer is arranged on the surface of the image sensor and comprises optical modulation structures, and the optical modulation structures are used for respectively carrying out different spectrum modulation on incident light entering different position points of the optical modulation structures so as to obtain information carried by the incident light corresponding to the different position points on the surface of the image sensor. Meanwhile, in the embodiment of the invention, the image sensor carries out the first nonlinear activation processing on the incident light carrying information corresponding to different position points after being modulated by the optical filter layer through square detection response and then converts the incident light carrying information into electric signals corresponding to different position points, and sends the electric signals corresponding to different position points to the processor, the processor carries out full connection processing on the electric signals corresponding to different position points, or the processor carries out full connection processing and secondary nonlinear activation processing on the electric signals corresponding to different position points to obtain the output signal of the artificial neural network, thus showing that, in the chip for monitoring the smelting endpoint of the optical artificial neural network, the optical filter layer corresponds to an input layer and a linear layer of the artificial neural network and the connection weight from the input layer to the linear layer, the square detection response of the image sensor corresponds to a first-order nonlinear activation function in a nonlinear layer of the artificial neural network; the processor corresponds to the full connection and output layer of the artificial neural network, or the processor corresponds to the full connection of the artificial neural network, the second nonlinear activation function and output layer in the nonlinear layer, that is, the optical filter layer and the image sensor in the optical artificial neural network smelting endpoint monitoring chip realize the related functions of the input layer, the linear layer and part of the nonlinear activation function in the artificial neural network, that is, the embodiment of the invention strips the input layer, the linear layer and part or all of the nonlinear activation function in the artificial neural network realized by software in the prior art, and realizes the structures of the input layer, the linear layer and part or all of the nonlinear activation function in the artificial neural network by using a hardware mode, so that the optical artificial neural network smelting endpoint monitoring chip does not need to carry out the following processes of the artificial neural network intelligent processing with the input layer, The linear layer and a part or all of the complex signal processing and algorithm processing corresponding to the nonlinear activation function are only required to be carried out by the processor in the optical artificial neural network smelting endpoint monitoring chip through full connection processing or full connection with an electric signal and secondary nonlinear activation processing, so that the power consumption and the time delay during artificial neural network processing can be greatly reduced. Therefore, the embodiment of the invention takes the optical filter layer as the input layer and the linear layer of the artificial neural network and the connection weight of the input layer to the linear layer, and takes the square detection response of the image sensor as the first nonlinear activation function in the nonlinear layer of the artificial neural network; the processor is used as a full connection and output layer of the artificial neural network, or the processor is used for corresponding to a second nonlinear activation function and an output layer in the full connection and nonlinear layer of the artificial neural network, so that the embodiment of the invention not only can save complex signal processing and algorithm processing corresponding to an input layer, a linear layer and a part of nonlinear activation function in the prior art, but also actually simultaneously utilizes image information, spectrum information, incident light angle and incident light phase information of the steelmaking furnace mouth, namely, the incident light carrying information at different points of the steelmaking furnace mouth space, so that the embodiment of the invention is visible, because the incident light carrying information at different points of the steelmaking furnace mouth space covers the information of the steelmaking furnace mouth such as image, composition, shape, three-dimensional depth, structure and the like, when the identification processing is carried out according to the incident light carrying information at different points of the steelmaking furnace mouth space, the optical artificial neural network smelting end point monitoring chip provided by the embodiment of the invention can simultaneously meet the effects of low power consumption, low time delay and high identification rate, thereby being capable of rapidly and accurately identifying the smelting end point.
Therefore, the embodiment of the invention provides the novel photoelectric chip for accurately identifying the smelting end point, and the chip partially embeds the artificial neural network into hardware equipment to realize safe, reliable, rapid and accurate smelting end point control.
Based on the content of the above embodiment, in this embodiment, the smelting endpoint monitoring task further includes identifying the carbon content and/or the molten steel temperature in the smelting process, and the smelting endpoint monitoring result includes identifying the carbon content and/or the molten steel temperature in the smelting process.
Based on the content of the above embodiment, in this embodiment, the optical artificial neural network smelting endpoint monitoring chip includes a trained optical modulation structure, an image sensor, and a processor;
the trained optical modulation structure, the image sensor and the processor are the optical modulation structure, the image sensor and the processor which meet the training convergence condition and are obtained by training an optical artificial neural network smelting endpoint monitoring chip comprising different optical modulation structures, image sensors and processors with different full-connection parameters by utilizing input training samples and output training samples corresponding to the smelting endpoint monitoring task; or the trained optical modulation structure, the image sensor and the processor are optical modulation structures, image sensors and processors which meet the training convergence condition and are obtained by training an optical artificial neural network smelting endpoint monitoring chip of the processor, which comprises different optical modulation structures, image sensors and different full-connection parameters and different second nonlinear activation parameters, by using input training samples and output training samples corresponding to the smelting endpoint monitoring task;
wherein the input training samples comprise incident light reflected, transmitted and/or radiated by a steelmaking taphole from smelt to finish and not smelt to finish; and the output training sample comprises a judgment result of whether smelting is completed to the end point.
Based on the content of the above embodiments, in this embodiment, when the smelting end-point monitoring task further includes identifying carbon content and/or molten steel temperature during smelting, accordingly, the input training sample further includes incident light reflected, transmitted and/or radiated by a steelmaking furnace mouth smelting to different carbon content and/or molten steel temperature, and the output training sample further includes corresponding carbon content and/or molten steel temperature.
Therefore, the detectable sample of the embodiment includes but is not limited to the end point control of converter steelmaking, and the artificial neural network training can be introduced to detect the temperature and the material elements of the smelting furnace mouth, so that the integration of a rear-end industrial control system is very easy, the recognition accuracy is high, and the qualitative analysis is more accurate.
In this embodiment, it is understood that the input training samples include incident light reflected, transmitted and/or radiated by the steelmaking tuyeres in the respective intelligent processing tasks; the output training sample comprises intelligent processing results of the steelmaking furnace mouth, such as a smelting end point recognition result, a temperature recognition result in the smelting process, a carbon content recognition result in the smelting process and the like.
In this embodiment, reflected light, transmitted light and/or radiated light of the steelmaking converter mouth enters the trained optical artificial neural network smelting endpoint monitoring chip to obtain intelligent processing results such as whether the smelting endpoint is achieved, the converter mouth temperature and the content of substance elements.
In this embodiment, a smelting endpoint recognition task is taken as an example for explanation, and it can be understood that when the optical artificial neural network smelting endpoint monitoring chip is used for recognizing a task, the optical artificial neural network smelting endpoint monitoring chip needs to be trained first, where the training of the optical artificial neural network smelting endpoint monitoring chip refers to determining an optical modulation structure suitable for a current recognition task through training, and a full connection parameter and a nonlinear activation parameter suitable for the current recognition task.
It can be understood that, because the filtering effect of the optical filter layer on the incident light entering the optical filter layer corresponds to the connection weight from the artificial neural network input layer to the linear layer, when training, changing the optical modulation structure in the optical filter layer is equivalent to changing the connection weight from the artificial neural network input layer to the linear layer, and through training convergence conditions, determining the optical modulation structure suitable for the current recognition task, and the full connection parameter and the nonlinear activation parameter suitable for the current recognition task, thereby completing the training of the optical artificial neural network smelting endpoint monitoring chip.
It can be understood that after the optical artificial neural network smelting endpoint monitoring chip is trained, the optical artificial neural network smelting endpoint monitoring chip can be used for executing a recognition task. Specifically, after incident light carrying steelmaking furnace mouth image information and spatial spectrum information enters an optical filter layer 1 of a trained optical artificial neural network smelting end point monitoring chip, an optical modulation structure in the optical filter layer 1 modulates the incident light, the intensity of the modulated optical signal is detected by an image sensor 2 and converted into an electric signal, and then a processor 3 performs full connection processing or performs full connection and secondary nonlinear activation processing simultaneously, so that whether the current identification result is a smelting end point can be obtained.
As shown in fig. 4, the complete process of identifying the steelmaking converter mouth to determine whether it is the smelting end point is as follows: the light emitted by the fire hole flame 100 of the steel converter 200 is collected by the optical artificial neural network smelting end point monitoring chip 300, and is processed by an optical filter layer, an image sensor and a processor in the optical artificial neural network smelting end point monitoring chip, so that whether the current recognition result is the smelting end point can be obtained.
Therefore, the purpose of the embodiment is to provide a novel photoelectric chip for smelting end point control, the chip comprises an input layer and a linear layer of an optical artificial neural network formed by an optical filter layer, a part of nonlinear activation functions of the nonlinear layer of the optical artificial neural network formed by an image sensor collects image information and spectral information of a furnace mouth in a smelting process on line, and the smelting end point, the furnace mouth temperature measurement, the detection and the qualitative analysis of carbon elements and the like can be realized quickly, accurately, safely and reliably.
The chip directly prepares a micro-nano modulation structure on the surface of a photosensitive area of an image sensor, a plurality of discrete or continuous micro-nano structures form a unit, and the micro-nano modulation structures at different positions have different spectrum modulation effects on incident light to jointly form an optical filter layer. The modulation intensity of the micro-nano modulation structures on different wavelength components of incident light corresponds to the connection intensity (linear layer weight) of the artificial neural network, meanwhile, square detection response of the image sensor carries out first nonlinear activation processing on incident light carrying information corresponding to different position points after being modulated by an optical filter layer, the incident light carrying information is converted into electric signals corresponding to different position points, the electric signals corresponding to the different position points are sent to the processor, the processor carries out full connection processing on the electric signals corresponding to the different position points, or the processor carries out full connection processing and second nonlinear activation processing on the electric signals corresponding to the different position points, and output signals of the artificial neural network are obtained. For smelting end point control, the converter mouth image and the spectrum information at the end point moment can be collected firstly, the weight of the linear layer, namely the system function of the optical filter layer, is obtained through data training, and the required optical filter layer structure can be designed reversely and integrated above the image sensor. In actual use, in the actual smelting end point control process, the output of the manufactured optical filter layer is reused, the weight of the electric signal full-connection layer is further trained and optimized, the high-accuracy optical artificial neural network can be realized, and the accurate judgment of the end point time in the smelting process is completed.
The chip can be understood to actually utilize the converter mouth image information and the spectrum information at the end point moment at the same time, so that the accurate online converter mouth temperature and the accurate element content are obtained, and the accuracy of the judgment of the smelting end point moment is improved; and an artificial neural network is partially realized on hardware, so that the speed of judging the smelting end point moment is increased. In addition, the chip scheme can realize mass production by utilizing the existing CMOS process, reduces the volume, power consumption and cost of devices, and is convenient to integrate with a subsequent control system.
It can be understood that, for the recognition task, since the optical artificial neural network smelting endpoint monitoring chip provided in this embodiment has an advantage of being able to acquire image information, spectrum information, angle information of incident light, and phase information of incident light at different points in the recognition object space, to fully utilize this advantage, a real recognition object is preferentially adopted for the recognition object sample serving as the input training sample, rather than a two-dimensional image of the recognition object. Of course, this does not mean that the two-dimensional image may not be used as the recognition object sample.
In the embodiment, the optical filter layer 1 is used as an input layer and a linear layer of the neural network, the image sensor 2 is used as a part of a nonlinear layer of the neural network (that is, the square detection response of the image sensor 2 is used as a first nonlinear activation function of the neural network), in order to minimize a loss function of the neural network, the modulation intensity of the optical modulation structure in the optical filter layer on different wavelength components in the incident light of the steelmaking furnace mouth is used as the connection weight from the input layer to the linear layer of the neural network, and the modulation intensity of the different wavelength components in the incident light of the steelmaking furnace mouth can be adjusted by adjusting the structure of the optical filter layer, so that the adjustment of the connection weight from the input layer to the linear layer is realized, and the training of the neural network is further optimized.
Therefore, in this embodiment, the optical modulation structure is obtained based on neural network training, a computer is used to perform optical simulation on a training sample to obtain the sample modulation intensities of the optical modulation structure on different wavelength components of incident light at a steelmaking converter mouth in an intelligent processing task in the training sample, the sample modulation intensities are used as the connection weights from an input layer to a linear layer of the neural network to perform nonlinear activation, and the training sample corresponding to the intelligent processing task is used to perform neural network training until the neural network converges, and the corresponding training sample optical modulation structure is used as an optical filter layer corresponding to the intelligent processing task.
It can be seen that, in the present embodiment, by implementing the input layer and the linear layer (optical filter layer) of the neural network and a part of the nonlinear layer (square detection response of the image sensor 2 as the first nonlinear activation function of the neural network) in the physical layer, complicated signal processing and algorithm processing corresponding to the input layer, the linear layer, and a part or all of the nonlinear activation functions in the prior art can be omitted, and thus the processing speed can be increased and the time delay can be reduced. Meanwhile, the embodiment of the invention actually utilizes the image information, the spectral information, the angle of the incident light and the phase information of the incident light of the steelmaking furnace mouth at the same time, namely the incident light carrying information at different points of the steelmaking furnace mouth space, so that the incident light carrying information at different points of the steelmaking furnace mouth space covers the information of the image, the composition, the shape, the three-dimensional depth, the structure and the like of the steelmaking furnace mouth, when the identification processing is carried out according to the incident light carrying information at different points of the steelmaking furnace mouth space, the multidimensional information of the image, the composition, the shape, the three-dimensional depth, the structure and the like of the steelmaking furnace mouth can be covered, and the problem that the identification accuracy is difficult to ensure by adopting the two-dimensional image information of the steelmaking furnace mouth mentioned in the background technology part can be solved.
Based on the content of the above embodiment, in this embodiment, when training an optical artificial neural network smelting endpoint monitoring chip that includes different optical modulation structures, image sensors, and processors with different full connection parameters, or training an optical artificial neural network smelting endpoint monitoring chip that includes different optical modulation structures, image sensors, and processors with different full connection parameters and different second nonlinear activation parameters, the different optical modulation structures are designed and implemented by adopting a computer optical simulation design.
In the embodiment, the light modulation structure is designed through computer optical simulation, the light modulation structure is adjusted through the optical simulation, and the corresponding light modulation structure is determined to be the light modulation structure size which needs to be manufactured finally until the neural network is converged, so that the prototype manufacturing time and cost are saved, the product efficiency is improved, and the complex optical problem is easily solved. For example, the light modulation structure can be simulated and designed through FDTD software, and the light modulation structure is changed in optical simulation, so that the modulation intensity of the light modulation structure on different incident lights can be accurately predicted, the modulation intensity can be used as the connection weight of a neural network input layer and a linear layer, an optical artificial neural network smelting endpoint monitoring chip is trained, and the light modulation structure can be accurately obtained.
It can be understood that the converter mouth image and the spectrum information at the end point moment can be collected for multiple times in advance and subjected to data training, a required micro-nano modulation structure can be designed and prepared, and the first-time nonlinear activation function in the input layer, the linear layer and the nonlinear layer of the artificial neural network is realized on a chip.
Therefore, the light modulation structure is designed in a computer optical simulation design mode, so that the time and the cost for manufacturing the prototype of the light modulation structure are saved, and the product efficiency is improved.
It can be understood that the chip actually utilizes the image information, the spectrum information, the angle information of the incident light and the phase information of the incident light at different points of the furnace mouth, so as to improve the accuracy of the smelting endpoint recognition. In addition, the chip scheme can realize mass production by utilizing the existing CMOS process, and the volume, the power consumption and the cost of the device are reduced.
The optical artificial neural network smelting end point monitoring chip based on the micro-nano modulation structure and the image sensor provided by the embodiment has a schematic structural diagram shown in fig. 2, and comprises an optical filter layer 1 and an image sensor 2And a processor 3. The optical filter layer 1 corresponds to an input layer and a linear layer of the optical artificial neural network, wherein a plurality of discrete or continuous micro-nano modulation structures form a unit, the modulation structures have different broadband spectrum modulation effects on incident light, the micro-nano modulation structures contained in different units can be the same or different, and spatial reconstruction can be performed according to images. Each cell corresponds to a plurality of image sensor light-sensitive pixels in a vertical direction. The square detection response of the image sensor 2 corresponds to a part of the nonlinear activation function of the optical artificial neural network, and the emergent light field E of the optical filter layer 1 N Conversion to photocurrent response I of image sensor N The above. The processor 3 comprises a signal readout circuit and a computer, the signal readout circuit in the processor 3 reads out the photocurrent response I N And transmitting the signal to a computer, carrying out full connection processing of the electric signal or carrying out nonlinear activation processing again by the computer, and finally outputting the result.
It can be understood that, for the smelting end point control, the modulation intensity (transmissivity) of the modulation structure to different wavelength components of incident light can be obtained by optically simulating the micro-nano modulation structure on a computer, the modulation intensity (transmissivity) is used as the connection weight from the input layer of the artificial neural network to the linear layer, the square detection response of the image sensor is simultaneously utilized to carry out the first nonlinear activation processing on the incident light carrying information corresponding to different position points after being modulated by the optical filter layer, the incident light carrying information is converted into electric signals corresponding to different position points, the electric signals corresponding to different position points are sent to the processor, the processor carries out the full connection processing on the electric signals corresponding to different position points, or the processor carries out the full connection processing and the second nonlinear activation processing on the electric signals corresponding to different position points, so as to obtain the output signal of the artificial neural network, the method is characterized in that the furnace mouth image and the spectrum information of the converter at the end point moment are collected for multiple times in advance and are subjected to data training, a required micro-nano modulation structure can be designed and prepared, and the first nonlinear activation function in an input layer, a linear layer and a nonlinear layer of an artificial neural network is realized on a chip.
Viewed in the longitudinal direction, as shown in FIG. 2, the optical filterEach micro-nano modulation structure in the layer is obtained through a pre-artificial neural network training design, and can be prepared by directly growing one or more layers of media or metal materials on the image sensor and then etching. The overall size of each modulation element in the optical filter layer is typically λ 2 ～10 5 λ 2 The thickness is usually 0.1 λ to 10 λ, λ being the central wavelength of the target band. Each modulating cell structure in the optical filter layer corresponds to a plurality of pixels on the image sensor. The optical filter layer is fabricated directly on the image sensor, with the image sensor and the processor being connected by electrical contacts.
It can be understood that the optical filter layer and the CIS wafer (the CIS wafer is used as a special image sensor) can be manufactured by a semiconductor CMOS integration process, the optical filter layer is monolithically integrated on the image sensor directly from the wafer level, and the chip can be manufactured by one-time chip flow in the CMOS process, so that monolithic integration can be realized at the wafer level, which is beneficial to reducing the distance between the sensor and the optical filter layer, reducing the volume of the device, and reducing the packaging cost.
Therefore, in the embodiment, the optical filter layer corresponds to the input layer and the linear layer of the artificial neural network and the connection weight from the input layer to the linear layer, the square detection response of the image sensor corresponds to the first nonlinear activation function in the nonlinear layer of the artificial neural network, the space spectrum information of the converter mouth is projected to the photocurrent response of the image sensor, the full connection and the second nonlinear activation of the electric signal are realized in the processor, and thus, the identification of whether the smelting end point is reached or not can be quickly realized.
The optical artificial neural network smelting end point monitoring chip based on the micro-nano modulation structure and the image sensor has the following effects: A. the artificial neural network is partially embedded into an image sensor comprising various optical filter layers, so that safe, reliable, rapid and accurate smelting end point control is realized. B. The detectable sample comprises but is not limited to the end point control of converter steelmaking, the artificial neural network is introduced to train and detect the temperature and the material elements of the smelting furnace mouth, the integration of a rear-end industrial control system is very easy, the identification accuracy is high, and the qualitative analysis is accurate. C. The preparation of the chip can be completed through one-time chip flow of a CMOS process, the failure rate of a device is reduced, the finished product yield of the device is improved, and the cost is reduced. D. The monolithic integration is realized at the wafer level, the distance between the sensor and the optical filter layer can be reduced to the greatest extent, the size of a unit is reduced, and the size and the packaging cost of a device are reduced.
Based on the content of the above embodiments, in the present embodiment, the light modulation structure in the optical filter layer includes a regular structure and/or an irregular structure; and/or, the light modulating structures in the optical filter layer comprise discrete type structures and/or continuous type structures.
In this embodiment, the light modulation structure in the optical filter layer may only include a regular structure, may also only include an irregular structure, and may also include both a regular structure and an irregular structure.
In this embodiment, where the light modulation structure includes a regular structure, it may mean: the minimum modulation units included in the light modulation structure are regular structures, for example, the minimum modulation units can be in regular patterns such as rectangles, squares and circles. Further, where the light modulating structure comprises a regular structure, it may also refer to: the arrangement of the minimum modulation units included in the light modulation structure is regular, for example, the arrangement may be in a regular array form, a circular form, a trapezoidal form, a polygonal form, and the like. Further, where the light modulating structure comprises a regular structure, it may also refer to: the minimum modulation units included in the light modulation structure are regular structures, and the arrangement mode of the minimum modulation units is also regular.
In this embodiment, the light modulation structure including the irregular structure may refer to: the minimum modulation units included in the light modulation structure are irregular structures, for example, the minimum modulation units may be irregular patterns such as irregular polygons and random shapes. Further, where the light modulating structure comprises an irregular structure, it may also mean: the arrangement of the minimum modulation units included in the light modulation structure is irregular, for example, the arrangement may be in an irregular polygon form, a random arrangement form, or the like. Further, where the light modulating structure includes an irregular structure, it may also mean: the minimum modulation units included in the light modulation structure are irregular structures, and the arrangement mode of the minimum modulation units is also irregular.
In this embodiment, the optical modulation structure in the optical filter layer may include a discrete structure, a continuous structure, or both a discrete structure and a continuous structure.
In this embodiment, where the light modulation structure includes a continuous type structure, it may mean: the light modulation structure is formed by continuous modulation patterns; where the light modulating structure comprises a discrete structure may refer to: the light modulating structure is formed by a discrete modulation pattern.
It will be appreciated that the continuous modulation pattern herein may refer to a rectilinear pattern, a wavy pattern, a broken pattern, and the like.
It is understood that a discrete modulation pattern may refer to a modulation pattern formed by a discrete pattern (e.g., discrete dots, discrete squares, discrete irregular multi-deformations, etc.).
In this embodiment, it should be noted that the optical modulation structure has different modulation effects on light with different wavelengths, and specific modulation methods include, but are not limited to, scattering, absorption, interference, surface plasmon, resonance enhancement, and the like. By designing different filter structures, corresponding transmission spectrums are different after light passes through different groups of filter structures.
Based on the content of the above embodiments, in the present embodiment, the optical filter layer has a single-layer structure or a multi-layer structure.
In this embodiment, the optical filter layer may have a single-layer filter structure, or may have a multi-layer filter structure, for example, a multi-layer structure including two, three, or four layers.
In the present embodiment, as shown in fig. 1, the optical filter layer 1 is a single-layer structure, the thickness of the optical filter layer 1 is related to the target wavelength range, and the thickness of the grating structure may be 50nm to 5 μm for wavelengths of 400nm to 10 μm.
It is understood that since the optical filter layer 1 serves to spectrally modulate incident light, it is preferable to fabricate materials with high refractive index and low loss, such as silicon, germanium, silicon-germanium materials, silicon compounds, germanium compounds, III-V materials, and the like, wherein silicon compounds include, but are not limited to, silicon nitride, silicon dioxide, silicon carbide, and the like.
In addition, it should be noted that, in order to form more or more complex connection weights between the input layer and the linear layer, preferably, the optical filter layer 1 may be set to be a multilayer structure, and the light modulation structures corresponding to each layer may be set to be different structures, so as to increase the spectral modulation capability of the optical filter layer on incident light, so as to form more or more complex connection weights between the input layer and the linear layer, and further improve the accuracy of the optical artificial neural network smelting endpoint monitoring chip when processing an intelligent task.
In addition, for the filter layer including a multi-layer structure, the material of each layer structure may be the same or different, for example, for the optical filter layer 1 having two layers, the first layer may be a silicon layer, and the second layer may be a silicon nitride layer.
The thickness of the optical filter layer 1 is related to the target wavelength range, and the total thickness of the multilayer structure may be 50nm to 5 μm for wavelengths of 400nm to 10 μm.
Based on the content of the above embodiment, in this embodiment, the light modulation structure in the optical filter layer includes a unit array composed of a plurality of micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor; the structures of the micro-nano units are the same or different.
In this embodiment, in order to obtain connection weights (used for connecting connection weights between the input layer and the linear layer) distributed in an array so as to facilitate the processor to perform subsequent full-connection and nonlinear activation processing, preferably, in this embodiment, the light modulation structure is in an array structure form, specifically, the light modulation structure includes a unit array composed of a plurality of micro-nano units, and each micro-nano unit corresponds to one or more pixel points on the image sensor. It should be noted that the structures of the micro-nano units may be the same or different. In addition, the structure of each micro-nano unit may be periodic or non-periodic. In addition, it should be noted that each micro-nano unit may further include multiple groups of micro-nano structure arrays, and the structures of the micro-nano structure arrays of each group are the same or different.
As illustrated in fig. 5 to 9, in this embodiment, as shown in fig. 5, the optical filter layer 1 includes a plurality of repeated continuous or discrete micro-nano units, such as 11, 22, 33, 44, 55, and 66, each micro-nano unit has the same structure (and each micro-nano unit has a non-periodic structure), and each micro-nano unit corresponds to one or more pixel points on the image sensor 2; as shown in fig. 6, the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66, each micro-nano unit has the same structure (different from fig. 5 in that each micro-nano unit in fig. 6 is a periodic structure), and each micro-nano unit corresponds to one or more pixel points on the image sensor 2; as shown in fig. 7, the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, 66, each micro-nano unit has the same structure (and each micro-nano unit is a periodic structure), each micro-nano unit corresponds to one or more pixel points on the image sensor 2, and the difference from fig. 6 is that the unit shape of the periodic array in each micro-nano unit in fig. 7 has quadruple rotational symmetry; as shown in fig. 8, the optical filter layer 1 includes a plurality of micro-nano units, such as 11, 22, 33, 44, 55, and 66, and the difference from fig. 6 lies in that each micro-nano unit has a different structure, and each micro-nano unit corresponds to one or more pixel points on the image sensor 2, in this embodiment, the optical filter layer 1 includes a plurality of micro-nano units that are different from each other, that is, the modulation effect of different regions on incident light on the optical artificial neural network smelting endpoint monitoring chip is different, so that the degree of freedom of design is improved, and further, the accuracy of identification can be improved. As shown in fig. 9, the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66, each of which has the same structure, and is different from fig. 5 in that each of the micro-nano units is composed of a discrete non-periodic array structure, and each of the micro-nano units corresponds to one or more pixels on the image sensor 2.
In this embodiment, the micro-nano unit has different modulation effects on light with different wavelengths, and specific modulation methods include, but are not limited to, scattering, absorption, interference, surface plasmons, resonance enhancement, and the like. By designing different filter structures, corresponding transmission spectrums are different after light passes through different groups of filter structures.
Based on the content of the above embodiment, in this embodiment, the micro-nano unit includes a regular structure and/or an irregular structure; and/or the micro-nano unit comprises a discrete structure and/or a continuous structure.
In this embodiment, the micro-nano unit may only include a regular structure, may also only include an irregular structure, and may also include both a regular structure and an irregular structure.
In this embodiment, the micro-nano unit including a regular structure may refer to: the minimum modulation unit contained in the micro-nano unit is of a regular structure, for example, the minimum modulation unit can be a regular pattern such as a rectangle, a square and a circle. In addition, the micro-nano unit including a regular structure here may also mean: the arrangement mode of the minimum modulation units contained in the micro-nano units is regular, for example, the arrangement mode can be a regular array form, a circular form, a trapezoidal form, a polygonal form and the like. In addition, the micro-nano unit including a regular structure here can also mean: the minimum modulation units contained in the micro-nano units are of a regular structure, and the arrangement mode of the minimum modulation units is also regular.
In this embodiment, the micro-nano unit including an irregular structure may refer to: the minimum modulation unit contained in the micro-nano unit is of an irregular structure, for example, the minimum modulation unit can be an irregular figure such as an irregular polygon, a random shape and the like. In addition, the micro-nano unit including an irregular structure here can also mean: the arrangement mode of the minimum modulation units contained in the micro-nano units is irregular, for example, the arrangement mode can be an irregular polygon form, a random arrangement form and the like. In addition, the micro-nano unit comprising an irregular structure here can also mean: the minimum modulation units contained in the micro-nano units are of irregular structures, and the arrangement mode of the minimum modulation units is also irregular.
In this embodiment, the micro-nano unit in the optical filter layer may include a discrete structure, may also include a continuous structure, and may also include both a discrete structure and a continuous structure.
In this embodiment, the micro-nano unit including a continuous structure may refer to: the micro-nano unit is formed by continuous modulation patterns; here, the micro-nano unit including a discrete structure may mean: the micro-nano unit is formed by discrete modulation patterns.
It will be appreciated that the continuous modulation pattern herein may refer to a rectilinear pattern, a wavy pattern, a broken pattern, and the like.
It is to be understood that a discrete modulation pattern herein may refer to a modulation pattern formed by discrete patterns (e.g., discrete dots, discrete triangles, discrete stars, etc.).
In this embodiment, it should be noted that different micro-nano units have different modulation effects on light with different wavelengths, and specific modulation methods include, but are not limited to, scattering, absorption, interference, surface plasmons, resonance enhancement, and the like. By designing different micro-nano units, corresponding transmission spectrums are different after light passes through different groups of micro-nano units.
Based on the content of the above embodiment, in this embodiment, the micro-nano unit includes multiple sets of micro-nano structure arrays, and the structures of the micro-nano structure arrays of the sets are the same or different.
In this embodiment, as shown in fig. 5, the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66, each micro-nano unit includes a plurality of micro-nano structure arrays, for example, the micro-nano unit 11 includes 4 different micro-nano structure arrays 110, 111, 112, and 113, and the filter unit 44 includes 4 different micro-nano structure arrays 440, 441, 442, and 443. As shown in fig. 10, the optical filter layer 1 includes a plurality of micro-nano units, such as 11, 22, 33, 44, 55, 66, each micro-nano unit includes a plurality of micro-nano structure arrays, for example, the micro-nano unit 11 includes 4 identical micro-nano structure arrays 110, 111, 112, and 113.
It should be noted that, here, the micro-nano units including four groups of micro-nano structure arrays are only used for illustration, and do not play a limiting role, and in practical application, the micro-nano units including six groups, eight groups, or other number of groups of micro-nano structure arrays may also be set as needed.
In this embodiment, each group of micro-nano structure arrays in the micro-nano unit has different modulation effects on light with different wavelengths, and the modulation effects on input light between each group of filtering structures are also different, and specific modulation modes include, but are not limited to, scattering, absorption, interference, surface plasmons, resonance enhancement, and the like. By designing different micro-nano structure arrays, corresponding transmission spectrums are different after light passes through different groups of micro-nano structure arrays.
Based on the content of the above embodiment, in this embodiment, each group of micro-nano structure arrays has a function of broadband filtering or narrowband filtering.
In this embodiment, in order to obtain the modulation intensities of the different wavelength components of the incident light at the steelmaking furnace mouth as the connection weights of the neural network input layer and the linear layer, the broadband filtering and the narrow-band filtering are realized by adopting different micro-nano structure arrays, so that the micro-nano structure arrays in this embodiment obtain the modulation intensities of the different wavelength components of the incident light at the steelmaking furnace mouth by performing the broadband filtering or the narrow-band filtering on the incident light at the steelmaking furnace mouth. As shown in fig. 11 and 12, each group of micro-nano structure array in the optical filter layer has a function of broadband filtering or narrowband filtering.
It can be understood that each group of micro-nano structure arrays can have a broadband filtering function, can also have a narrowband filtering function, can also partially have a broadband filtering function, and partially have a narrowband filtering function. In addition, the wide-band filtering range and the narrow-band filtering range of each group of micro-nano structure arrays can be the same or different. For example, by designing the size parameters such as the period, duty ratio, radius, side length and the like of each group of micro-nano structures in the micro-nano unit, the micro-nano unit has a narrow-band filtering effect, that is, only light with one (or a few) wavelength can pass through the micro-nano unit. For another example, by designing the dimensional parameters such as the period, duty ratio, radius, side length and the like of each group of micro-nano structures in the micro-nano unit, the micro-nano unit has a broadband filtering effect, that is, light with more wavelengths or all wavelengths can be allowed to pass through.
It can be understood that, in specific use, the filtering state of each group of micro-nano structure arrays can be determined in a manner of performing broadband filtering, narrow-band filtering or a combination thereof according to an application scene.
Based on the content of the foregoing embodiment, in this embodiment, each group of micro-nano structure arrays is a periodic structure array or a non-periodic structure array.
In this embodiment, each micro-nano structure array may be a periodic structure array, or may be a non-periodic structure array, or may be a partial periodic structure array and a partial non-periodic structure array. The periodic structure array is easy to carry out optical simulation design, and the non-periodic structure array can realize more complex modulation action.
In this embodiment, as shown in fig. 5, the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, 66, each micro-nano unit is composed of a plurality of micro-nano structure arrays, the micro-nano structure arrays have different structures, and the micro-nano structure arrays are non-periodic structures. The aperiodic structure refers to that the shapes of the modulation holes on the micro-nano structure array are arranged according to a non-periodic arrangement mode. As shown in fig. 5, the micro-nano unit 11 includes 4 different aperiodic structure arrays 110, 111, 112, and 113, the micro-nano unit 44 includes 4 different aperiodic structure arrays 440, 441, 442, and 443, and the micro-nano structure array with aperiodic structure is designed by training neural network data aiming at an intelligent processing task in the previous stage, and is usually an irregularly shaped structure. As shown in fig. 6, the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, 66, each of which is composed of a plurality of sets of micro-nano structure arrays, the structures of the micro-nano structure arrays are different from each other, and unlike fig. 5, the micro-nano structure array is a periodic structure. The periodic structure refers to that the shapes of modulation holes on the micro-nano structure array are arranged according to a periodic arrangement mode, and the size of the period is usually 20 nm-50 mu m. As shown in fig. 6, the micro-nano unit 11 includes 4 different periodic structure arrays 110, 111, 112, and 113, the micro-nano unit 44 includes 4 different periodic structure arrays 440, 441, 442, and 443, and the filter structure of the periodic structure is designed by training neural network data for an intelligent processing task in an early stage, and is usually an irregularly shaped structure. As shown in fig. 7, the optical filter layer 1 includes a plurality of micro-nano units, such as 11, 22, 33, 44, 55, and 66, which are different from each other, each micro-nano unit is composed of a plurality of micro-nano structure arrays, the micro-nano structure arrays are different from each other, and the micro-nano structure arrays are periodic structures. The periodic structure refers to the shape of the filter structure arranged in a periodic arrangement, and the period is usually 20nm to 50 μm. As shown in fig. 7, the micro-nano structure arrays of the micro-nano unit 11 and the micro-nano unit 12 are different from each other, the micro-nano unit 11 includes 4 different periodic structure arrays 110, 111, 112, and 113, the micro-nano unit 44 includes 4 different periodic structure arrays 440, 441, 442, and 443, and the micro-nano structure array of the periodic structure is obtained by training and designing neural network data aiming at an intelligent processing task in an early stage, and is generally an irregular structure.
It should be noted that each of the micro-nano units in fig. 5 to 9 includes four micro-nano structure arrays, the four micro-nano structure arrays are formed by four modulation holes with different shapes, and the four micro-nano structure arrays are used for modulating incident light differently. It should be noted that, here, the micro-nano units including four groups of micro-nano structure arrays are only used for illustration, and do not play a limiting role, and in practical application, the micro-nano units including six groups, eight groups, or other number of groups of micro-nano structure arrays may also be set as needed. In the present embodiment, the four different shapes may be (without limitation) a circle, a cross, a regular polygon, and a rectangle.
In this embodiment, each group of micro-nano structure arrays in the micro-nano unit has different modulation effects on light with different wavelengths, and the modulation effects on input light between each group of micro-nano structure arrays are also different, and specific modulation modes include, but are not limited to, scattering, absorption, interference, surface plasmons, resonance enhancement, and the like. By designing different micro-nano structure arrays, corresponding transmission spectrums are different after light passes through different groups of micro-nano structure arrays.
Based on the content of the foregoing embodiment, in this embodiment, the micro-nano unit includes one or more groups of hollow structures in a plurality of groups of micro-nano structure arrays.
For example, referring to an example shown in fig. 9, in this embodiment, as shown in fig. 9, the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66, each micro-nano unit is composed of a plurality of micro-nano structure arrays, structures corresponding to the plurality of micro-nano structure arrays are different from each other, the micro-nano structure arrays are periodic structures, and different from the above embodiment, for any micro-nano unit, one or more groups of empty structures are included, and the empty structures are used for directly passing through incident light. It can be understood that when the micro-nano structure arrays comprise one or more groups of hollow structures, a richer spectrum modulation effect can be formed, so that the spectrum modulation requirement under a specific scene (or the specific connection weight requirement between the input layer and the linear layer under the specific scene) is met.
As shown in fig. 9, each micro-nano unit includes a group of micro-nano structure arrays and three groups of hollow structures, the micro-nano unit 11 includes 1 aperiodic structure array 111, the micro-nano unit 22 includes 1 aperiodic structure array 221, the micro-nano unit 33 includes 1 aperiodic structure array 331, the micro-nano unit 44 includes 1 aperiodic structure array 441, the micro-nano unit 55 includes 1 aperiodic structure array 551, and the micro-nano unit 66 includes 1 aperiodic structure array 661, where the micro-nano structure arrays are used for performing different modulations on incident light. It should be noted that, here, the example is only given by including a group of micro-nano structure arrays and three groups of hollow structures, and does not play a limiting role, and in practical application, micro-nano units including a group of micro-nano structure arrays and five groups of hollow structures or other numbers of groups of micro-nano structure arrays may also be set according to needs. In this embodiment, the micro-nano structure array may be made of modulation holes having a circular shape, a cross shape, a regular polygon shape, and a rectangular shape (not limited thereto).
It should be noted that, the micro-nano units may not include any empty structure in the multi-group micro-nano structure array, that is, the multi-group micro-nano structure array may be a non-periodic structure array or a periodic structure array.
Based on the content of the foregoing embodiment, in this embodiment, the micro-nano unit has a polarization-independent property.
In the embodiment, the micro-nano unit has a polarization-independent characteristic, so that the optical filter layer is insensitive to the polarization of incident light, and the optical artificial neural network smelting endpoint monitoring chip insensitive to an incident angle and polarization is realized. The optical artificial neural network smelting end point monitoring chip provided by the embodiment of the invention is insensitive to the incident angle and the polarization characteristic of incident light, namely, the measurement result is not influenced by the incident angle and the polarization characteristic of the incident light, so that the stability of the spectral measurement performance can be ensured, and the stability of intelligent processing, such as the stability of intelligent perception, the stability of intelligent recognition, the stability of intelligent decision and the like, can be further ensured. The micro-nano cells may also have polarization dependent properties.
Based on the content of the foregoing embodiment, in this embodiment, the micro-nano unit has quadruple rotational symmetry.
In this embodiment, it should be noted that the quadruple rotational symmetry belongs to a specific case of the polarization-independent characteristic, and the requirement of the polarization-independent characteristic can be satisfied by designing the micro-nano unit to have a structure with quadruple rotational symmetry.
As illustrated below with reference to the example shown in fig. 7, in this embodiment, as shown in fig. 7, the optical filter layer 1 includes a plurality of repeated micro-nano units, such as 11, 22, 33, 44, 55, and 66, each micro-nano unit is composed of a plurality of micro-nano structure arrays, the structures corresponding to the plurality of micro-nano structure arrays are different from each other, the micro-nano structure arrays are periodic structures, and different from the above embodiments, the structure corresponding to each micro-nano structure array may be a structure having quadruple rotational symmetry, such as a circle, a cross, a regular polygon, and a rectangle, that is, after the structure is rotated by 90 °, 180 °, and 270 °, the structure is overlapped with the original structure, so that the structure has a polarization-independent characteristic, and the same intelligent identification effect can be obtained when different polarized light is incident.
Based on the contents of the above embodiments, in the present embodiment, the optical filter layer is composed of one or more layers;
each layer of structure is prepared by one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials and perovskite materials; and/or the filter layer is prepared from one or more of a photonic crystal, a super Surface, a random structure, a nano structure, a metal Surface Plasmon Polaritons (SPP) micro-nano structure and a tunable Fabry-Perot Cavity (FP Cavity).
The semiconductor material comprises one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed according to a preset proportion and direct band gap compound semiconductor materials; and/or the nanostructure comprises one or more of a nanodot two-dimensional material, a nanocolumn two-dimensional material and a nanowire two-dimensional material.
The photonic crystal, the super surface and the random structure combination can adopt CMOS (complementary metal oxide semiconductor) process compatibility, and have better modulation effect, and micro-nano modulation structure micropores can also be filled with other materials for surface smoothing; the quantum dots and the perovskite can utilize the spectral modulation characteristics of the material to minimize the volume of a single modulation structure; the SPP is small in size, and polarization-dependent light modulation can be realized; the liquid crystal can be dynamically regulated and controlled by voltage, so that the spatial resolution is improved; the adjustable Fabry-Perot resonant cavity can be dynamically adjusted and controlled, and the spatial resolution is improved.
Based on the contents of the above embodiments, in the present embodiment, the thickness of the optical filter layer is 0.1 λ to 10 λ, where λ denotes the center wavelength of incident light.
In this embodiment, it should be noted that if the thickness of the optical filter layer is much smaller than the central wavelength of the incident light, the effective spectrum modulation effect cannot be achieved; if the thickness of the optical filter layer is much larger than the center wavelength of the incident light, it is difficult to fabricate in the process and a large optical loss is introduced. Therefore, the present embodiment is easy to manufacture for reducing optical loss and ensures effectivenessThe overall size (area) of each micro-nano unit in the optical filter layer 1 is usually lambda under the action of spectrum modulation 2 ～10 5 λ 2 The thickness is usually 0.1 to 10 lambda (lambda represents the central wavelength of incident light to the steel-making spout). As shown in FIG. 5, the overall size of each micro-nano unit is 0.5 μm 2 ～40000μm 2 The dielectric material in the optical filter layer 1 is polysilicon with a thickness of 50 nm-2 μm.
Based on the content of the above embodiments, in the present embodiment, the image sensor is any one or more of the following:
CMOS Image Sensors (CIS), Charge Coupled Devices (CCD), Single Photon Avalanche Diode (SPAD) arrays, and focal plane photo-electric Image Sensor arrays.
In this embodiment, it should be noted that, by using the wafer-level CMOS image sensor CIS, monolithic integration is implemented at a wafer level, which may reduce a distance between the image sensor and the optical filter layer to the maximum extent, and is beneficial for reducing a size of a cell and reducing a device volume and a package cost, the SPAD may be used for weak light detection, and the CCD may be used for strong light detection.
In this embodiment, the optical filter layer and the image sensor may be manufactured by a Complementary Metal Oxide Semiconductor (CMOS) integrated process, which is beneficial to reducing the failure rate of the device, improving the yield of the device, and reducing the cost. For example, the optical filter layer can be prepared by growing one or more layers of dielectric material directly on the image sensor, etching, depositing a metal material before removing the sacrificial layer used for etching, and finally removing the sacrificial layer.
Based on the content of the above embodiment, in the present embodiment, the types of the artificial neural network include: a neural network is fed forward.
In the present embodiment, a feed Forward Neural Network (FNN), also called a Deep feed forward Network (DFN), and a Multi-Layer Perceptron (MLP) are the simplest Neural networks, and each neuron is arranged in a hierarchical manner. Each neuron is connected to only the neuron in the previous layer. And receiving the output of the previous layer and outputting the output to the next layer, wherein no feedback exists between the layers. The feedforward neural network has simple structure, easy realization on hardware and wide application, can approach any continuous function and square integrable function with any precision, and can accurately realize any finite training sample set. The feed forward network is a static non-linear mapping. Complex non-linear processing capabilities can be obtained by complex mapping of simple non-linear processing units.
Based on the content of the foregoing embodiments, in this embodiment, a light-transmitting medium layer is disposed between the optical filter layer and the image sensor.
In this embodiment, it should be noted that, by disposing the transparent medium layer between the optical filter layer and the image sensor, the optical filter layer and the image sensor layer can be effectively separated from each other, and mutual interference between the optical filter layer and the image sensor layer is avoided.
Based on the content of the above embodiments, in the present embodiment, the image sensor is a front-illuminated type, including: the optical filter layer is integrated on one surface, far away from the optical detection layer, of the metal wire layer; or the like, or a combination thereof,
the image sensor is of a back-illuminated type, including: the optical filter comprises an optical detection layer and a metal wire layer which are arranged from top to bottom, wherein an optical filter layer is integrated on one surface of the optical detection layer, which is far away from the metal wire layer.
In the present embodiment, as shown in fig. 13, which is a front-illuminated image sensor, the silicon detection layer 21 is below the metal line layer 22, and the optical filter layer 1 is directly integrated onto the metal line layer 22.
In the present embodiment, unlike fig. 13, fig. 14 shows a back-illuminated image sensor, in which a silicon detection layer 21 is above a metal wire layer 22 and an optical filter layer 1 is directly integrated onto the silicon detection layer 21.
It should be noted that, for the back-illuminated image sensor, the silicon detection layer 21 is above the metal wire layer 22, so that the influence of the metal wire layer on the incident light can be reduced, and the quantum efficiency of the device can be improved.
According to the above, in the embodiment, the optical filter layer is used as the input layer and the linear layer of the artificial neural network, the image sensor is used as a part of the nonlinear layer of the artificial neural network (the square detection response of the image sensor is used as the first nonlinear activation function of the artificial neural network), the filtering effect of the optical filter layer on the incident light entering the optical filter layer is used as the connection weight from the input layer to the linear layer, the optical filter layer and the image sensor in the optical artificial neural network smelting endpoint monitoring chip provided by the embodiment realize the relevant functions of the input layer, the linear layer and the part of the nonlinear activation function in the artificial neural network in a hardware manner, so that the complicated signal processing and algorithm processing corresponding to the input layer, the linear layer and the part of the nonlinear activation function are not required to be performed when the optical artificial neural network smelting endpoint monitoring chip is used for intelligent processing in the following steps, therefore, the power consumption and the time delay during the artificial neural network processing can be greatly reduced. In addition, the embodiment simultaneously utilizes the image information of the steelmaking furnace mouth and the spectrum information of different points in space, thereby more accurately realizing the intelligent processing of the steelmaking furnace mouth.
Therefore, in the embodiment of the invention, the optical filter layer is used as the input layer and the linear layer of the artificial neural network, the image sensor is used as a part of the nonlinear layer of the artificial neural network, the spatial spectrum information of the object is projected into the photocurrent response of the image sensor, the full connection and the secondary nonlinear activation of the electric signal are realized in the processor, and the functions of intelligent sensing, identification and/or decision making with low power consumption, low time delay and high accuracy are realized. The optical artificial neural network smelting end point monitoring chip based on the optical filter and the image sensor has the following effects: the artificial neural network is partially embedded into an image sensor comprising various optical filter layers, so that the functions of quick and accurate intelligent perception, identification and/or decision making are realized. In addition, the embodiment of the invention can realize monolithic integration at the wafer level, thereby reducing the distance between the sensor and the optical filter layer to the greatest extent, being beneficial to reducing the size of a unit and reducing the volume and the packaging cost of a device.
Based on the same inventive concept, another embodiment of the present invention provides an intelligent smelting control apparatus, comprising: the chip for monitoring the smelting endpoint of the optical artificial neural network is as described in the above embodiment. The intelligent smelting control device may include various devices related to smelting process control, which is not limited in this embodiment.
Because the intelligent smelting control device provided by the embodiment includes the optical artificial neural network smelting endpoint monitoring chip described in the above embodiment, the intelligent smelting control device provided by the embodiment has all the beneficial effects of the optical artificial neural network smelting endpoint monitoring chip described in the above embodiment, and because the above embodiment has been described in detail, this embodiment is not described again.
Based on the same inventive concept, another embodiment of the present invention provides a method for preparing an optical artificial neural network smelting endpoint monitoring chip according to the above embodiment, as shown in fig. 15, the method specifically includes the following steps:
step 1510, preparing an optical filter layer including an optical modulation structure on the surface of the image sensor;
step 1520, generating a processor with the function of performing full connection processing on the signal or generating a processor with the functions of performing full connection processing and secondary nonlinear activation processing on the signal;
step 1530, connecting the image sensor and the processor;
the optical filter layer is used for respectively carrying out different spectrum modulation on incident light entering different position points of the optical modulation structure through the optical modulation structure so as to obtain incident light carrying information corresponding to the different position points on the surface of the image sensor; the incident light carrying information comprises light intensity distribution information, spectrum information, angle information of the incident light and phase information of the incident light;
the image sensor carries out first nonlinear activation processing on incident light carrying information corresponding to different position points after being modulated by the optical filter layer through square detection response, converts the incident light carrying information into electric signals corresponding to the different position points, and sends the electric signals corresponding to the different position points to the processor;
and the processor performs full connection processing on the electric signals corresponding to the different position points, or performs full connection processing and secondary nonlinear activation processing on the electric signals corresponding to the different position points to obtain an output signal of the artificial neural network.
In this embodiment, the method further includes a training process of the optical artificial neural network smelting endpoint monitoring chip, specifically including:
training an optical artificial neural network smelting endpoint monitoring chip comprising different optical modulation structures, image sensors and processors with different full-connection parameters by using input training samples and output training samples corresponding to the smelting endpoint monitoring task to obtain the optical modulation structures, the image sensors and the processors meeting the training convergence conditions, and taking the optical modulation structures, the image sensors and the processors meeting the training convergence conditions as the trained optical modulation structures, the trained image sensors and the trained processors;
or training an optical artificial neural network smelting endpoint monitoring chip comprising different optical modulation structures, image sensors and processors with different full-connection parameters and different second nonlinear activation parameters by using input training samples and output training samples corresponding to the smelting endpoint monitoring task to obtain the optical modulation structures, the image sensors and the processors meeting the training convergence conditions, and taking the optical modulation structures, the image sensors and the processors meeting the training convergence conditions as the trained optical modulation structures, the trained image sensors and the trained processors.
It can be understood that when the optical artificial neural optical network smelting endpoint monitoring chip comprising different optical modulation structures, image sensors and processors with different full connection parameters is trained, or the optical artificial neural optical network smelting endpoint monitoring chip comprising different optical modulation structures, image sensors and processors with different full connection parameters and different secondary nonlinear activation parameters is trained, the different optical modulation structures are designed and realized by adopting a computer optical simulation design mode.
It can be understood that the converter mouth image and the spectrum information at the end point moment can be collected for multiple times in advance and subjected to data training, a required micro-nano modulation structure can be designed and prepared, and the first-time nonlinear activation function in the input layer, the linear layer and the nonlinear layer of the artificial neural network is realized on a chip.
In this embodiment, preparing an optical filter layer including a light modulation structure on a surface of a photosensitive region of the image sensor includes:
growing one or more layers of preset materials on the surface of the image sensor;
performing dry etching on the one or more layers of preset materials to obtain an optical filter layer containing an optical modulation structure;
or the one or more layers of preset materials are subjected to imprinting transfer to obtain an optical filter layer containing an optical modulation structure;
or the one or more layers of preset materials are subjected to additional dynamic regulation to obtain an optical filter layer containing an optical modulation structure;
or printing the one or more layers of preset materials in a partition mode to obtain an optical filter layer containing an optical modulation structure;
or carrying out partition material growth on the one or more layers of preset materials to obtain an optical filter layer containing an optical modulation structure;
or quantum dot transfer is carried out on the one or more layers of preset materials to obtain the optical filter layer containing the optical modulation structure.
When the optical artificial neural network smelting end point monitoring chip is used for an intelligent processing task of a steelmaking furnace mouth, training the optical artificial neural network smelting end point monitoring chip comprising different optical modulation structures, image sensors and processors with different full-connection parameters by using input training samples and output training samples corresponding to the intelligent processing task to obtain the optical modulation structures, the image sensors and the processors meeting the training convergence conditions; or training an optical artificial neural network smelting endpoint monitoring chip comprising different optical modulation structures, image sensors and processors with different full-connection parameters and different second nonlinear activation parameters to obtain the optical modulation structures, the image sensors and the processors meeting the training convergence conditions.
In this embodiment, it should be noted that, as shown in fig. 1, the optical filter layer 1 may be prepared by directly growing one or more layers of dielectric materials on the image sensor 2, then etching, depositing a metal material before removing the sacrificial layer for etching, and finally removing the sacrificial layer. By designing the size parameters of the light modulation structure, each unit can have different modulation effects on light with different wavelengths in a target range, and the modulation effects are insensitive to incident angles and polarization. Each cell in the optical filter layer 1 corresponds to one or more pixels on the image sensor 2. 1 is prepared directly on 2.
In this embodiment, it should be noted that, as shown in fig. 14, assuming that the image sensor 2 is a back-illuminated structure, the optical filter layer 1 may be prepared by directly etching on the silicon image sensor layer 21 of the back-illuminated image sensor and then depositing metal.
In addition, it should be noted that the optical modulation structure on the optical filter layer may be dry-etched by performing a pattern of the optical modulation structure on one or more layers of the preset materials, where the dry-etching is to directly remove an unnecessary portion of the one or more layers of the preset materials on the surface of the photosensitive area of the image sensor, so as to obtain the optical filter layer including the optical modulation structure; or one or more layers of preset materials are subjected to imprinting transfer, the imprinting transfer is to prepare a required structure on other substrates through etching, and the structure is transferred to a photosensitive area of the image sensor through PDMS and other materials to obtain an optical filter layer containing an optical modulation structure; or one or more layers of preset materials are subjected to external dynamic regulation and control, wherein the external dynamic regulation and control adopts active materials, and then an external electrode is used for regulating and controlling the light modulation characteristics of corresponding areas by changing voltage to obtain an optical filter layer containing a light modulation structure; or one or more layers of preset materials are printed in a partition mode, and the partition printing is to obtain an optical filter layer containing the light modulation structure by adopting a printing technology in the partition mode; or carrying out partition material growth on one or more layers of preset materials to obtain an optical filter layer containing an optical modulation structure; or quantum dot transfer is carried out on one or more layers of preset materials to obtain the optical filter layer containing the optical modulation structure.
In addition, it should be noted that, because the preparation method provided in this embodiment is the preparation method of the optical artificial neural network smelting endpoint monitoring chip in the foregoing embodiment, for details of some principles, structures, and other aspects, reference may be made to the description of the foregoing embodiment, and details of this embodiment are not repeated here.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (21)

1. An optical artificial neural network smelting end point monitoring chip is characterized by being used for a smelting end point monitoring task and comprising the following steps: an optical filter layer, an image sensor, and a processor; the optical filter layer corresponds to an input layer and a linear layer of the artificial neural network and a connection weight of the input layer to the linear layer, and the square detection response of the image sensor corresponds to a first nonlinear activation function in a nonlinear layer of the artificial neural network; the processor corresponds to a full connection and output layer of the artificial neural network, or corresponds to a second nonlinear activation function and an output layer in the full connection and nonlinear layer of the artificial neural network;
the optical filter layer is arranged on the surface of the image sensor and comprises an optical modulation structure, and the optical modulation structure is used for respectively carrying out different spectrum modulation on incident light entering different position points of the optical modulation structure so as to obtain incident light carrying information corresponding to different position points on the surface of the image sensor; the incident light comprises reflected light, transmitted light and/or radiated light of a steelmaking furnace mouth;
the image sensor carries out first nonlinear activation processing on incident light carrying information corresponding to different position points after being modulated by the optical filter layer through square detection response, then converts the incident light carrying information into electric signals corresponding to the different position points, and sends the electric signals corresponding to the different position points to the processor;
the processor performs full connection processing on the electric signals corresponding to different position points, or performs full connection processing and secondary nonlinear activation processing on the electric signals corresponding to different position points to obtain a smelting end point monitoring result;
the smelting end point monitoring task comprises the step of identifying a smelting end point, and the smelting end point monitoring result comprises a smelting end point identification result.
2. The smart agriculture precision control chip of claim 1 wherein the incident light carrying information includes at least one of light intensity distribution information, spectral information, angle information of the incident light, and phase information of the incident light.
3. The optical artificial neural network smelting endpoint monitoring chip according to claim 1, wherein the smelting endpoint monitoring task further comprises identifying carbon content and/or molten steel temperature in the smelting process, and the smelting endpoint monitoring result comprises identifying results of carbon content and/or molten steel temperature in the smelting process.
4. The optical artificial neural network smelting end point monitoring chip according to any one of claims 1 to 3, wherein the optical artificial neural network smelting end point monitoring chip comprises a trained optical modulation structure, an image sensor and a processor;
the trained optical modulation structure, the image sensor and the processor are the optical modulation structure, the image sensor and the processor which meet the training convergence condition and are obtained by training an optical artificial neural network smelting endpoint monitoring chip comprising different optical modulation structures, image sensors and processors with different full-connection parameters by utilizing input training samples and output training samples corresponding to the smelting endpoint monitoring task; or the trained optical modulation structure, image sensor and processor are optical modulation structure, image sensor and processor which meet the training convergence condition and are obtained by training an optical artificial neural network smelting endpoint monitoring chip which comprises different optical modulation structures, image sensors and processors with different full connection parameters and different second nonlinear activation parameters by using input training samples and output training samples corresponding to the smelting endpoint monitoring task;
wherein the input training samples comprise incident light reflected, transmitted and/or radiated by a steelmaking taphole from smelt to finish and not smelt to finish; and the output training sample comprises a judgment result of whether smelting is completed to the end point.
5. The OPN smelting endpoint monitoring chip of claim 4, wherein when the smelting endpoint monitoring task further comprises identifying carbon content and/or molten steel temperature during smelting, the input training samples further comprise incident light reflected, transmitted and/or radiated by a steelmaking taphole smelted to different carbon content and/or molten steel temperature, and the output training samples further comprise corresponding carbon content and/or molten steel temperature, accordingly.
6. The chip according to claim 4, wherein the different light modulation structures are designed by adopting a computer optical simulation design mode when training the chip for monitoring the smelting endpoint of the artificial optical neural network, which comprises different light modulation structures, image sensors and processors with different full-connection parameters, or training the chip for monitoring the smelting endpoint of the artificial optical neural network, which comprises different light modulation structures, image sensors and processors with different full-connection parameters and different second nonlinear activation parameters.
7. The optical artificial neural network smelting endpoint monitoring chip of claim 1, wherein the optical modulation structure in the optical filter layer comprises a regular structure and/or an irregular structure; and/or the light modulating structures in the optical filter layer comprise discrete structures and/or continuous structures.
8. The chip for monitoring the smelting endpoint of the optical artificial neural network according to claim 1, wherein the optical modulation structure in the optical filter layer comprises a unit array consisting of a plurality of micro-nano units, and each micro-nano unit corresponds to one or more pixel points on an image sensor; the structures of the micro-nano units are the same or different.
9. The chip for monitoring the smelting endpoint of the optical artificial neural network according to claim 8, wherein the micro-nano unit comprises a regular structure and/or an irregular structure; and/or the micro-nano unit comprises a discrete structure and/or a continuous structure.
10. The chip for monitoring the smelting endpoint of the optical artificial neural network according to claim 8, wherein the micro-nano unit comprises a plurality of groups of micro-nano structure arrays, and the structures of the micro-nano structure arrays in each group are the same or different.
11. The chip for monitoring the smelting endpoint of the optical artificial neural network according to claim 10, wherein each group of the micro-nano structure array has the function of broadband filtering or narrowband filtering.
12. The optical artificial neural network smelting endpoint monitoring chip of claim 10, wherein each group of micro-nano structure array is a periodic structure array or a non-periodic structure array.
13. The chip for monitoring the smelting endpoint of the optical artificial neural network according to claim 10, wherein the micro-nano unit comprises one or more groups of hollow structures in a plurality of groups of micro-nano structure arrays.
14. The optical artificial neural network smelting endpoint monitoring chip of claim 10, wherein the micro-nano unit has quadruple rotational symmetry.
15. The optical artificial neural network smelting endpoint monitoring chip of claim 1, wherein the optical filter layer is composed of one or more layers;
each layer of structure is prepared by one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials and perovskite materials; and/or the filter layer is prepared from one or more of photonic crystals, super surfaces, random structures, nano structures, metal Surface Plasmon Polariton (SPP) micro-nano structures and adjustable Fabry-Perot resonant cavities.
16. The chip for monitoring the smelting endpoint of the optical artificial neural network according to claim 15, wherein the semiconductor material comprises one or more of silicon, silicon oxide, silicon nitride, titanium oxide, composite materials mixed according to a preset proportion and direct band gap compound semiconductor materials; and/or the nanostructure comprises one or more of a nanodot two-dimensional material, a nanocolumn two-dimensional material and a nanowire two-dimensional material.
17. The chip for monitoring the smelting endpoint of the optical artificial neural network as claimed in claim 1, wherein the thickness of the optical filter layer is 0.1 λ -10 λ, and λ represents the central wavelength of incident light.
18. An intelligent smelting control device, which is characterized by comprising the optical artificial neural network smelting endpoint monitoring chip as claimed in any one of claims 1 to 17.
19. The method for preparing the optical artificial neural network smelting endpoint monitoring chip according to any one of claims 1 to 17, comprising the following steps:
preparing an optical filter layer containing an optical modulation structure on the surface of the image sensor;
generating a processor with a function of performing full connection processing on the signal or generating a processor with a function of performing full connection processing and secondary nonlinear activation processing on the signal;
connecting the image sensor and the processor;
the optical filter layer is used for respectively carrying out different spectrum modulation on incident light entering different position points of the optical modulation structure through the optical modulation structure so as to obtain optical field distribution signals corresponding to the different position points on the surface of the image sensor;
the image sensor carries out first nonlinear activation processing on the light field distribution signals corresponding to the different position points after being modulated by the optical filter layer through square detection response, then converts the light field distribution signals into electric signals corresponding to the different position points, and sends the electric signals corresponding to the different position points to the processor;
and the processor performs full connection processing on the electric signals corresponding to the different position points, or performs full connection processing and secondary nonlinear activation processing on the electric signals corresponding to the different position points to obtain a smelting end point monitoring result.
20. The method for preparing an optical artificial neural network smelting endpoint monitoring chip according to claim 19, further comprising: the training process of the optical artificial neural network smelting end point monitoring chip specifically comprises the following steps:
training an optical artificial neural network smelting endpoint monitoring chip comprising different optical modulation structures, image sensors and processors with different full-connection parameters by using input training samples and output training samples corresponding to the smelting endpoint monitoring task to obtain the optical modulation structures, the image sensors and the processors meeting the training convergence conditions, and taking the optical modulation structures, the image sensors and the processors meeting the training convergence conditions as the trained optical modulation structures, the image sensors and the processors;
or training an optical artificial neural network smelting endpoint monitoring chip comprising different optical modulation structures, image sensors and processors with different full-connection parameters and different second nonlinear activation parameters by using input training samples and output training samples corresponding to the smelting endpoint monitoring task to obtain the optical modulation structures, the image sensors and the processors meeting the training convergence conditions, and taking the optical modulation structures, the image sensors and the processors meeting the training convergence conditions as the trained optical modulation structures, the trained image sensors and the trained processors.
21. The method for preparing an optical artificial neural network smelting endpoint monitoring chip according to claim 19, wherein preparing an optical filter layer containing an optical modulation structure on the surface of the image sensor comprises:
growing one or more layers of preset materials on the surface of the image sensor;
etching the light modulation structure pattern of the one or more layers of preset materials to obtain an optical filter layer containing a light modulation structure;
or carrying out impression transfer on the one or more layers of preset materials to obtain an optical filter layer containing an optical modulation structure;
or the one or more layers of preset materials are subjected to additional dynamic modulation to obtain an optical filter layer containing an optical modulation structure;
or printing the one or more layers of preset materials in a partition mode to obtain an optical filter layer containing an optical modulation structure;
or carrying out partition growth on the one or more layers of preset materials to obtain an optical filter layer containing an optical modulation structure;
or quantum dot transfer is carried out on the one or more layers of preset materials to obtain the optical filter layer containing the optical modulation structure.
CN202110172852.9A 2021-02-08 2021-02-08 Optical artificial neural network smelting end point monitoring chip and preparation method thereof Pending CN114912602A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110172852.9A CN114912602A (en) 2021-02-08 2021-02-08 Optical artificial neural network smelting end point monitoring chip and preparation method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110172852.9A CN114912602A (en) 2021-02-08 2021-02-08 Optical artificial neural network smelting end point monitoring chip and preparation method thereof

Publications (1)

Publication Number Publication Date
CN114912602A true CN114912602A (en) 2022-08-16

Family

ID=82761134

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110172852.9A Pending CN114912602A (en) 2021-02-08 2021-02-08 Optical artificial neural network smelting end point monitoring chip and preparation method thereof

Country Status (1)

Country Link
CN (1) CN114912602A (en)

Similar Documents

Publication Publication Date Title
WO2022166189A1 (en) Optical artificial neural network smart chip and manufacturing method
US9000353B2 (en) Light absorption and filtering properties of vertically oriented semiconductor nano wires
EP3567416A1 (en) Photonic crystal spectrometer
WO2022032842A1 (en) Miniature spectrum chip based on units in random shapes
WO2022166188A1 (en) Optical artificial neural network smart chip, smart processing device, and manufacturing method
KR20150067141A (en) Multispectral imaging using silicon nanowires
CN113497065B (en) Imaging spectrum chip with spectrum and imaging functions and preparation method thereof
CN113447118B (en) Multispectral imaging chip capable of realizing color imaging and color imaging method
TWI798940B (en) Spectral imaging chip and information processing method thereof
CN101430268B (en) Integral inverse-calculation method for grain diameter measurement
CN111811651B (en) Spectrum chip, spectrometer and spectrum chip preparation method
CN114518168A (en) Spectral imaging chip, preparation method thereof and information processing method
WO2020122038A1 (en) Solid-state imaging element, method for producing solid-state imaging element, and electronic device
CN114912602A (en) Optical artificial neural network smelting end point monitoring chip and preparation method thereof
CN109323661A (en) Based on the highly sensitive angle displacement sensor of light beam space Gu Si-Hansen displacement
CN114943320A (en) Smelting end point monitoring chip, intelligent smelting control equipment and preparation method
WO2022032843A1 (en) Micro spectral chip based on units of different shapes
CN108831898A (en) A kind of solid-state multispectral sensor
CN114912601A (en) Optical artificial neural network environment-friendly monitoring chip and preparation method thereof
Crozier et al. Controlling the light absorption in a photodetector via nanowire waveguide resonances for multispectral and color imaging
US11158661B2 (en) Image sensor with micro-structured color filter
CN114912603A (en) Optical artificial neural network enhanced machine vision chip and preparation method thereof
Puiu Color sensors and their applications
WO2022033353A1 (en) Miniature spectrum chip based on units of different shapes, and method for generating micro-nano structure array in miniature spectrum chip
CN114912600A (en) Intelligent agricultural accurate control chip of optical artificial neural network and preparation method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination