CN110097182B - Three-dimensional Hopfield neural network model realization circuit controlled by nerve activation gradient lambda - Google Patents
Three-dimensional Hopfield neural network model realization circuit controlled by nerve activation gradient lambda Download PDFInfo
- Publication number
- CN110097182B CN110097182B CN201910283951.7A CN201910283951A CN110097182B CN 110097182 B CN110097182 B CN 110097182B CN 201910283951 A CN201910283951 A CN 201910283951A CN 110097182 B CN110097182 B CN 110097182B
- Authority
- CN
- China
- Prior art keywords
- operational amplifier
- circuit
- neural network
- output
- resistor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000004913 activation Effects 0.000 title claims abstract description 89
- 238000003062 neural network model Methods 0.000 title claims abstract description 18
- 210000005036 nerve Anatomy 0.000 title description 4
- 238000013528 artificial neural network Methods 0.000 claims abstract description 76
- 230000001537 neural effect Effects 0.000 claims abstract description 43
- 210000002569 neuron Anatomy 0.000 claims description 54
- 230000010354 integration Effects 0.000 claims description 27
- 239000003990 capacitor Substances 0.000 claims description 19
- 230000004044 response Effects 0.000 claims description 10
- 230000000694 effects Effects 0.000 claims description 7
- 230000000946 synaptic effect Effects 0.000 claims description 7
- 230000005674 electromagnetic induction Effects 0.000 claims description 5
- 239000012528 membrane Substances 0.000 claims description 2
- 238000000034 method Methods 0.000 claims description 2
- 230000005611 electricity Effects 0.000 claims 1
- 230000006399 behavior Effects 0.000 description 16
- 238000010586 diagram Methods 0.000 description 11
- 238000004088 simulation Methods 0.000 description 10
- 238000012795 verification Methods 0.000 description 9
- 230000000638 stimulation Effects 0.000 description 7
- 230000008859 change Effects 0.000 description 3
- 230000000739 chaotic effect Effects 0.000 description 3
- 210000004556 brain Anatomy 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000001747 exhibiting effect Effects 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000010587 phase diagram Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003071 parasitic effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Neurology (AREA)
- Feedback Control In General (AREA)
Abstract
The invention provides a three-dimensional Hopfield neural network model realization circuit controlled by a neural activation gradient lambda, which comprises a negative output hyperbolic tangent function realization circuit based on an activation gradient and a three-dimensional Hopfield neural network main circuit. The invention provides a circuit for realizing a three-dimensional Hopfield neural network model controlled by a neural activation gradient lambda, which is based on a conventional three-dimensional Hopfield neural network, introduces the neural activation gradient into a hyperbolic tangent function as a controlled quantity lambda of the model to form an activation function tanh (lambda x) based on the neural activation gradient, the activation function is used as a module of the neural network, and the dynamic control of the three-dimensional Hopfield neural network can be realized by the neural activation gradient lambda.
Description
Technical Field
The invention relates to the technical field of neural network control, in particular to a three-dimensional Hopfield neural network model realization circuit controlled by a neural activation gradient lambda.
Background
The Hopfield Neural Network (HNN) composed of neurons is an extremely important model in artificial neural networks. In the past few years, a great deal of control over HNN dynamics has been reported by varying synaptic weights between different neurons. The hyperbolic function is a nonlinear function, which can be used as an activation function of a neuron to simulate the electrical activity behavior of the neuron, the slope of the hyperbolic function, namely the response speed of the electrical activity of the neuron, can be changed by changing the activation gradient lambda, and the control of the Hopfield neural network by using the neural activation gradient lambda is not researched yet. Therefore, it is necessary and meaningful to further study the dynamics of this neural network, especially the Hopfield neural network controlled by the neural activation gradient λ.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: in order to overcome the defects in the prior art, the invention provides a three-dimensional Hopfield neural network model realizing circuit controlled by a neural activation gradient lambda.
The technical scheme adopted for solving the technical problems is as follows: a three-dimensional Hopfield neural network model realization circuit controlled by a neural activation gradient lambda comprises a negative output hyperbolic tangent function realization circuit based on an activation gradient and a three-dimensional Hopfield neural network main circuit. Based on a conventional three-dimensional Hopfield neural network, the invention introduces a neural activation gradient in a hyperbolic tangent function as a control quantity lambda of a model to form an activation function-tanh (lambda x) based on negative output of the neural activation gradient, and the activation function is used as a module of the neural network and influences the dynamics of the whole neural network, so that the dynamics control of the three-dimensional Hopfield neural network can be realized through the neural activation gradient lambda.
A conventional three-dimensional Hopfield neural network model can be expressed as:
in the conventional neural network model (1), typical values of a and b are given, i.e., a =0.7, b = -2. A neural activation gradient is introduced in the activation function tanh (x) as a control quantity λ of the model, which may represent a response speed of the neuron after stimulation. The neural activation gradient thus constitutes an activation function tanh (λ x) based on the neural activation gradient control quantity, which acts as a module of the neural network and which influences the dynamic behavior of the entire neural network.
On the basis of a conventional three-dimensional Hopfield neural network model, introducing a neural activation gradient function lambda as a control variable, wherein the neural network model is expressed by a first-order ordinary differential equation system as follows:
in the formula，x 1 、x 2 And x 3 Three state variables of the neuron are respectively; the parameter a is the synaptic weight connecting the third neuron with the first neuron, and the parameter b is the synaptic weight connecting the second neuron with the first neuron, and is usually set as a =0.7 and b = -2 respectively; the control variable lambda is the activation gradient of the neuron and represents the response speed of the electrical activity of the neuron under electromagnetic induction; notably, within a reasonable range of the control variable λ (0.7)<λ<1.5 Model (1) has one zero equilibrium point and two non-zero equilibrium points, which exhibit different states with increasing λ, exhibiting complex dynamic behavior.
The three equations in the equation (2) are respectively realized by adopting an integration channel one, an integration channel two and an integration channel three, the three integration channels are used as main circuits of the three-dimensional Hopfield neural network, typical values of a and b are given, namely a =0.7, b = -2, and according to the Kirchoff circuit law and the electrical characteristics of circuit components, the circuit equation corresponding to the equation (2) can be expressed as follows:
in the formula (3), v 1 、v 2 And v 3 Are three circuit variables that correspond to x in a three-dimensional Hopfield neural network system 1 、x 2 And x 3 Representing the membrane potential of three neurons in the neural network; the nonlinear function can be used as an activation function of a neural network and can be represented by a monotonous and slightly bounded function, and the hyperbolic tangent function is the activation function in the system and represents the state change of a neuron after being stimulated by the outside; the control variable λ is the gradient of activation of the neuron, representing the speed of response of the electrical activity of the neuron under electromagnetic induction. When lambda is>1, it means that the neuron responds rapidly after being subjected to external stimulation; when λ =1, it represents a normal response of the neuron after being subjected to an external stimulus; when lambda is<1, indicates that the neuron responds slowly to an external stimulus.
v a 、v b 、v c 、–v a And-v b The input ends are connected with different integral channels and also used as feedback ports of the neural network; v. of 1 、v 2 And v 3 Are internal outputs of the neural network and external outputs under external stimulus, which can be connected to different channels of an oscilloscope for observation. If some neuron in the neural network is subjected to external stimulation, that is, is interfered by the induced current, a resistor R is added to transmit current before an integration channel realized by a certain neuron circuit, and the input end of the resistor R is connected with a certain function/value of the external stimulation, which is the external input end of the neural network. For example, if the first neuron is stimulated by a DC voltage of 1V, then the integrating channel of the first neuron will be followed by an R 0 A resistor of 10k omega, a DC power supply V with 1V input connected in front of the resistor DC The output is v 1 The change is not changed; for another example, if the second neuron is stimulated with an alternating current of 2V amplitude and 60Hz frequency, then an R signal will follow the integration channel of the second neuron 0 A resistor of 10k omega, an AC voltage source V with 2V input and 60Hz frequency is connected in front of the resistor AC The output is v 2 And is not changed. When the neural networks are not stimulated by external stimuli, they are interacted by each neuron, namely, the three-dimensional Hopfield neural network researched now, and the network can be controlled by the neural activation gradient lambda, so that different forms of dynamic behaviors are generated and the state response of the human brain is simulated. The kinetic behavior after the addition of external stimuli is the focus of our next study.
The realization circuit of the first integral channel comprises an input end v inside a neural network a 、v b 、v c Operational amplifier U 1 And U 2 And the activation gradient tanh (λ x) function achieves the circuit Tg 1 Input v within the neural network a 、v b 、v c Respectively connected with resistors R in series 1 、R 2 And R 3 Is connected to an operational amplifier U 1 The inverting input terminal of (1); operational amplifier U 1 Between the inverting input terminal and the output terminal of the first transistor, a capacitor C of 10nF and a capacitor C of 10k omega are connected in parallel"resistance R. Because the tanh module in the neural network circuit should be a hyperbolic tangent function with a negative output, a hyperbolic tangent function implementation circuit with a negative output is designed in the circuit design. Operational amplifier U 1 Voltage v output from the output terminal 1 Realization of circuit Tg by means of activation gradient tanh (lambda x) function 1 Then obtaining an output voltage v a (ii) a Output voltage v a Is connected with an operational amplifier U after being connected with a resistor R of 10k omega in series 2 The inverting input terminal of (1); operational amplifier U 2 A 10k omega resistor R is connected in parallel between the inverting input end and the output end of the operational amplifier U 2 Output of-v a Output-v a As an input signal of the integrating channel two implementation circuit; operational amplifier U 1 And U 2 The non-inverting input ends of the two-way switch are grounded;
the second integration channel realization circuit comprises an input end-v inside the neural network a 、v b Operational amplifier U 3 And U 4 And the activation gradient tanh (λ x) function achieves the circuit Tg 2 Input terminal-v a And v b Respectively connected with resistors R in series 4 And R 5 Connected with an operational amplifier U 3 The inverting input terminal of (a); operational amplifier U 3 A capacitor of 10nF and a resistor R of 10k omega are connected in parallel between the inverting input end and the output end of the capacitor; operational amplifier U 3 Voltage v output from the output terminal 2 Realization of circuit Tg by means of activation gradient tanh (lambda x) function 2 Then obtaining an output voltage v b (ii) a Output voltage v b A 10k omega resistor R is connected in series with an operational amplifier U 4 The inverting input terminal of (a); operational amplifier U 4 A 10k omega resistor R is connected in parallel between the inverting input end and the output end of the operational amplifier U 4 Is output to obtain an output-v b Output-v b As an input signal of the integration channel three implementation circuit; operational amplifier U 3 And U 4 The non-inverting input ends of the two-way switch are grounded;
the third integration channel realization circuit comprises an input end v inside the neural network a 、-v b 、v c Transporting and transportingComputing amplifier U 5 And the activation gradient tanh (λ x) function achieves the circuit Tg 3 Input terminal v a 、–v b And v c Respectively connected with resistors R in series 6 、R 7 And R 8 Connected with an operational amplifier U 5 The inverting input terminal of (1); operational amplifier U 5 A capacitor C of 10nF and a resistor R of 10k omega are connected in parallel between the inverting input end and the output end of the capacitor; operational amplifier U 5 Voltage v output from the output terminal 3 Realization of circuit Tg by means of activation gradient tanh (lambda x) function 3 Then obtaining an output voltage v c (ii) a Output voltage v c Feeding back the signal as an input signal to an integrating channel I realizing circuit; operational amplifier U 5 The non-inverting input terminals of the two-way switch are all connected with the ground.
The activation gradient tanh (λ x) function implementation circuit comprises: integrator, triode T 1 And T 2 DC voltage source E and DC current source I 0 Wherein the integrator comprises an operational amplifier U o And U i The concrete connection mode is as follows: input terminal v i A resistor R of 10k omega is connected in series and then connected with an operational amplifier U i The inverting input terminal of (a); operational amplifier U i Between the inverting input and the output, a variable resistor R is connected in parallel F (ii) a Operational amplifier U i The output end of the transistor is connected with a triode T 1 A base electrode of (1); triode T 1 The collector of the circuit is divided into two paths, one path is connected to an operational amplifier U through a resistor R of 10k omega o The other path of the input voltage is connected with a resistor R of 10k omega C Is connected to a direct current voltage source E; triode T 2 Emitter and triode T 1 Is connected to and is simultaneously connected to a direct current source I 0 In this embodiment, the current value of the dc current source is "1.19mA". Triode T 2 The collector of which is also connected with a resistor R C Is connected to a DC voltage source E, and the other end is transversely connected with a 10k omega resistor R to an operational amplifier U o The inverting input terminal of (1); triode T 2 The base electrode of the operational amplifier is grounded, and then is transversely connected with a 10k omega resistor R to an operational amplifier U o The same direction input end of the input terminal. Operational amplifier U o Is transmitted in reverse phaseThe input end is connected with a resistor R of 10k omega to the output end v o 。
The direct-current voltage source E is a fixed 15V direct-current power supply and controls a resistor R of a neural network F The adjustable resistor is adjustable, the adjustable range is 500-900 omega, and the dynamic behavior of the neural network can be directly controlled.
The neural activation gradient λ may reflect the response speed of the electrical activity of the neuron under electromagnetic induction stimulation. The invention is based on a three-dimensional Hopfield neural network model, and introduces a neural activation gradient function lambda as a control variable in order to better realize the control of the Hopfield neural network. The invention therefore proposes a Hopfield neural network controlled with a neural activation gradient lambda. The dynamic behavior of the Hopfield neural network controlled by the neuron activation gradient lambda is researched by taking the neuron activation gradient lambda as an adjustable control parameter and through stability analysis based on balance points, numerical analysis of a mathematical model and hardware experimental verification. The results show that complex kinetic behavior occurs in the HNN model. The proposed neural network system is designed and experimentally verified, and the experimental result is found to be well matched with the numerical simulation result.
The invention has the beneficial effects that: the invention provides a circuit for realizing a three-dimensional Hopfield neural network model controlled by a neural activation gradient lambda, and a Hopfield neural network equivalent circuit with complex dynamic behaviors is realized. The circuit structure is clear, the used components are simple and available, and theoretical analysis and circuit integration are easy to realize. The circuit is controlled by the nerve activation gradient lambda, and complex dynamic behaviors can be generated by only changing one variable resistance value, so that the circuit has a great value for artificial neural network research in engineering application.
Drawings
The invention is further illustrated by the following figures and examples.
FIG. 1 is a circuit implemented with a three-dimensional Hopfield neural network system model controlled by a neural activation gradient λ;
FIG. 2 is a circuit for implementing a hyperbolic function tanh (λ x) based on a neural activation gradient λ;
FIG. 3 is a diagram of modulating neuro-excitationsActive gradient λ =0.93, x 1 –x 3 MATLAB numerical simulation phase track diagram and experimental verification result on the plane;
FIG. 4 is a graph of modulated neural activation gradient λ =1, x 1 –x 3 MATLAB numerical simulation phase track diagram and experimental verification result on the plane;
FIG. 5 is a graph of modulated neural activation gradient λ =1.1, x 1 –x 3 MATLAB numerical simulation phase track diagram and experimental verification result on the plane;
fig. 6 is x when the neural activation gradient λ =1.2 is modulated 1 –x 3 MATLAB numerical simulation phase track diagram and experimental verification result on the plane;
FIG. 7 is a graph of modulated neural activation gradient λ =1.3, x 1 –x 3 MATLAB numerical simulation phase orbit diagram and experimental verification result on the plane;
fig. 8 is a graph of x when the neural activation gradient λ =1.5 is modulated 1 –x 3 MATLAB numerical simulation phase track diagram and experimental verification result on the plane;
Detailed Description
The present invention will now be described in detail with reference to the accompanying drawings. This figure is a simplified schematic diagram, and only illustrates the basic structure of the present invention in a schematic manner, and therefore it only shows the constitution related to the present invention.
As shown in fig. 1 and 2, the circuit includes: a hyperbolic function tanh (lambdx) realization circuit diagram 2 based on an activation gradient lambda and a three-dimensional Hopfield neural network realization circuit diagram 1; the hyperbolic function circuit tanh (lambdx) of fig. 2 is introduced into a three-dimensional Hopfield neural network realization circuit to form a novel Hopfield neural network verification circuit controlled by a neural activation gradient lambada, and after the same ports of fig. 1 and fig. 2 are connected in sequence, the non-inverting input ends of operational amplifiers are all connected with the ground, so that complex dynamic behaviors can be presented.
The hyperbolic function circuit based on the activation gradient comprises: integrator, triode, direct current voltage source and direct current source etc.. The concrete connection mode is as follows: input terminal "v i A 10k omega resistor is connected in series and then connected with an operational amplifier U i The inverting input terminal of (1); operationsAmplifier U i Is connected in parallel with a changeable R between the inverting input terminal and the output terminal F A resistance; u shape i The output end of the transistor is connected with a triode T 1 A base electrode of (1); triode T 1 The collector is transversely connected with a resistor of 10k omega and then connected with a resistor R of 10k omega C ；R C The other end of the resistor is connected to a small amplitude dc voltage source E, preferably "15V" in this embodiment. Triode T 2 Emitter and triode T 1 Is connected to and is simultaneously connected to a direct current source I 0 Its value is "1.19mA". Triode T 2 Is also connected with an R C The resistor is connected to a DC voltage source, and the other end is transversely connected with a 10k omega resistor R to an operational amplifier U 0 The inverting input terminal of (1); triode T 2 The base electrode of the operational amplifier is grounded, and then is transversely connected with a 10k omega resistor R to an operational amplifier U 0 The same direction input end of the input terminal. Operational amplifier U 0 The inverting input of the resistor is connected with a resistor R of 10k omega to the output end v o 。
The three-dimensional Hopfield neural network system model realization circuit comprises an integration channel I, an integration channel II and an integration channel III.
v a 、v b 、v c 、–v a And-v b The input ends are connected with different integral channels and also used as feedback ports of the neural network; v. of 1 、v 2 And v 3 Are internal outputs of the neural network and external outputs under external stimulus, which can be connected to different channels of an oscilloscope for observation. If some neuron in the neural network is subjected to external stimulation, that is, is interfered by the induced current, a resistor R is added to transmit current before an integration channel realized by a certain neuron circuit, and the input end of the resistor R is connected with a certain function/value of the external stimulation, which is the external input end of the neural network. For example, if the first neuron is stimulated by a DC voltage of 1V, then the integrating channel of the first neuron will be followed by an R 0 A resistor of 10k omega, a DC power supply V with 1V input connected in front of the resistor DC The output is v 1 The change is not changed; for another example, if the second neuron is stimulated with an alternating current of 2V amplitude and 60Hz frequency, then an R signal will follow the integration channel of the second neuron 0 A resistor of 10k omega, an AC voltage source V with 2V input and 60Hz frequency is connected in front of the resistor AC The output is v 2 And is not changed. When the neural networks are not stimulated by external stimuli, they are interacted by each neuron, namely, the three-dimensional Hopfield neural network researched now, and the network can be controlled by the neural activation gradient lambda, so that different forms of dynamic behaviors are generated and the state response of the human brain is simulated. The kinetic behavior after the addition of external stimuli is the focus of our next study.
The realization circuit of the first integral channel comprises an input end v inside a neural network a 、v b 、v c Operational amplifier U 1 And U 2 And the activation gradient tanh (λ x) function achieves the circuit Tg 1 Input v within the neural network a 、v b 、v c Respectively connected with resistors R in series 1 、R 2 And R 3 Is connected to an operational amplifier U 1 The inverting input terminal of (1); operational amplifier U 1 A capacitor C of "10nF" and a resistor R of "10k Ω" are connected in parallel between the inverting input terminal and the output terminal of the capacitor C. Because the tanh module in the neural network circuit should be a hyperbolic tangent function with a negative output, a hyperbolic tangent function implementation circuit with a negative output is designed in the circuit design. Operational amplifier U 1 Voltage v output from the output terminal 1 Realization of circuit Tg via activation gradient tanh (λ x) function 1 Then obtaining an output voltage v a (ii) a Output voltage v a Is connected with an operational amplifier U after being connected with a resistor R of 10k omega in series 2 The inverting input terminal of (1); operational amplifier U 2 A 10k omega resistor R is connected in parallel between the inverting input end and the output end of the operational amplifier U 2 Output of-v a Output-v a As an input signal of the integration channel two implementation circuit; operational amplifier U 1 And U 2 The non-inverting input ends of the two-way switch are grounded;
realization circuit of integration channel twoComprising inputs-v within a neural network a 、v b Operational amplifier U 3 And U 4 And the activation gradient tanh (λ x) function achieves the circuit Tg 2 Input terminal-v a And v b Respectively connected with resistors R in series 4 And R 5 Connected with an operational amplifier U 3 The inverting input terminal of (1); operational amplifier U 3 A capacitor of 10nF and a resistor R of 10k omega are connected in parallel between the inverting input end and the output end of the capacitor; operational amplifier U 3 Voltage v output from the output terminal 2 Realization of circuit Tg via activation gradient tanh (λ x) function 2 Then obtaining an output voltage v b (ii) a Output voltage v b A 10k omega resistor R is connected in series with an operational amplifier U 4 The inverting input terminal of (1); operational amplifier U 4 A 10k omega resistor R is connected in parallel between the inverting input end and the output end of the operational amplifier U 4 Is output to obtain an output-v b Output-v b As an input signal of the integration channel three implementation circuit; operational amplifier U 3 And U 4 The non-inverting input ends of the two-way switch are grounded;
the third integration channel realization circuit comprises an input end v inside the neural network a 、-v b 、v c Operational amplifier U 5 And the activation gradient tanh (λ x) function achieves the circuit Tg 3 Input terminal v a 、–v b And v c Respectively connected with resistors R in series 6 、R 7 And R 8 Connected with an operational amplifier U 5 The inverting input terminal of (1); operational amplifier U 5 A capacitor C of 10nF and a resistor R of 10k omega are connected in parallel between the inverting input end and the output end of the capacitor; operational amplifier U 5 Voltage v output from the output terminal 3 Realization of circuit Tg by means of activation gradient tanh (lambda x) function 3 Then obtaining an output voltage v c (ii) a Output voltage v c Feeding back the signal as an input signal to an integrating channel I realizing circuit; operational amplifier U 5 The non-inverting input terminals of the two-way switch are all connected with the ground.
The Hopfield neural network circuit controlled by the neural activation gradient lambda is shown in figure 1, and the system side thereofThe program contains three state variables x 1 、x 2 And x 3 (ii) a Three variables v corresponding to the state equation of the circuit 1 、v 2 And v 3 。
Mathematical modeling: a circuit implementation of the Hopfield neural network controlled by the neural activation gradient λ of the present embodiment is constructed as shown in fig. 1. The invention is based on a three-dimensional Hopfield neural network, and introduces a neural activation gradient function lambda as a control variable in order to better realize the control of the Hopfield neural network. For ease of analysis and circuit implementation, the model can be described as a system of first order ordinary differential equations:
wherein x is 1 、x 2 And x 3 Three state variables of the neuron, respectively. The parameter a is the synaptic weight connecting the third neuron to the first neuron, and the parameter b is the synaptic weight connecting the second neuron to the first neuron. Typically set to a =0.7, b = -2, respectively. The control variable λ is the activation gradient of the neuron, representing the response speed of the electrical activity of the neuron under electromagnetic induction. Notably, within a reasonable range of the control variable λ (0.7)<λ<1.5 Model (1) has one zero equilibrium point and two non-zero equilibrium points, which exhibit different states with increasing λ, exhibiting complex dynamic behavior.
Numerical simulation: when the neural activation gradient λ is used as a parameter of the system, the MATLAB ODE23 algorithm was used to perform a numerical study on the kinetic behavior of the Hopfield neural network controlled with the neural activation gradient λ at two sets of initial values (0, 1, 0) and (0, -1, 0). When the control variables λ =0.93, λ =1, (a) in fig. 3 and (a) in fig. 4 depict two types of coexisting symmetric attractors at x 1 -x 3 Phase traces on the plane, which correspond to the coexisting upper and lower attractors at initial values (0, 1, 0) and (0, -1, 0), respectively, wherein (a) in fig. 3 shows the coexisting period 4 state and (a) in fig. 4 shows the coexisting chaotic helical attractors. When the controlled variable lambda =1.1,λ =1.2, at initial values (0, 1, 0) and (0, -1, 0), corresponding to the lower and upper attractors, respectively, (a) in fig. 5 and (a) in fig. 6 depict two types of coexisting symmetric attractors at x 1 -x 3 Phase diagram on a plane. When the variable λ =1.1, (a) in fig. 5 exhibits a coexisting helical chaotic attractor, and when the variable λ =1.2 is controlled, (a) in fig. 6 exhibits a coexisting multi-cycle state. Under the condition of an initial value of (0, 1, 0), (a) in fig. 7 exhibits a chaotic two-wrap state when the control variable λ = 1.3; when the control variable λ =1.5, in (a) in fig. 8, the system evolves into the cycle 1 limit cycle.
In fig. 2, the main circuit of the three-dimensional Hopfield neural network has three integration channels for implementing the first, second and third equations of equation (1). The circuit equation shown in FIG. 1 can be written as follows from the kirchhoff's law of circuits and the electrical characteristics of the circuit components
Wherein v is 1 、v 2 And v 3 Is a three circuit variable, v a 、v b And v c Is the output voltage of the circuit variable after passing through a gradient-based hyperbolic tangent function tanh (lambda x) — v a 、–v b And-v c Is the voltage value of the output voltage after passing through the inverting amplifier.
Taking the time accuracy as 0.1ms, i.e., R =10k Ω, C =10nF, by comparing expression (1) and expression (2), it is possible to obtain
Therefore, the invention constructs a Hopfield neural network controlled by a neural activation gradient lambda and an implementation scheme thereof.
And (3) experimental verification: the discrete device adopts TL082CP operational amplifier with the supply voltage of +/-15V, and the discrete elements adopt MPS2222 triode, metal film resistor, precise adjustable resistor anda monolithic capacitor. In the experimental process, a Take TDS 3054C digital fluorescence oscilloscope is used for testing the experimental result. When the nerve activation gradient control variable lambda =0.93 is adjusted, the resistance R can be adjusted F =450 Ω, trapped at v 1 –v 3 The phase diagram on the plane is shown in fig. 3 (b). Increasing the activation gradient λ value of the neuron, i.e. when λ =1, λ =1.1, λ =1.2, λ =1.3 and λ =1.5, in actual operation the adjustable resistance value corresponds to R respectively F ＝511Ω、R F ＝567Ω、R F ＝600Ω、R F =720 Ω and R F =850 Ω. Captured at v 1 –v 3 The phase rail diagram on the plane is shown in fig. 4 (b), fig. 5 (b), fig. 6 (b), fig. 7 (b), and fig. 8 (b). Neglecting some minor differences between the numerical simulation and the hardware circuit experiment caused by computational errors and parasitic circuit parameters, the experimental results are almost identical to the numerical simulation, which indicates that the proposed complex dynamics behavior formed by the Hopfield neural network controlled by the neural activation gradient lambda can be verified by experiments. Therefore, the Hopfield neural network controlled by the neural activation gradient lambda constructed by the invention has scientific theoretical basis and physical realizability, and can play a positive role in promoting the engineering application of neuron models and artificial neural networks.
In light of the foregoing description of preferred embodiments in accordance with the invention, it is to be understood that numerous changes and modifications may be made by those skilled in the art without departing from the scope of the invention. The technical scope of the present invention is not limited to the contents of the specification, and must be determined according to the scope of the claims.
Claims (3)
1. A three-dimensional Hopfield neural network model realization circuit controlled by a neural activation gradient lambda comprises a negative output hyperbolic tangent function realization circuit tanh module based on the activation gradient and a three-dimensional Hopfield neural network main circuit;
introducing a neural activation gradient function lambda as a control variable on the basis of a conventional three-dimensional Hopfield neural network model, wherein the neural network model is expressed by a first-order ordinary differential equation system as follows:
in the formula, x 1 、x 2 And x 3 Three state variables of the neuron are respectively; the parameter a is the synaptic weight connecting the third neuron with the first neuron, and the parameter b is the synaptic weight connecting the second neuron with the first neuron; the control variable lambda is the activation gradient of the neuron and represents the response speed of the electrical activity of the neuron under electromagnetic induction;
the three equations in the formula (2) are respectively realized by adopting an integration channel I, an integration channel II and an integration channel III, the three integration channels are used as a main circuit of the three-dimensional Hopfield neural network, and according to the kirchhoff circuit law and the electrical characteristics of circuit components, the circuit equation corresponding to the formula (2) can be expressed as follows:
in the formula (3), v 1 、v 2 And v 3 Are three circuit variables that correspond to x in a three-dimensional Hopfield neural network system 1 、x 2 And x 3 Representing the membrane potential of three neurons in the neural network;
the method is characterized in that: the realization circuit of the first integral channel comprises an input end v inside a neural network a 、v b 、v c Operational amplifier U 1 And U 2 And the activation gradient tanh (λ x) function achieves the circuit Tg 1 Input v inside the neural network a 、v b 、v c Respectively connected with resistors R in series 1 、R 2 And R 3 Is connected to an operational amplifier U 1 The inverting input terminal of (1); operational amplifier U 1 A capacitor C and a resistor R are connected in parallel between the inverting input end and the output end of the capacitor; operational amplifier U 1 Voltage output from the output terminalv 1 Realization of circuit Tg by means of activation gradient tanh (lambda x) function 1 Then obtaining an output voltage v a (ii) a Output voltage v a Is connected in series with a resistor R and then is connected to an operational amplifier U 2 The inverting input terminal of (1); operational amplifier U 2 Between the inverting input terminal and the output terminal of the operational amplifier U and a resistor R connected in parallel 2 Output of-v a Output-v a As an input signal of the integrating channel two implementation circuit; operational amplifier U 1 And U 2 The non-inverting input ends of the two-way switch are grounded;
the second integration channel realization circuit comprises an input end-v inside the neural network a 、v b Operational amplifier U 3 And U 4 And the activation gradient tanh (λ x) function implements the circuit Tg 2 Input terminal-v a And v b Respectively connected with resistors R in series 4 And R 5 Connected with an operational amplifier U 3 The inverting input terminal of (1); operational amplifier U 3 A capacitor C and a resistor R are connected in parallel between the inverting input end and the output end of the capacitor; operational amplifier U 3 Voltage v output from the output terminal 2 Voltage v of output 2 Realization of circuit Tg by means of activation gradient tanh (lambda x) function 2 Then obtaining an output voltage v b (ii) a Output voltage v b A resistor R is connected in series with an operational amplifier U 4 The inverting input terminal of (1); operational amplifier U 4 Between the inverting input terminal and the output terminal of the operational amplifier U and a resistor R connected in parallel 4 Is output to obtain an output-v b Output-v b As an input signal of the integration channel three implementation circuit; operational amplifier U 3 And U 4 The non-inverting input ends of the two-way switch are grounded;
the third integration channel realization circuit comprises an input end v inside the neural network a 、-v b 、v c Operational amplifier U 5 And the activation gradient tanh (λ x) function achieves the circuit Tg 3 Input terminal v a 、–v b And v c Respectively connected with resistors R in series 6 、R 7 And R 8 Connected with an operational amplifier U 5 The inverting input terminal of (1); operational amplifier U 5 A capacitor C and a resistor R are connected in parallel between the inverting input end and the output end of the capacitor; operational amplifier U 5 Voltage v output from the output terminal 3 Voltage v 3 Realization of circuit Tg by means of activation gradient tanh (lambda x) function 3 Then obtaining an output voltage v c (ii) a Output voltage v c As an input signal fed back to an integrating channel-implementing circuit, operational amplifier U 5 The non-inverting input terminals are all connected with the ground.
2. The three-dimensional Hopfield neural network model implementation circuit with neural activation gradient λ control of claim 1, wherein: the activation gradient tanh (λ x) function implementation circuit comprises: integrator, triode T 1 And T 2 DC voltage source E and DC current source I 0 Wherein the integrator comprises an operational amplifier U i And U o The concrete connection mode is as follows: input terminal v i Connected in series with a resistor R and then connected with an operational amplifier U i The inverting input terminal of (1); operational amplifier U i Between the inverting input and the output, a variable resistor R is connected in parallel F (ii) a Operational amplifier U i The output end of the transistor is connected with a triode T 1 A base electrode of (1); triode T 1 The collector is divided into two paths, one path is connected to an operational amplifier U through a resistor R o The other path of the input voltage is connected with a resistor R C Is connected to a direct current voltage source E; triode T 2 Emitter and triode T 1 Is connected to and is simultaneously connected to a direct current source I 0 (ii) a Triode T 2 The collector of which is also connected with a resistor R C Is connected to a DC voltage source E, and the other end is connected to a resistor R to an operational amplifier U o The inverting input terminal of (1); triode T 2 The base electrode of the triode T is grounded 2 The base electrode is connected with a resistor R in series to an operational amplifier U o The same-direction input end of the input end; operational amplifier U o Is connected with a resistor R to an output terminal v o 。
3. The implementation of electricity with the three-dimensional Hopfield neural network model of neural activation gradient lambda control as set forth in claim 2The road, its characterized in that: the direct-current voltage source E is a fixed 15V direct-current power supply and controls a resistor R of a neural network F The adjustable resistor is adjustable, the adjustable range is 500-900 omega, and the dynamic behavior of the neural network can be directly controlled.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910283951.7A CN110097182B (en) | 2019-04-10 | 2019-04-10 | Three-dimensional Hopfield neural network model realization circuit controlled by nerve activation gradient lambda |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910283951.7A CN110097182B (en) | 2019-04-10 | 2019-04-10 | Three-dimensional Hopfield neural network model realization circuit controlled by nerve activation gradient lambda |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110097182A CN110097182A (en) | 2019-08-06 |
| CN110097182B true CN110097182B (en) | 2023-03-24 |
Family
ID=67444519
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910283951.7A Active CN110097182B (en) | 2019-04-10 | 2019-04-10 | Three-dimensional Hopfield neural network model realization circuit controlled by nerve activation gradient lambda |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110097182B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113112010B (en) * | 2021-04-29 | 2022-09-27 | 齐鲁工业大学 | Nerve fiber equivalent circuit supporting soliton wave conduction |
| CN113379044B (en) * | 2021-05-21 | 2023-05-02 | 长沙理工大学 | Image encryption method of Hopfield neural network based on electromagnetic radiation effect |
| CN114881220B (en) * | 2022-05-17 | 2023-11-14 | 常州大学 | FHN neuron-based cubic nonlinear function fitting circuit |
| CN115062583B (en) * | 2022-06-15 | 2024-05-31 | 华中科技大学 | Hopfield network hardware circuit for solving optimization problem and operation method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106485317A (en) * | 2016-09-26 | 2017-03-08 | 上海新储集成电路有限公司 | A kind of neutral net accelerator and the implementation method of neural network model |
| CN106815636A (en) * | 2016-12-30 | 2017-06-09 | 华中科技大学 | A kind of neuron circuit based on memristor |
| CN107784359A (en) * | 2017-09-19 | 2018-03-09 | 常州大学 | A kind of more stable state oscillation circuits based on Hopfield neutral nets |
| CN108427843A (en) * | 2018-03-14 | 2018-08-21 | 常州大学 | It is a kind of that there is the three-dimensional memristor Hindmarsh-Rose precircuits hidden and asymmetric behavior coexists |
-
2019
- 2019-04-10 CN CN201910283951.7A patent/CN110097182B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106485317A (en) * | 2016-09-26 | 2017-03-08 | 上海新储集成电路有限公司 | A kind of neutral net accelerator and the implementation method of neural network model |
| CN106815636A (en) * | 2016-12-30 | 2017-06-09 | 华中科技大学 | A kind of neuron circuit based on memristor |
| CN107784359A (en) * | 2017-09-19 | 2018-03-09 | 常州大学 | A kind of more stable state oscillation circuits based on Hopfield neutral nets |
| CN108427843A (en) * | 2018-03-14 | 2018-08-21 | 常州大学 | It is a kind of that there is the three-dimensional memristor Hindmarsh-Rose precircuits hidden and asymmetric behavior coexists |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110097182A (en) | 2019-08-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110097182B (en) | Three-dimensional Hopfield neural network model realization circuit controlled by nerve activation gradient lambda | |
| Indiveri et al. | Neuromorphic silicon neuron circuits | |
| Ding et al. | Memristor synapse-coupled piecewise-linear simplified Hopfield neural network: Dynamics analysis and circuit implementation | |
| Zamarreño-Ramos et al. | On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex | |
| CN103650350B (en) | Spike domain circuit and modeling method | |
| CN104573238B (en) | A kind of circuit design method of memristor cell neural network | |
| Levi et al. | Digital implementation of Hodgkin–Huxley neuron model for neurological diseases studies | |
| Cai et al. | Smooth nonlinear fitting scheme for analog multiplierless implementation of Hindmarsh–Rose neuron model | |
| Bao et al. | Global multistability and analog circuit implementation of an adapting synapse-based neuron model | |
| CN113344191B (en) | Continuous Rulkov electronic neuron circuit with super multi-stability | |
| CN109978159B (en) | Simple Fitzhugh-Nagumo neuron circuit | |
| CN112884141A (en) | Memristive coupling Hindmarsh-Rose neuron circuit | |
| Aggarwal et al. | New memristor-less, resistor-less, two-OTA based grounded and floating meminductor emulators and their applications in chaotic oscillators | |
| Ranjbar et al. | An analog astrocyte–neuron interaction circuit for neuromorphic applications | |
| CN113054947B (en) | ReLU type memristor simulator | |
| CN105846990A (en) | Improved normative Chua's chaotic circuit | |
| CN111079365A (en) | Arc tangent trigonometric function memristor circuit model | |
| CN108427843A (en) | It is a kind of that there is the three-dimensional memristor Hindmarsh-Rose precircuits hidden and asymmetric behavior coexists | |
| JP4997495B2 (en) | Nerve equivalent circuit, synapse equivalent circuit and nerve cell body equivalent circuit | |
| Castaños et al. | Implementing robust neuromodulation in neuromorphic circuits | |
| CN113033793A (en) | Circuit for exciting two-dimensional Wilson neuron model by bipolar pulse current | |
| CN107784359A (en) | A kind of more stable state oscillation circuits based on Hopfield neutral nets | |
| Lai et al. | Simple cyclic memristive neural networks with coexisting attractors and large-scale amplitude control | |
| Borwankar et al. | An analog implementation of fitzhugh-nagumo neuron model for spiking neural networks | |
| CN115765964A (en) | Triangular wave memristor conservative signal generator with isomorphic amplitude modulation function |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20190806 Assignee: Changzhou Ruixinteng Microelectronics Co.,Ltd. Assignor: CHANGZHOU University Contract record no.: X2023980054127 Denomination of invention: Using neural activation gradients l Implementation of a circuit using a three-dimensional Hopfield neural network model for control Granted publication date: 20230324 License type: Common License Record date: 20231227 |