US8266085B1 - Apparatus and method for using analog circuits to embody non-lipschitz mathematics and properties using attractor and repulsion modes - Google Patents
Apparatus and method for using analog circuits to embody non-lipschitz mathematics and properties using attractor and repulsion modes Download PDFInfo
- Publication number
- US8266085B1 US8266085B1 US13/366,568 US201213366568A US8266085B1 US 8266085 B1 US8266085 B1 US 8266085B1 US 201213366568 A US201213366568 A US 201213366568A US 8266085 B1 US8266085 B1 US 8266085B1
- Authority
- US
- United States
- Prior art keywords
- neurons
- attractor
- network
- output
- repulsion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
- G06N3/065—Analogue means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0418—Architecture, e.g. interconnection topology using chaos or fractal principles
Definitions
- the Lipschitz continuity is a form of uniform continuity for functions which are limited to how fast the function can change, i.e., for every pair of points in a graph of a function, the secant of the line segment defined by the points has an absolute value no greater than a definite real number, which is referred to as the Lipschitz Constant.
- a function ⁇ : X ⁇ Y is called Lipschitz continuous if there exists a real constant K ⁇ 0 such that, for all x 1 and x 2 in X, d Y ( ⁇ ( x 1 ), ⁇ ( x 2 ) ⁇ Kd X ( x 1 ,x 2 ).
- K is referred to as a Lipschitz constant for the function ⁇ .
- the function is Lipschitz continuous if there exists a constant. K ⁇ 0 such that, for all x 1 ⁇ x 2 ,
- terminal attractors are introduced for an addressable memory in neural networks operating in continuous time. These attractors represent singular solutions of the dynamical system. They intersect (or envelope) the families of regular solutions while each regular solution approaches the terminal attractor in a finite time period. According to the author (Zak), terminal attractors can be incorporated into neural networks such that any desired set of these attractors with prescribed basins is provided by an appropriate selection of the weight matrix.
- U.S. Pat. No. 5,544,280 to Hua-Kuang Liu, et al. discloses a unipolar terminal-attractor based neural associative memory (TABAM) system with adaptive threshold for alleged “perfect” convergence.
- TABAM neural associative memory
- an associative memory or content-addressable memory is a special type of computer memory in which the user inputs a data word and the memory is searched for storage of the data word. If the data word is located in the CAM, the CAM returns a list of one or more locations or addresses where the data word is located.
- Zak's derivation shows that the Hopfield matrix only works if all the stored states in the network are orthogonal. However, since the synapses have changed from those determined by Hebb's law, Zak's model is different from the Hopfield model, except for the dynamical iteration of the recall process. According to the '280 patent, the improvement of the storage capacity of the Hopfield model by the terminal attractor cannot be determined based on Zak's model.
- the '280 patent discloses a TABAM system which, unlike the complex terminal attractor system of Zak, supra, is not defined by a continuous differential equation and therefore can be readily implemented optically.
- U.S. Pat. No. 6,188,964 hereby incorporated by reference, purportedly discloses a method for generating residual statics corrections to compensate for surface-consistent static time shifts in stacked seismic traces.
- the method includes a step of framing the residual static corrections as a global optimization problem in a parameter space.
- a plurality of parameters are introduced in N-dimensional space, where N is the total number of the sources and receivers.
- the objective function has a plurality of minimum in the N-dimensional space and at least one of the plurality of minimum is a global minimum.
- a preferred embodiment of the present invention is directed to the use of analog VLSI technology to implement non-Lipschitz dynamics in networks of coupled neurons for information processing.
- the attractor/repeller neuron may be programmable to become either a terminal attractor or terminal repeller.
- a preferred embodiment comprises a programmable interconnected network of eight non-Lipschitz neurons implemented in analog VLSI as illustrated in FIG. 6 . Measurements on the fabricated chip have confirmed the generation of proper terminal dynamics.
- a circuit board has been developed to interface the chip with instrumentation and to configure the chip in different modes of operation.
- the method and network use both terminal attractors and terminal repellers in the construction of an analog circuit to achieve a non-Lipschitz dynamics process which can solve important mathematical and physics, problems.
- the analog circuit uses “noise” in the circuit where the system is near the attractor/repeller points to introduce randomness into the branching of the dynamical paths. If the analog circuit were cooled to low temperature or otherwise implemented in a quantum system the noise there would come from quantum effects. Because of the non-Lipschitz properties, the method can be used to: a) make neural network analog circuits, b) improve the speed of circuits, and c) increase the speed of computer calculations.
- a preferred embodiment comprises a network of coupled neurons for implementing Non-Lipschitz dynamics for modeling nonlinear processes or conditions comprising: a plurality of attractor/repeller neurons, each of the plurality of neurons being configurable in attractor and repulsion modes of operation, the plurality of attractor/repeller neurons being programmable by an external control signal; a plurality of synaptic connections; at least a portion of the plurality of neurons being interconnected by the synaptic connections for passage of data from one neuron to another; and feedback circuitry for incrementing and decrementing an analog voltage output depending upon the output of the synaptic connection; whereby by the circuit solves Non-Lipschitz problems by programably controlling the attractor and repulsion modes of operation.
- the invention may be used to solve problems such as the Fokker-Plank equation, Schrödinger equation, and Neural computations.
- FIG. 6 is a schematic illustration of a preferred embodiment implemented architecture, i.e., the architecture and layout of an NLN (non-Lipschitz neuron) chip containing a coupled array of 8 neurons (center) and 2 individually coupled neurons (bottom).
- NLN non-Lipschitz neuron
- FIG. 7 illustrates sample output waveforms of a single neuron configured alternatingly in terminal attraction and repulsion modes.
- Top waveform Synaptic integration x(t); Central waveform: quantization q(t); Bottom waveform: neural state v(t).
- t 0 is an arbitrarily small (but finite) positive quantity.
- the rate of divergence (5A) can be defined in an arbitrarily small time interval, because the initial infinitesimal distance between the solutions (Equation 2A) becomes finite during the small interval t 0 . Recalling that in the classical case when the Lipschitz condition is satisfied, the distance between two diverging solutions can become finite only at t ⁇ 0 if initially this distance was infinitesimal.
- Equation ⁇ ⁇ 6 ⁇ A the singular solution (Equation 3A) is unstable, and it departs from rest following Eq. (3A).
- This solution has two (positive and negative) branches, and each branch can be chosen with the same probability %. It should be noticed that as a result of Equation (4A), the motion of the particle can be initiated by infinitesimal disturbances (that never can occur when the Lipschitz condition is in place since an infinitesimal initial disturbance cannot become finite in finite time).
- Equation 2A the solution (Equation 2A) is valid only in the time interval
- Equation (2A) becomes unstable, and the motion repeats itself to the accuracy of the sign in Equation (2A).
- the solution performs oscillations with respect to its zero value in such a way that the positive and negative branches of the solution (2A) alternate randomly after each period equal to
- variable y performs an unrestricted symmetric random walk: after each time period
- Equation (11A) defines f as a function of two discrete arguments:
- Equation ⁇ ⁇ 14 ⁇ A f ⁇ [ ⁇ - 1 ⁇ ( z ) , t ] ⁇ ⁇ d ⁇ - 1 d z ⁇ .
- Equation 18A Equation 18A
- Equation ⁇ ⁇ 19 ⁇ A i.e. the sign of ⁇ at the critical instances to time (Equation 19A) uniquely defines the evolution of the dynamical system (Equation 18A).
- non-Lipschitz dynamics include stochastic model fitting for identification of physical, biological and social systems, simulation of collective behavior, models of neural intelligence (as discussed in Zak; NI “Introduction to terminal dynamics,” Complex Systems 7, 59-87 (1993)]; and Zak, M “Physical models of cognition,” Int. J. of Theoretical Physics 5 (1994), both of which are hereby incorporated by reference).
- Non-Lipschitz coupled dynamics in neural networks offer an attractive computational paradigm for combining neural information processing with chaos complexity and quantum computation, as discussed in M. Zak, J P Zbilut and R E Meyers, From Instability to Intelligence , Lecture Notes in Physics 49, Springer-Verlag (1997).
- the singularity in the first order derivative now allows the state variable v to escape the equilibrium in finite time, even in the absence of noise ⁇ (t).
- Variable timing in the escape of terminal repellers contribute randomness to an otherwise deterministic system.
- FIG. 1 shows the implementation of the terminal attractor 10 using the quantization model (Equation 27A).
- Two complementary MOS differential pairs ( 2 , 3 ) steer a current of polarity controlled by q into a capacitor ( 6 ) on the node v, generating the dynamics (Equation 22A).
- the temporal scale of the terminal dynamics is given by the capacitance and the tail currents of the differential pairs, set by nMOS and pMOS bias voltages V tn and V tp , respectively.
- the terminal attractor circuit 10 can be modified to implement terminal repulsion by inclusion of an additional inversion stage between the inverting amplifier output and the current-steering differential pairs.
- Terminal repeller 12 comprises subcircuits 10 A (also appearing in FIG. 1) and 11 as shown in FIG. 2 .
- the inversion stages are shown in the terminal repeller 12 of FIG. 2 as two additional inverting amplifiers.
- the reason for using complementary (pseudo-nMOS and pseudo-pMOS) structures for inverting amplifiers is to provide hysteresis in the positive feedback dynamical response of q(v).
- the pseudo-nMOS amplifier (top) provides a threshold near the lower supply range, and the threshold of the pseudo-pMOS amplifier (bottom) approaches the higher supply.
- Synaptic connectivity according to Equation (26A) is implemented using the current-inode circuit cell shown in FIG. 4 .
- Both current sources are bipolar, and are implemented as the difference between positive currents collected on the drain terminals of nMOS 43 , 43 A and pMOS transistors 44 , 44 A.
- the nMOS current sinking transistors 43 , 43 A are supplied with bias voltages T ij n+ and T ij n ⁇ (near GND) on the gate terminals, and similarly the pMOS current sourcing transistors 44 , 44 A are supplied with bias voltages T ij p+ and T ij p ⁇ (near Vdd).
- the hashmarks shown in FIG. 4 indicate that the states q j couple to several x i (integration nodes 42 ) in the form of several synapses T ⁇ ij ⁇ .
- the source terminals are switched by q j (and q j ) rather than gate terminals, to avoid switch injection noise during transients and to allow precise control of very small currents, as needed to attain large dynamic range in neural integration times.
- Cdx i /dt sum j (( T ij p+ ⁇ T ij n+ ) q j +( T ij p ⁇ ⁇ T ij n ⁇ ) q j ) where q j takes-values 0 (logic low) and 1 (logic high), and q j is its complement.
- the constant values for the generated currents T ij n+ , T ij n ⁇ , T ij p+ , T ij p ⁇ in this model are controlled by the voltages (and are exponential in the voltages) on these T ij n+ , T ij n ⁇ , T ij p+ , T ij p ⁇ nodes.
- the resulting analog voltage function V i is the analog level defining the state of the memory, and ⁇ the size of the partial increments, the procedure combines the consecutive steps of binary quantization q j ( ): R ⁇ + ⁇ 1, +1 ⁇ ; and Incremental Refresh: V i ⁇ V i + ⁇ T ij Iteration of this procedure yields a stored memory value V i as described further Cauwenberghs '94.
- This charge-pump type of implementation of the increment/decrement device in CMOS technology, in contact with a capacitive storage device 51 results in fixed charge increments or decrements on.
- Capacitive storage device 51 by selectively activating one of four supplied constant currents T ij n+ , T ij n ⁇ , T ij p+ , T ij p ⁇ of sometimes opposite polarity, over a fixed time interval.
- FIGS. 3 , 4 , and 5 are connected together through the nodes v, q, and x, as labeled in the figures.
- FIG. 6 is a schematic illustration of a preferred embodiment implemented architecture, i.e., the architecture and layout of an NLN (non-Lipschitz neuron) chip.
- the top of FIG. 6 shows a network of eight “nearest-neighbor coupled” non-Lipschitz neurons 13 , with identical nearest-neighbor synaptic connections (T 12 and T 21 (double line)), included along with the neuron circuits in each rectangular box 13 ).
- the synaptic connections T 12 and T 21 are between neighbors in the ring of 8 neurons.
- Each neuron 13 connects to its upper neighbor neuron 13 with synaptic strength T 12 and to its lower neighbor neuron 13 with synaptic strength T 21 .
- the multiplexer (MUX) on the left allows one to look at the signal variables of any of the seven left neurons 13 , together with those for the eighth neuron 13 on the right.
- MUX multiplexer
- FIG. 6 illustrates an additional two individual coupled neurons 13 ′ that have been implemented on a 2 mm ⁇ 2 mm chip in 1.2 ⁇ m CMOS technology.
- T′ 22 , T′ 11 , T′ 12 , T′ 21 in the lower block represent synaptic connections T′ ij between presynaptic neuron terminals qj and postsynaptic neuron terminals x i , as defined in FIGS. 4 and 5 , where both indices i and j take values 1 and 2 (there are two neurons).
- Each of the two neurons 13 ′ has the feedback element of FIG. 5 (with input current x i ), connecting to the non-Lipschitz element of FIG. 3 , and the sampler in the bottom of FIG.
- the T ij elements are positioned in the top part ( 40 B) of FIG. 4 (with sampled neural states q j and qj as input, and current x i as output). There are four of these, connecting each neuron to itself and the other neuron.
- FIG. 7 illustrates sample output waveforms of a single neuron configured alternatingly in terminal attraction and repulsion mode.
- Top waveform of FIG. 7 illustrates synaptic integration x(t).
- the central waveform illustrates quantization q(t).
- the bottom waveform of FIG. 7 illustrates neural state v(t).
- neuroneuron means an electrically excitable cell or element that processes and transmits information by electrical signals. Each neuron has at least one input and at least one output.
- the terminology “attractor” relates to elements that provide, singular solutions of the dynamical system. They intersect (or envelope) the families of regular solutions while each regular solution approaches the terminal attractor in a finite time period.
- the methodology may utilize a closed form of the model to analytically reduce the system dynamics onto a stable invariant manifold, onto which empirical data is “attracted.”
- a trajectory of the dynamical system in the attractor does not have to satisfy any special constraints except for remaining on the attractor.
- the trajectory may be periodic or chaotic. If a set of points is periodic or chaotic, but the flow in the neighborhood is away from the set, the set is not an attractor, but instead is called a repeller (or repellor).
- Tij represents interconnections between the neurons. Iterative adjustments of Tij may be a result of comparison of the net output with known correct answers (supervised learning) or as a result of creating new categories from the correlations of input data when the correct answers are not known (unsupervised learning).
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Neurology (AREA)
- Image Analysis (AREA)
Abstract
Description
ƒ:X→Y
is called Lipschitz continuous if there exists a real constant K≧0 such that, for all x1 and x2 in X,
d Y(ƒ(x 1),ƒ(x 2)≦Kd X(x 1 ,x 2).
where K is referred to as a Lipschitz constant for the function ƒ. The function is Lipschitz continuous if there exists a constant. K≧0 such that, for all x1≠x2,
x=x 1/3 sin ωt, ω=cos t (Equation 1A)
{dot over (x)}=x 1/3 sin ωt, ω=cos t (Equation 1A)
and a singular solution (an equilibrium point):
x=0. (Equation 3A)
where t0 is an arbitrarily small (but finite) positive quantity. The rate of divergence (5A) can be defined in an arbitrarily small time interval, because the initial infinitesimal distance between the solutions (Equation 2A) becomes finite during the small interval t0. Recalling that in the classical case when the Lipschitz condition is satisfied, the distance between two diverging solutions can become finite only at t→0 if initially this distance was infinitesimal.
the singular solution (Equation 3A) is unstable, and it departs from rest following Eq. (3A). This solution has two (positive and negative) branches, and each branch can be chosen with the same probability %. It should be noticed that as a result of Equation (4A), the motion of the particle can be initiated by infinitesimal disturbances (that never can occur when the Lipschitz condition is in place since an infinitesimal initial disturbance cannot become finite in finite time).
and at
coincides with the singular solution (Equation 3A). For
Eq. (2A) becomes unstable, and the motion repeats itself to the accuracy of the sign in Equation (2A).
{dot over (y)}=x, (y=0 at x=0). (Equation 8A)
y=±h±h. (
it changes its value on ±h. The probability f (y, t) is governed by the following difference equation:
where h is expressed by Eq. (9A).
z=φ(y), y=(φ−1(z), (Equation 13A)
one can obtain a stochastic process with a prescribed probability distribution:
implemented by the dynamical system (Equation 1A), (Equation 8A), and (Equation 13A).
instead of (8A), one arrives at a non-Markov stochastic process with the correlation time (n+1)τ. The deterministic part of the process can be controlled if instead of (Equation 8A) one applies the following change of variables:
{dot over (u)}(t)=x(t)+X(t−τ) (Equation 17A)
{dot over (x)}=x 1/3 sin ωt+ε, ε→0 (Equation 18A)
. . . etc. Indeed, at these instants, the solution to (18A) would have a choice to be positive or negative if ε=0, (see Equation (2A)).
i.e. the sign of ε at the critical instances to time (Equation 19A) uniquely defines the evolution of the dynamical system (Equation 18A).
where
{dot over (v)}=q(v)+ε(t) (Equation 22A)
where the function q(v) is monotonically decreasing and zero at the origin, but with a singularity in the first-order derivative. The singularity at the origin allows the state variable v to reach the stable equilibrium point v=0 in finite time, and remain there indefinitely.
{dot over (v)}=−q(v)+ε(t) (Equation 23A)
with an unstable equilibrium point at the origin. The singularity in the first order derivative now allows the state variable v to escape the equilibrium in finite time, even in the absence of noise ε(t). Variable timing in the escape of terminal repellers contribute randomness to an otherwise deterministic system.
{dot over (v)}=s(t)q(v)+ε(t) (Equation 24A)
where s(t)>0 for terminal attraction, s(t)<0 for terminal repulsion, and s(t)=0 for stationary dynamics. The timing of the signal s(t) allows to enforce zero initial conditions on the state variable v(t) in attractor mode, for subsequent repulsion as influenced by ε. The critical dependence of the trajectory v(t) on initial conditions in the noise (or signal) ε(t) can be exploited to generate coupled nonlinear chaotic dynamics in a network of neurons. The model of neural feedback can be generally written with activation function
εi =f(x i) (Equation 25A)
and with synaptic coupling dynamics
x i=Σj T ij q j (Equation 26A)
Various instances of the model with different activation functions f(.), synaptic coupling strengths Tij, and modulation sequence s(t), give rise to a vast array of entirely different dynamics that model a variety of physical phenomena from turbulence in fluid mechanics to information processing in biological systems. See, for example, M. Zak, JP Zbilut and R E Meyers, From Instability to Intelligence, Lecture Notes in Physics 49, Springer-Verlag (1997) (hereby incorporated by reference)
Circuit Model
q(v)=·sgn(v) (Equation 27A)
conveniently implemented using a high-
Cdx i /dt=sum j((T ij p+ ·T ij n+)q j+(T ij p− −T ij n−)
where qj takes-values 0 (logic low) and 1 (logic high), and
V i ≈V i +ΣT ij
Iteration of this procedure yields a stored memory value Vi as described further Cauwenberghs '94. This charge-pump type of implementation of the increment/decrement device in CMOS technology, in contact with a
Claims (19)
εi =f(x i).
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/366,568 US8266085B1 (en) | 2012-02-06 | 2012-02-06 | Apparatus and method for using analog circuits to embody non-lipschitz mathematics and properties using attractor and repulsion modes |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/366,568 US8266085B1 (en) | 2012-02-06 | 2012-02-06 | Apparatus and method for using analog circuits to embody non-lipschitz mathematics and properties using attractor and repulsion modes |
Publications (1)
Publication Number | Publication Date |
---|---|
US8266085B1 true US8266085B1 (en) | 2012-09-11 |
Family
ID=46760752
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/366,568 Expired - Fee Related US8266085B1 (en) | 2012-02-06 | 2012-02-06 | Apparatus and method for using analog circuits to embody non-lipschitz mathematics and properties using attractor and repulsion modes |
Country Status (1)
Country | Link |
---|---|
US (1) | US8266085B1 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20170155410A1 (en) * | 2015-11-30 | 2017-06-01 | Metal Industries Research & Development Centre | Error correcting method |
CN108872777A (en) * | 2018-05-31 | 2018-11-23 | 浙江大学 | Winding in Power Transformer state evaluating method based on improved system delay Order- reduction |
Citations (28)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US3646329A (en) | 1968-11-20 | 1972-02-29 | Matsushita Electric Ind Co Ltd | Adaptive logic circuit |
US4660166A (en) | 1985-01-22 | 1987-04-21 | Bell Telephone Laboratories, Incorporated | Electronic network for collective decision based on large number of connections between signals |
US4866645A (en) | 1987-12-23 | 1989-09-12 | North American Philips Corporation | Neural network with dynamic refresh capability |
US4873455A (en) | 1987-10-09 | 1989-10-10 | Thomson-Csf | Programmable ferroelectric polymer neural network |
US4903226A (en) | 1987-08-27 | 1990-02-20 | Yannis Tsividis | Switched networks |
US4904881A (en) | 1989-02-10 | 1990-02-27 | Intel Corporation | EXCLUSIVE-OR cell for neural network and the like |
US4950917A (en) | 1988-07-27 | 1990-08-21 | Intel Corporation | Semiconductor cell for neural network employing a four-quadrant multiplier |
US4961005A (en) | 1989-04-25 | 1990-10-02 | Board Of Trustees Operating Michigan State University | Programmable neural circuit implementable in CMOS very large scale integration |
US5021693A (en) | 1989-04-05 | 1991-06-04 | Kabushiki Kaisha Toshiba | Control circuit for floating gate four-quadrant analog multiplier |
US5045713A (en) | 1989-02-10 | 1991-09-03 | Kabushiki Kaisha Toshiba | Multi-feedback circuit apparatus |
US5059814A (en) | 1988-11-30 | 1991-10-22 | The California Institute Of Technology | Winner-take-all circuits for neural computing systems |
US5083044A (en) | 1989-03-10 | 1992-01-21 | Synaptics, Incorporated | Synaptic element and array |
US5093803A (en) | 1988-12-22 | 1992-03-03 | At&T Bell Laboratories | Analog decision network |
US5159661A (en) | 1990-10-05 | 1992-10-27 | Energy Conversion Devices, Inc. | Vertically interconnected parallel distributed processor |
US5172204A (en) | 1991-03-27 | 1992-12-15 | International Business Machines Corp. | Artificial ionic synapse |
US5195170A (en) | 1991-08-12 | 1993-03-16 | The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration | Neural-network dedicated processor for solving assignment problems |
US5220202A (en) | 1990-02-14 | 1993-06-15 | Olympus Optical Co., Ltd. | Memory device and memory apparatus using the same suitable for neural network |
US5329610A (en) | 1992-05-19 | 1994-07-12 | Intel Corporation | Neural network employing absolute value calculating synapse |
US5341051A (en) | 1992-11-25 | 1994-08-23 | California Institute Of Technology | N-dimensional basis function circuit |
US5381516A (en) | 1991-10-31 | 1995-01-10 | France Telecom | Binary resistor network and its use for labelling related components of digitised images in artificial vision |
US5509105A (en) | 1992-06-09 | 1996-04-16 | University Of Cincinnati | Electronic neuron apparatus |
US5517139A (en) | 1994-02-02 | 1996-05-14 | Gold Star Electron Co., Ltd. | Non-linear circuit and chaotic neuron circuit using the same |
US5519811A (en) | 1991-10-17 | 1996-05-21 | Kawasaki Steel Corporation | Neural network, processor, and pattern recognition apparatus |
US5544280A (en) | 1993-06-07 | 1996-08-06 | The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration | Unipolar terminal-attractor based neural associative memory with adaptive threshold |
US5648926A (en) | 1991-11-06 | 1997-07-15 | Medical Research Council | Silicon neuron |
US5717833A (en) | 1996-07-05 | 1998-02-10 | National Semiconductor Corporation | System and method for designing fixed weight analog neural networks |
US6188964B1 (en) | 1999-09-14 | 2001-02-13 | Ut-Battelle, Llc | Method for using global optimization to the estimation of surface-consistent residual statics |
US6341275B1 (en) | 1999-04-27 | 2002-01-22 | Winbond Electrnics Corp. | Programmable and expandable hamming neural network circuit |
-
2012
- 2012-02-06 US US13/366,568 patent/US8266085B1/en not_active Expired - Fee Related
Patent Citations (28)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US3646329A (en) | 1968-11-20 | 1972-02-29 | Matsushita Electric Ind Co Ltd | Adaptive logic circuit |
US4660166A (en) | 1985-01-22 | 1987-04-21 | Bell Telephone Laboratories, Incorporated | Electronic network for collective decision based on large number of connections between signals |
US4903226A (en) | 1987-08-27 | 1990-02-20 | Yannis Tsividis | Switched networks |
US4873455A (en) | 1987-10-09 | 1989-10-10 | Thomson-Csf | Programmable ferroelectric polymer neural network |
US4866645A (en) | 1987-12-23 | 1989-09-12 | North American Philips Corporation | Neural network with dynamic refresh capability |
US4950917A (en) | 1988-07-27 | 1990-08-21 | Intel Corporation | Semiconductor cell for neural network employing a four-quadrant multiplier |
US5059814A (en) | 1988-11-30 | 1991-10-22 | The California Institute Of Technology | Winner-take-all circuits for neural computing systems |
US5093803A (en) | 1988-12-22 | 1992-03-03 | At&T Bell Laboratories | Analog decision network |
US5045713A (en) | 1989-02-10 | 1991-09-03 | Kabushiki Kaisha Toshiba | Multi-feedback circuit apparatus |
US4904881A (en) | 1989-02-10 | 1990-02-27 | Intel Corporation | EXCLUSIVE-OR cell for neural network and the like |
US5083044A (en) | 1989-03-10 | 1992-01-21 | Synaptics, Incorporated | Synaptic element and array |
US5021693A (en) | 1989-04-05 | 1991-06-04 | Kabushiki Kaisha Toshiba | Control circuit for floating gate four-quadrant analog multiplier |
US4961005A (en) | 1989-04-25 | 1990-10-02 | Board Of Trustees Operating Michigan State University | Programmable neural circuit implementable in CMOS very large scale integration |
US5220202A (en) | 1990-02-14 | 1993-06-15 | Olympus Optical Co., Ltd. | Memory device and memory apparatus using the same suitable for neural network |
US5159661A (en) | 1990-10-05 | 1992-10-27 | Energy Conversion Devices, Inc. | Vertically interconnected parallel distributed processor |
US5172204A (en) | 1991-03-27 | 1992-12-15 | International Business Machines Corp. | Artificial ionic synapse |
US5195170A (en) | 1991-08-12 | 1993-03-16 | The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration | Neural-network dedicated processor for solving assignment problems |
US5519811A (en) | 1991-10-17 | 1996-05-21 | Kawasaki Steel Corporation | Neural network, processor, and pattern recognition apparatus |
US5381516A (en) | 1991-10-31 | 1995-01-10 | France Telecom | Binary resistor network and its use for labelling related components of digitised images in artificial vision |
US5648926A (en) | 1991-11-06 | 1997-07-15 | Medical Research Council | Silicon neuron |
US5329610A (en) | 1992-05-19 | 1994-07-12 | Intel Corporation | Neural network employing absolute value calculating synapse |
US5509105A (en) | 1992-06-09 | 1996-04-16 | University Of Cincinnati | Electronic neuron apparatus |
US5341051A (en) | 1992-11-25 | 1994-08-23 | California Institute Of Technology | N-dimensional basis function circuit |
US5544280A (en) | 1993-06-07 | 1996-08-06 | The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration | Unipolar terminal-attractor based neural associative memory with adaptive threshold |
US5517139A (en) | 1994-02-02 | 1996-05-14 | Gold Star Electron Co., Ltd. | Non-linear circuit and chaotic neuron circuit using the same |
US5717833A (en) | 1996-07-05 | 1998-02-10 | National Semiconductor Corporation | System and method for designing fixed weight analog neural networks |
US6341275B1 (en) | 1999-04-27 | 2002-01-22 | Winbond Electrnics Corp. | Programmable and expandable hamming neural network circuit |
US6188964B1 (en) | 1999-09-14 | 2001-02-13 | Ut-Battelle, Llc | Method for using global optimization to the estimation of surface-consistent residual statics |
Non-Patent Citations (18)
Title |
---|
Bollt, Erik, "Attractor Modeling and Empiricalnonlinear Model Reduction of Issipative Dynamical Systems," International Journal of Bifurcation and Chaos, vol. 17, No. 4 (2007) 1199-1219. |
Cauwenberghs, G, "Analog VLSI stochastic perturbative learning architectures," Int. J. Analog Integrated Circuits and Signal Processing, 13(1/2), 195-209 (1997). |
Cauwenberghs, G. et al., "Fault-tolerant dynamic multi-level storage in analog VLSI," IEEE Trans. Circuits and Systems II: Analog and Digital Signal Processing 41(12), 827-829 (1994). |
Conte, E., et al., "A Brief Note on Possible Detection of Physiological Singulariites in Respiratory Dynamics by Recurrence Quantification Analysis of Lung Sounds," Chaos, Solitons, & Fractals, vol. 21, Issue 4 (40, pp. 869-877 (2004). |
Devoue, V., "Generalized Solutions to a Non Lipschitz-Cauchy Problem," Journal of Applied Analysis vol. 15, No. 1 (2009), pp. 1-32. |
Fukuda, O. et al., "Pattern Classification of EEG Signals Using a Log-linearized Gaussian Mixture Neural Network," Neural Networks, 1995. Proceedings., IEEE International Conference on, vol. 5, pp. 2479-2484 (Nov./Dec. 1995). |
Guillani, A., "A Nonlinear Explanation of Aging-induced Changes in Heartbeat Dynamics," Am. J. Physiology-Heart and Circ. Physiol., 275(4), pp. H1455-H1461, 1998. |
J. J. Hopfield, "Neural Networks and Physical Systems with Emergent Collective Computational Abilities," Proc. Nat. Acad. Sci, U.S.A., vol. 79 p. 2254-2258 (1982). |
Wang et al., Neural Network-BAsed Adaptive Dynamic Surface COntrol for a class of Uncertain Nonlinear Systems in Strict-Feedback Form, 2005, IEEE, pp. 1-8 * |
Wang et al., Robust Adaptive Neural Network Control of Uncertain Nonholonomic Systems With Strong Nonlinear Drifts, 2004, IEEE, pp. 1-2. * |
Zak, M "Physical models of cognition," Int. J. of Theoretical Physics, vol. 33, 5, pp. 1114-1161 (1994)). |
Zak, M., "Introduction to terminal dynamics," Complex Systems 7, 59-87 (1993). |
Zak, M., et al. "Non-Lipschitz Dynamics Approach to Discrete Event Systems," Mathematical Modelling and Scientific Computing: An International Journal (Dec. 1995). |
Zak, Michail, et al. "Non-Lipschitz Dynamics Approach to Discrete Event Systems," by Michail Zak and Ronald Meyers (Mathematical Modelling and Scientific Computing: An International Journal, Dec. 1995. |
Zak, Michail, et al., "From Instability to Intelligence, Lecture Notes in Physics," 49, Springer-Verlag (1997). |
Zak, Michail,"Terminal Attractors for Addressable Memory in Neural Networks," Physics Letters A, vol. 133, Issues 1-2, pp. 18-22 (Oct. 31, 1988). |
Zak, Self-Supervised dynamical systems, 2004, Science direct, pp. 1-22. * |
Zbilut, J, et al. "A Terminal Dynamics Model of the Heartbeat," Biol. Cybern. 75, pp. 277-280. |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20170155410A1 (en) * | 2015-11-30 | 2017-06-01 | Metal Industries Research & Development Centre | Error correcting method |
US10097214B2 (en) * | 2015-11-30 | 2018-10-09 | Metal Industries Research & Development Centre | Error correcting method |
CN108872777A (en) * | 2018-05-31 | 2018-11-23 | 浙江大学 | Winding in Power Transformer state evaluating method based on improved system delay Order- reduction |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11875249B2 (en) | Counter based resistive processing unit for programmable and reconfigurable artificial-neural-networks | |
US9779355B1 (en) | Back propagation gates and storage capacitor for neural networks | |
CN107924227B (en) | Resistance processing unit | |
US10740671B2 (en) | Convolutional neural networks using resistive processing unit array | |
US10755170B2 (en) | Resistive processing unit with hysteretic updates for neural network training | |
US8694452B2 (en) | Methods and systems for CMOS implementation of neuron synapse | |
US11087204B2 (en) | Resistive processing unit with multiple weight readers | |
Rahimi Azghadi et al. | Tunable low energy, compact and high performance neuromorphic circuit for spike-based synaptic plasticity | |
US11195089B2 (en) | Multi-terminal cross-point synaptic device using nanocrystal dot structures | |
Koyama et al. | Guaranteed storage and stabilization of desired binary periodic orbits in three-layer dynamic binary neural networks | |
US8266085B1 (en) | Apparatus and method for using analog circuits to embody non-lipschitz mathematics and properties using attractor and repulsion modes | |
Merkel et al. | Neuromemristive extreme learning machines for pattern classification | |
Wu et al. | CMOS current-mode neural associative memory design with on-chip learning | |
Hadaeghi | Neuromorphic electronic systems for reservoir computing | |
Vidhya et al. | Stability of impulsive stochastic reaction diffusion recurrent neural network | |
Ghosh et al. | OTA-based neural network architectures with on-chip tuning of synapses | |
Marrone et al. | Tunable chaos in memristor circuits for pattern recognition tasks | |
US11133063B1 (en) | Suppressing undesired programming at half-selected devices in a crosspoint array of 3-terminal resistive memory | |
Dolzhikova et al. | Memristor-based synaptic sampling machines | |
Faridi et al. | Memristor-a promising candidate for neural circuits in neuromorphic computing systems | |
Vuppala | Analog implementation of a current-mode Rectified Linear Unit (ReLU) for artificial neural networks | |
Choi et al. | Programmable synapse and neuron circuits in VLSI for perceptions and cellular neural networks | |
Babacan et al. | Neuron Circuit Based on CCII‐ | |
Oren et al. | Ultra-Low Power Electronic Circuits Inspired by Biological Genetic Processes. | |
Li et al. | Binary‐Stochasticity‐Enabled Highly Efficient Neuromorphic Deep Learning Achieves Better‐than‐Software Accuracy |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: ARMY, THE UNITED STATES OF AMERICA AS REPRESENTED Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MEYERS, RONALD E.;DEACON, KEITH S.;CAUWENBERGHS, GERT J.;SIGNING DATES FROM 20120119 TO 20120208;REEL/FRAME:027731/0611 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20200911 |