WO2006005957A2

WO2006005957A2 - Signal processing circuit

Info

Publication number: WO2006005957A2
Application number: PCT/GB2005/002754
Authority: WO
Inventors: Il-Song Han
Original assignee: University Of Sheffield
Priority date: 2004-07-14
Filing date: 2005-07-14
Publication date: 2006-01-19
Also published as: GB0415735D0; GB2416236A; WO2006005957A3; GB2416236B

Abstract

Embodiments of the present invention relate to a signal processing circuit and, more particularly, to an arithmetic circuit for performing arithmetic operations such as, for example, multiplication and/or addition/subtraction operations, as well as to a circuit having a voltage-controlled transconductance. The circuit comprises a first MOSFET arrangement for performing an arithmetic operation for given first operands and, preferably, at least one other MOSFET for performing substantially the same arithmetic operation for given second operands with the second operands being chosen to the influence the output of the first MOSFET circuit by at least reducing, and preferably, eliminating, the presence of an undesirable component of the output of the first MOSFET circuit.

Description

SIGNAL PROCESSING CIRCUIT

Field of the Invention

The present invention relates to a signal processing circuit and, more particularly, to a VLSI signal processing circuit.

Background to the Invention

The role played by VLSI technology in realising neural networks is well understood by those skilled in the art. Neural networks or, more particularly, neural networks circuits comprise a number of processing elements, connected by weighted synaptic inputs and outputs, to perform particular tasks. A processing element typically comprises a number of multipliers to apply weightings to the synaptic inputs and outputs and a summer for producing a synaptic output from the weighted synaptic inputs.

Those skilled in the art understand that the implementation of a multiplication function performs a pivotal role in the performance of the neural networks. Any such multiplier should preferably be flexible to accommodate various interfaces to existing technologies and algorithms such as, for example, different neural networks paradigms or neural data representations.

Previously three approaches have been taken to addressing these technical problems. A first approach was to use conventional digital computer technology in implementing neural networks. This approach has been relatively successful up to a certain level of complexity or for dedicated applications. However, a significant disadvantage of this approach is limited capacity even though an advantage of the first approach is the flexible interface to conventional computer technology. Examples of this approach are, for example, HNC 100NAP, Adaptive solution CNAPS, Intel Nestor NiIOOO and IBM ZISC036. A second approach has been to construct dedicated analogue circuits for simulating neural network operations. This approach has the advantage of being able to realise significantly larger neural networks and/or enables flexible sensory interfaces to be implemented. However, it can be difficult to implement such circuits since they require specialised manufacturing technology and it can be difficult to interface such an analogue realisation with conventional computer technologies. An example of this approach can be found in the Intel 8017ON ETANN product. The third approach has been to attempt to combine the advantages of both the analogue and digital approaches. However, one encounters manufacturing technology issues that are similar to those encounted for the second approach.

It is an object of embodiments of the present invention to at least mitigate some of the problems associated with the prior art.

Summary of Invention

Accordingly, an aspect of embodiments of the present invention provides method of analogue signal processing using non-linear devices operable in respective triode regions (operable with non-linear current-voltage relationship); the method comprising the steps of producing an input current (IIN⁼ID_SI+I_DS2) using respective currents of a pair of the non-linear devices, the input current having a linear current- voltage characteristic with respect to a selected parameter (VD_S) of the pair of non- linear devices; the input current comprising a first portion (α(Vc.Vχ)) representing an arithmetic operation between first (Vc) and second voltages (VIN=VX+VTP) applied to first and second terminals associated with the non-linear devices to induce the flow of the respective currents.

A further aspect of embodiments of the present invention provides a signal processing circuit comprising at least a pair of non-linear devices operable together to induce a current (IIN) varying linearly with respect to the parameter (VD_S); wherein respective currents of the pair of non-linear devices vary non-linearly (0(2)) with, respect to the parameter (VDS); at least a first device of the pair of non-linear devices comprising a first input terminal for receiving a first input signal to induce the flow of the respective currents.

Advantageously, embodiments of the present invention allow mixed analogue and digital signal processing to be realised. A further advantage is that a multiplication function suitable for use in neural networks can be realised that has at least one of the following advantages (a) low power consumption, (b) high-speed operation and (c) supports asynchronous operation. Embodiments of the present invention use a voltage controlled linearised transconductance or resistance to produce various signal processing elements or computing components. The signal processing elements can comprise, for example, an electronically tuneable filter, a programmable transconductance amplifier, a polyphase filter for communication systems, an analogue decoder or an artificial neural network element.

Embodiments provide a method of analogue signal processing using non-linear devices operable with non-linear current-voltage relationships with respected to a selected parameter; the method comprising the steps of producing an input current (IH^ID_SI+ID_SI) using respective currents of a pair of the non-linear devices, the input current having a linear current-voltage characteristic with respect to the selected parameter (V_DS) of the pair of non-linear devices; the input current comprising a first portion (α(Vc-Vχ)) representing an arithmetic operation between first (VQ) and second voltages (VIN=V_X+V_TP) applied to first and second terminals associated with the non-linear devices to induce the flow of the respective currents.

It can be appreciated that the signal processing elements are realised using a relatively small number of transistors, preferably MOSFET transistors, arranged to compensate for the nonlinearity of MOSFET resistance in the triode region.

The signal processing elements advantageously implement functions that are equivalent to the linear equations for multiplication or summation using a small number of MOSFET devices. Furthermore, the flexibility in circuit configurations accommodates either a pulse-based implementation or an analogue based implementation of electronic synapses that use multiplication or summation.

A still further advantage of embodiments of the present invention is the scalability of the signal processing elements and/or that a low power consumption is achieved for large system integration since only the parts of the signal processing element that are active consume power.

A still further advantage of embodiments of the present invention is that they are operable using a single voltage source a rather than the typical positive and negative voltage sources used in prior art implementations see, for example, UK patent application GB 2 261 093 A, which discloses a MOSFET multiplier using positive and negative, Vx and -Vx, voltage sources.

Brief Description of the Drawings

Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings in which:

figure 1 shows a first embodiment of a signal processing element;

figure 2 illustrates a second embodiment of a signal processing element having an improved offset, that is, improved accuracy;

figure 3 shows a third embodiment of a signal processing element without an offset;

figure 4 depicts a fourth embodiment of a signal processing element; and

figure 5 illustrates a first embodiment of a signal processing element also having an improved offset.

Detailed Description of Preferred Embodiments

Referring to figure 1, there are shown a MOSFET circuit 100 comprising first Ml and second M2 MOSFETs arranged in parallel. The sources of the first Ml and second M2 MOSFETs are connected to ground. The drains of the first Ml and second M2 MOSFETs are connected to the drain of a third MOSFET M3, the source of which is connected to a first input terminal 102 for receiving a first input signal, Vm. It will be appreciated that the first input signal represents an embodiment of an operand or a signal from which such as operand can be derived.

The sources of the first Ml and the second M2 MOSFETs are connected to the respective substrates of those MOSFETs. Similarly, the source of the third MOSFET M3 is connected to its substrate. It can also be appreciated that the drains of the first Ml to third M3 MOSFETs are connected to the gate of the third MOSFET M3. The gate of the second MOSFET M2 is also connected to the first input terminal 102 for input signal, VIN- The gate of the first MOSFET Ml is connected to a second input terminal 104 used to receive an input signal, Vc, which represents a second operand or at least a signal from which a second operand can be derived. Optionally, the signal representing the second operand can be Vc=Vc_DC ⁺v_Si_g, where V_CDC represents the DC component of Vc and v_si_g represents the AC component of Vc.

The gate of the third MOSFET M3 is connected to the gate of a fourth MOSFET M4. The source of the fourth MOSFET M4 is also connected to the first input terminal 102. The drain of the fourth MOSFET M4 is connected to the drain of a fifth MOSFET M5. The source of the fifth MOSFET M5 is connected to ground. Therefore, it can be appreciated that the fourth and fifth MOSFETs are connected in series between the first input terminal and ground. It will be appreciated by those skilled in the art that the fourth MOSFET M4 acts as a current mirror to mirror the current flowing through the third MOSFET M3. The fifth MOSFET M5 is used as a bias to control or at least provide an output current offset. It can be appreciated that the sources of the fourth and fifth MOSFETs M4 and M5 are connected to their respective substrates.

It will be appreciated by those skilled in the art that the first, second and fifth MOSFETs Ml , M2 and M5 are n-type MOSFETs, that is, nMOSFETs. It will also be appreciated that the third and fourth MOSFETs M3 and M4 are p-type MOSFETs, that is, pMOSFETs.

The operation of the basic embodiment of the present invention shown in figure 1 can be described by the following equations, hi general, the drain-source current, ID_S, for a MOSFET operating in the linear or triode region is given by I_DS=<xW_GS-V_T)*V_DS-V_DS ²12} (1)

It will be appreciated by those skilled in the art that α is the MOSFET process parameter, V_GS, VD_S and V_T are the transistor gate-source voltage, transistor drain- source voltage and the threshold voltage respectively. It will be appreciated by those skilled in the art that the combination of MOSFETs Ml and M2 allow a linear voltage multiplier to be realised. The purpose of the second MOSFET M2 is to remove the second order term V²D_S/2. Therefore, the currents, I_DSI and ID_S2, flowing through the first and second MOSFETs Ml and M2 are given by

I_DSI = <Wc ^{~ V}τ) • V_∞ - Vj /2] (2)

I_DS2 = aW_m -V_T)*V_DS - V_DS ² /2] (3)

Therefore, the total current, Im, for the first and second MOSFETs Ml and M2 is given by

= aWc-V_T)*V_DS-V_DS ² /2 + (V_m-V_T).V_DS-V_DS ² /2]

=aKV_c+V_ffl-2.V_T).V_DS-V_DS ²] ₍₄₎

It will be appreciated that the third MOSFET, M3, functions as a diode by dropping the voltage, VIN, presented at the first input terminal 102 by an amount equal to the threshold voltage for the third MOSFET, M3. Therefore, assuming the voltage at the drains of the first and second MOSFETs Ml and M2 to be Vx, that voltage is given by VX=(VIN-V_TP), where V_n- is the pMOS threshold voltage of the third MOSFET, M3. It will be appreciated that Vrp is usually of the order of Vj. It will be also appreciated that the current, IM₃, of the third MOSFET, M2, is equal to Iπsr- Therefore, the current through the third MOSFET₅ M3, is given by

=aW_c+V_x+V_IT-2»V_T). V_x -V_x ²] = oi(y_c +V₇₁, -2. V_T) .V_X + V_x ² -V_x ²]

= a(V_c + V_τp -2.V_τ) . V_x

= a(V_c +V_φet) . V_x (5)

Since the fourth. MOSFET, M4, acts as a current mirror to mirror the current flowing through the third MOSFET₅ M3, that is, to mirror IM₃, the current in M4 is linearly proportional to both the control voltage, Vc, and the input voltage, VIN, (or Vχ+Vχp), or the multiplied result of Vc and Vx.

Furthermore, by controlling the gate bias voltage, V_bi_as, applied to the fifth MOSFET, M5, the offset value, αV_OffsetVχ, of the output current can be controlled, that is, the current flowing through the fifth MOSFET, M5, is subtracted from the current flowing through the fourth MOSFET₅ M4, to give an output current, I out, as

^ OUT ⁼ ^1M3 ~~ ^BIAS (")

It will be appreciated by those skilled in the art that both excitation and inhibition properties of a synaptic current can be realised using the output current, I_OU_T, that is, addition and/or subtraction can be performed according to the bias voltage applied to the fifth MOSFET, M5. Furthermore, the output current, lour, is linearly proportional to Vc or VIN.

Referring to figure 2, there is shown a second embodiment of a MOSFET circuit 200. The second MOSFET circuit 200 comprises two circuits 202 and 204. It can be appreciated that the circuits 202 and 204 are substantially similar to the circuit 100 of the first embodiment. However, the biasing for the fifth MOSFET₅ M5, is derived from the second circuit 204. More particularly, the bias voltage, Vbias, for the fifth MOSFET₅ M5, is derived from the voltage at the drains of fourth and fifth MOSFETs₅ M4A and M5A, of 1he second circuit 204. It can be appreciated that first and second input terminals 206 and 208 are used to supply operands, VJN and Vc, respectively, to the first MOSFET circuit 202. It can be appreciated that third and fourth input terminals 210 and 212 are used to supply operands, V_IN and V_CD_C, respectively, to the second MOSFET circuit 202. It will also be appreciated that the second circuit 204 is substantially identical to the first circuit 100 but for the gate of the fifth MOSFET, M5A, being tied to its drain. Those skilled in the art will appreciate that the current flowing through the fourth MOSFET, M4, of the first circuit 202 will be influenced by the operation of the first, MlA, to fifth, M5A, MOSFETs of the second circuit 204. The arrangement of the fifth MOSFETs, M5 and M5A, in operation, replicates the current flowing through the fourth MOSFET, M4A, of the second circuit 204 to the fifth MOSFET, M5, of the first circuit 202. The current, I_BIAS, in the fifth MOSFET, M5, of the first circuit 202 mirrors the currents flowing in any of the fifth, fourth of third MOSFETs, M5A, M4A or M3A, of the second circuit 204. This bias current, I_BIAS, is, therefore, given by

^■'^■ BIAS ⁼ -*M3A

^ a(Fa* + F_n, -2. ^ . K_x (7),

where VC_DC is a DC (or offset) component of Vc, that is, Vc=VcDC+v_si_g. It will be appreciated that the output current, lour, &om the equations (5), (6) and (7), by

J OUT ⁼ ^m ^~ *BiAs

⁼ -*M3 ~ *MIA

= cc(V_c + V_TP -2.V_τ) . V_x - a(V_CDC + V_JP -2. V_τ) . V_x

= Φ_sig)»v_x

= a(y_stg) . {V_1N -V_7P) (8)

It will be appreciated from an examination of equation (8) that the output current, lour_? it can be thought of as implementing a multiplication function. Furthermore, it can also be appreciated that the MOSFET circuit 200 shown in figure 2 is operable to provide a voltage-controlled linear transconductance. Still further, assuming that V_JN comprises both DC and AC components, V₁N_Dc and Vi_n, such that the V_1N=V₁N_Dc⁺Vi_n, then the output current, IQU_T, can be regarded as a linear calculation. Figure 3 shows a MOSFET circuit 300 according to a third embodiment. The MOSFET circuit 300 comprises first 302 and second 304 circuits. It will be appreciated that the first circuit 302 is substantially identical to the MOSFET circuit 200 of the second embodiment. The operation of the first circuit 302, therefore, will not be described in detail as its operation can be understood from the description given in relation to the MOSFET circuit 200 of the second embodiment. It will also be appreciated that the second circuit 304 of the third embodiment is substantially similar to the MOSFET circuit 200 described above. However, it should be noted that the configurations of the fifth MOSFETs, M5B and M5C, are different to the configurations of the fifth MOSFETs, M5 and M5A, of the second embodiment of the MOSFET circuit 200. The fifth MOSFET, M5B, of the second circuit 304 is arranged so its gate is coupled to its drain. The other fifth MOSFET, M5C, is arranged so that its gate is coupled to the drain or gate of the fifth MOSFET, M5B. It can be appreciated that an output terminal is provided that is connected to the drains of the fourth MOSFETs M4 and M5C of the first 302 and the second 304 circuits of the third embodiment.

Referring to the first circuit 302 of the third MOSFET circuit 300, it can be seen that a total of four inputs terminals 302a to 304d are provided. The upper circuit 306 of the first circuit 302 comprises two inputs terminals 302a and 302b for receiving operands V_EST and Vc- The lower circuit 308 of the first circuit 302 of the third embodiment comprises two inputs terminals 302c and 302d for operands VIN and V_CD_C- The second circuit 304 comprises two inputs terminals 304a and 304b for operands VIND_C and Vc for the upper circuit 310 and two inputs terminals 304c and 304d for operands VINDC and V_CD_C for the lower circuit 312.

The operation of the MOSFET circuit 300 according to the third embodiments can be described by, and understood from, the following equations.

^■* OUT ⁼ -*M3 ^"~ *M3A ~ *M3B ⁺ *M3C

⁼ \*M3 ~ *M3A) ~ \*M3B ~ -*M3c)

= Φ_stg) • (y_M - Vjp) - a(v_sig) • (V_JMC - V₇₁.)

= Φ_sig) <v_in) (9) Therefore, the MOSFET circuit 300 according to the third embodiments can be used as a very precise analogue multiplier, where the output is provided as a current, which can be summed by wired-OR connections. Furthermore, when the MOSFET circuit 300 is used as a transconductance amplifier, the transconductance, G_m, is programmable in response to VIN or Vj_n according to the degree of precision required by an application for that amplifier.

It can be appreciated that the above embodiments provide varying levels of arithmetic accuracy and complexity such that an appropriate embodiment can be matched with a corresponding application requiring a respective degree of accuracy.

Using wired-OR logic it is possible to provide functions necessary for a soft- exclusive-OR gate or a large neural network implementation. The soft-exclusive-OR gate, which is a sum-product module, can be described as follows

P₃(O) P₃(I)

where P₁(O) and Pi(I) are the probabilities of "0" and "1", P₁(O and P₂(.) are inputs respectively, as is well understood by those skilled in the art see, for example, "Decoding in Analog VLSF', Hans-Andrea Loeliger, Felix Tarkδy, Felix Lustenberger and Markus Helfenstein, IEEE Communications Magazine, April 1999, pp 99-101, which is incorporated herein in its entirety by reference for all purposes.

Referring to figure 4, there is shown a fourth MOSFET circuit 400, which is a MOSFET resistance based analogue multiplier. The fourth MOSFET circuit 400 comprises a first MOSFET, Ml, connected in series with a second MOSFET, MD2. The drains of the first Ml and second MD2 MOSFETs are coupled together. The sources of the first Ml and second MD2 MOSFETs are connected to ground and to an input terminal 402 respectively. The input terminal 402 is used to supply an operand, VIN, to the circuit. It can be appreciated that the substrates of both MOSFETs Ml and MD2 are coupled to their respective sources. The gate of the second MOSFET, MD2, is connected to its drain. The first MOSFET Ml has its gate connected to a second input terminal 404 for receiving a second operand, Vc. The fourth MOSFET circuit 400 also comprises a series of arrangement of a p-type MOSFET, MD3, and an n-type MOSFET, M2. The drains of these MOSFETs, MD3 and M2, are coupled together. The sources of these two MOSFETs are coupled to the input terminal 402 and ground respectively. It can be appreciated that the substrates of these two MOSFETs are connected to their respective sources. The gate of the third MOSFET, MD3, is connected to the gate of the second MOSFET, MD2. An output terminal 406 is taken from the coupled drains of the third and fourth MOSFETs, MD3 and M2.

The fourth MOSFET circuit 400 further comprises a fifth n-type MOSFET, MR2, arranged in series between the input terminal 402 and ground, with a sixth, n-type,

MOSFET, MD4. The substrate of the fifth MOSFET, MR2, is coupled to its source.

The source of the fifth MOSFET and the drain of the sixth the MOSFET are coupled together. The fourth MOSFET, M2, is arranged, or operable, as a current mirror to mirror the current flowing through the sixth MOSFET, MD4. The current flowing through the fifth MOSFET, MR2, is IMR2- It can be appreciated that the gate of the fifth MOSFET, MR2, is coupled to a third input terminal 408 for receiving a respective operand. It can be appreciated that in the embodiment shown the respective operand is V_CDC- The current, IMI, of the first MOSFET transistor, Ml, and the current, IM_R2S of the fourth MOSFET transistor, M2, influence the output current, lour, since the third MOSFET, MD3, mirrors the current of the second MOSFET,

MD2.

It can be appreciated that the fourth MOSFET circuit 400 comprises three input terminal 402, 404 and 408 for receiving operands Vm, Vc and V_CDC respectively, that are connected to the sources of MOSFET transistors MD2, MD3 and MR2, and the gates of MOSFET Ml and MOSFET MR2 respectively.

The operation of the fourth embodiment, that is, the fourth MOSFET circuit 400, can be explained in terms of the following equations

I_m = a[(V_c - V_T) . V_x -V_x ² /2] (11)

I_MR2 = ocWc_Dc -V_τ -V_T) * V_X -V_x ² /2] (12) ^■'^■ OUT ~ *M\ ~ *MR2

= ct[(V_c -V_CDC -V_τ) . V_x]

= φ_sis -v_τ)»v_x

= Φ _sig -V_T) ^V_m -V₁₇,) (13)

It can be appreciated that the output current, lour, of equation (13) is very similar to the output current defined by equation (5) and can be used for the same purposes or substantially the same purposes.

Similarly, a MOSFET circuit 500 according to a fifth embodiment, as shown in figure 5, operates in a substantially similar manner to the embodiment corresponding to equation (9), that is, to the third embodiment shown in figure 3. It can be appreciated that the current, IM2A, of MOSFET M2RA and the current, I_MIA, of MOSFET MlA, produce a similar result to the corresponding currents in equation (9), where V_IND_C replaces VIN in equations (11) and (12). Therefore, the operation of the MOSFET circuit 500 according to the fifth embodiment can be described by the following equations:

I OUT ⁼ * Ml ~ *M2 ⁺ ^MlA ~ *M1A

⁼ \*M\ ~ ^M2J ~ \*M\A ~ *M2A) = <y_sig -V_T) * (V_m - V_JF) - a{y_sig -V_T) * {V_imc -V_TP) (16)

It can be appreciated that the expression for the output current, I_OU_T, of the MOSFET circuit 500 according to a fifth embodiment, has the same properties as equation (9) relating to the third embodiment. Therefore, the fifth MOSFET circuit 500 can also be used to implement signal processing functions, neural networks, tuneable filters, polyphase filters, programmable transconductance amplifiers, analogue decoding elements such as, for example, soft gates, etc.

It will be appreciated that further embodiments of the present invention are described in the technical papers included in the appendix of the present application. Those technical papers are incorporated expressly into the application. It will be appreciated that embodiments of the present invention progressively reduce, and, preferably, remove undesirable currents from the output current until a desired degree of arithmetical accuracy is attained.

Although the above embodiments have been described with reference to nMOS transistors operating in the triode region, embodiments can equally well be realised in which pMOS transistors are arranged to operate in the triode region. In effect, the n- type and p-type transistors in the above-described embodiments are switched for p- type and n-type transistors.

The reader's attention is directed to all papers and documents which are filed concurrently with or previous to this specification in connection with this application and which are open to public inspection with this specification, and the contents of all such papers and documents are incorporated herein by reference.

All of the features disclosed in this specification (including any accompanying claims, abstract and drawings) and/or all of the steps of any method or process so disclosed, may be combined in any combination, except combinations where at least some of such features and/or steps are mutually exclusive.

Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise. Thus, unless expressly stated otherwise, each feature disclosed is one example only of a generic series of equivalent or similar features.

The invention is not restricted to the details of any foregoing embodiments. The invention extends to any novel one, or any novel combination, of the features disclosed in this specification (including any accompanying claims, abstract and drawings), or to any novel one, or any novel combination, of the steps of any method or process so disclosed. APPENDIX

A new CMOS analogue four quadrant multiplier based on voltage-controlled conductance

I. S. Han

A new CMOS analogue four quadrant multiplier based on the compensated voltage- controlled conductance in the triode region is introduced. The multiplier requires four linear transconductors whose two transistors in the triode region feature the linear voltage-controlled conductance. The simulation results exhibit that the total harmonic distortion is less than 0.1% with the differential input range of 0.6 Vpp and a supply voltage less than 1.8 V.

Introduction: A CMOS analogue multiplier has been used for various analogue signal processing, such as modulators, filters, or neural networks. There are many CMOS multiplier circuits, which operate in the various modes of saturation, triode, weak inversion and others. The recent development of low voltage operation inspired the low voltage CMOS implementation with transistors operating in the triode region [I]. Some circuits are based on the transconductance amplifier operating in the saturation region [2]. For a low-voltage CMOS multiplier, the transconductance based on the triode region was also introduced [3]. In this letter, a CMOS transconductor with the voltage-controlled conductance of transistors in the triode region is proposed. The proposed transconductor is used to implement the new analogue four quadrant multiplier.

Circuit description: The voltage-controlled linear conductance by two parallel connection of triode-regionMOS transistors can be utilised for a transconductor [4]. In the case of two triode-region MOS transistors in Fig. l(a), the current of each half may be expressed as

ID₁ = £ [(V_GS- VTH)V_DS- V_DS ²/2]

ID₂ = K [(VC - VTH)V_DS- V_DS ²/2] (1)

where

K= μ_egC_oχW/(2L) parameter of MOS transistor

VGS

± ΔVi_n gate-source voltage of transistor Ml

VDS ⁼ VC-VBIAS drain-source voltage of transistors

Vc control voltage for the conductance

ViπDc DC offset of input voltage the index of 1 and 2 refers to MOS transistor Ml and M2.

Generally, the DC voltage of VBIA_S in Fig- l(a) or Fig. l(b) maintains the triode- region operation of M2. If neglecting the non-ideal characteristics of VBIAS depending on the variation of Vj_nDc or Vc in Fig. l(b), the I-V characteristics of transconductor circuit in Fig.l is determined by

ID = IDI + ID₂=^(VGS-VBIAS-IVTH)(VC-VBIAS) (2)

The 2ⁿ order distortion in (1) can be removed by the matched pair of Ml and M2 in the triode region. The triode-region transconductor in Fig. 1 (b) is proposed as a simplified CMOS transconductor circuit using a saturation-region transistor M3 in diode connection as a voltage source of VBIAS- In this letter, a new analogue multiplier is discussed for the linear characteristics, which is based on the parallel combination of a basic structure of transconductor in Fig. 1 (b). The basic principle of proposed CMOS transconductor is to output the differential current to improve the non-ideal characteristics of the circuit in Fig. l(b), by connecting two of them as in Fig. 2.

Two basic transconductors operate in the same condition, other than the small signal of differential input applied to the gate of Ml and MlA in Fig. 2. Therefore the gate-source voltage of M3 and M3A is almost same, though the small signal amplitude can cause a slight difference. The transistor pairs of (M3, M4), (M3A, M4A), and (M5A, M5) act as current mirrors. M4 copies the drain current I_D3 of M3 and M5 copies the drain current of M5A, which is equal to the drain current I_D3A of M3 A. Based on (2), I_D3 and I_D3_A can be given by

ID₃=G¹W(VGSI- VBIAS- 2V_TH)

VBIAS- 2V_TH) where V_Gsi=Vi_πDC+ΔV_m/2 gate-source voltage of transistor Ml

gate-source voltage of transistor MlA

Gm=K(Vc-VBiAs) Therefore, the output current lour is lour = Gm ΔVin (3) where I_OUT = ID3 — I_D3A- The compensation of dc components is achieved except Gm.

The analysis of (3) is also based on the triode-region operation of transistors Ml, M2, MlA, and M2A. For the transistors Ml and MlA, the triode-region condition of ( V_GS - V_TH ≥ V_DS ) yields the dynamic range of input signals. Hence, the circuit dynamic range is approximately

V_in>Vc-V_BIAS where V_BIA_S is larger than V_TH of M3 and M3A, Vc is larger than VTH of M2 and

M2A.

Simulation results: The transconductor in Fig. 2 was simulated in HSPICE using typical 0.18 μm CMOS process parameter and a Level 49 model. The simulation result in Fig. 3 shows the analogue two quadrant multiplication, with the sinusoidal differential input signal Vj₃₁ and the linearly increasing voltage Vc. The transistor sizes used for the simulation result of Fig. 3 are W₁ZL₁ = W₂ZL₂ = W₁AZL₁A = W_2AZL₂A = 0.9μm Z7.2μm, W₃ZL₃ = W₄ZL₄ = W₃AZL₃A = W_4AZL_4A = 3.6μm Z0.36μm, W5ZL5 = W5AZL5A = 1.8μm Z0.36μm. The input Vj_n is 10 MHz sinusoidal signal

( 1.25V ± 0.2V) and the voltage V_c is 1.2V± 0.2V.

From (3) and the output current in Fig. 3(b), the four quadrant analogue multiplier can be implemented by compensating the dc component of Vc. The circuit in Fig. 4 is a parallel connection of the circuit in Fig. 2. Hence, the output current of each module can be given by

lour. ⁼ £(Vjny -VBIAS) AVh_1x

Ioυτ2 = K(V ^y- -VBIAS) C-AVH_1X) (4) where AV_h1x= V_111x -V_JI1X-

V_11Jy= Vj_nyDC +ΔVi₁₁y Z2

VinyDC " AVjny Z2.

Therefore, the output current lour of Fig. 4 is lour =iTAV_inxAV_iny (5) where lour ⁼ Iouri + Iouπ- The new CMOS analogue four quadrant multiplier in Fig. 4 is verified with HSPICE simulation result in Fig. 5. The linearity is evaluated by Total Harmonic Distortion (THD), which is less than 0.1 % with the differential input range of 0.6 Vpp and the whole simulation is based on ± 1% mismatch of gate geometry for all transistors of the circuit in Fig. 4. The overall power consumption is evaluated as 200 μW.

THD 0.03% of the proposed circuit is lower than the 0.25% of the circuit [2] in saturation region with the input amplitude of 20OmV. THD 0.1% of the proposed circuit is also lower than the 0.2 % of the circuit [1] in triode region with a 10MHz sinewave input amplitude of 0.6 Vpp .

Conclusion: A new voltage-controlled linear transconductor by CMOS transistors and analogue four quadrant multiplier is proposed and their performance is verified by HSPICE simulation. The circuit exhibits the low distortion, the low power and low voltage operation.

References

1. RAMIREZ-ANGIJLO, J., THOUTAM, S., LOPEZ-MARTIN, A., and CARVAJAL R. G.: 'Low-voltage CMOS analogue four quardrant multiplier based on flipped voltage followers,' Electronics Letters, 2003, Vol. 39, No. 25, pp. 1171-1172

2. Maundy, B., and MAINI, M. r 'A comparison of three multipliers based on V_gs ² technique for low voltage applications', IEEE Trans. Circuit & Systems-I, 2003, 50, (7), pp. 937-940 3. COBAN₅ A. L. and ALLEN P. E. : 'Low-voltage, four quadrant analogue CMOS multiplier', Electronics Letters, 1994, Vol. 30, No. 13, pp. 1044-1045

4. HAN, I. S., and PARK. S.B.: 'Voltage-controlled linear resistor by two MOS transistors and its application to active RC filter MOS integration,' Proceedings of the IEEE, 1984, 72, (11), pp. 1655-1657

Authors' affiliations:

I. S. Han ( Department of Electronic and Electrical Engineering, University of Sheffield, Mappin Street, Sheffield, Sl 3JD, United Kingdom) E-mail address : i.s.han@sheffield.ac.uk

Figure captions:

Fig. 1. Basic circuit configuration of CMOS transconductors based on the voltage- controlled conductance of triode-region MOS transistors.

Fig. 2. Transconductor by a parallel combination of basic element in Fig. 1 (b), W₁ZL₁ = W₂ZL₂ = W_1AZL_1A = W_2AZL_2A = 0.9Z7.2, W₃ZL₃ = W₄ZL₄ = W_3AZL_3A = W_4AZL_4A = 3.6/036, W₅ZL₅ = W₅AZL₅A = 1.8Z0.36 (unit μm).

Fig. 3. Simulated output current of transconductor in Fig. 2 (a) Vj_n; 0.4 VPP at 10MHz and Vc; 1.2 V ± 0.2V (b) the output current lour of multiplying function.

Fig. 4. Proposed analogue four quadrant multiplier, W₁ZL₁ = W₂ZL₂ = W₁AZL_1A = W₂AZL₂A = W_1BZL_1B = W_2BZL_2B = WicZLic = W_2CZL_2C = 0.9Z7.2, W₃ZL₃ = W₄ZL₄ = W_3AZL₃A = W_4AZL_4A = W_3BZL₃B = W_4BZL₄B = W_3CZL_3C = W_4CZL_4C = 3.6Z0.36, W₅ZL₅ = W_5AZL_5A = W₅BZL₅B = W_5CZL_5C = 1.8Z0.36 (unit μm).

Fig. 5. Simulation results of analogue multiplier circuit of Fig. 4. (Vi]_3x andVj_nx- are 10 MHz sinusoidal differential signals of ± 0.2 V : Vi_ny and V_my. are 0.5 MHz triangular signals of ± 0.2 V)

Fig. 1

(a) (b)

Fig. 2

Fig. 4

Fig. 5

Biologically plausible VLSI neural network implementation with asynchronous neuron and spike-based synapse

L Song Han Dept. of Electronic and Electrical Engineering University of Sheffield Sheffield, Sl 3JD, ILK E-mail i.s.han(5),sheffield.ac.uk

Abstract -This paper describes a new asynchronous spike This paper introduces the new development in analog-mixed based neural networks VLSI implementation, inspired by neural networks VLSI with advantages of up-to-date advanced the biological plausibility and low power requirement. The fabrication technology and fiilly asynchronous spike operation. voltage-controlled linear conductance produces the synaptic function of multiplication, weight programming, and H. ANALOGUE-MIXED NEURAL NETWORK VLSI BY summation of synaptic spike currents for the neuron. The CONTOROLLED CONDUCTANCE OF MOSFET operation speed of synaptic computation is up to 300 Mega operations with a small power consumption of 33 The current-voltage (I-V) relationship of MOSFET in the microwatts. The overall power consumption can be less in Mode region could be clearly adopted to implement electronic real applications, as individual synapse only consumes the synapses, as it provides a multiplication for the basic function of power when there is an active neural input. neural computation. The equation of interest is that the

The neuron is based on multiple combinations of drain-source current I_DS for a MOSFET in the linear or triode synapses and the HSPICE simulation demonstrates the region: asynchronous spike behavior of mtegration-and-Sring ™ith a refractory period. The advantages of asynchronous I_DS = α [(V_GS-V_T)V_DS-V_DS ²/2] (1) operation, removal of reference clocks, and low voltage operation are exhibited compared to previous pulse-based Here, α is the MOSFET process parameter, VQ_S, V_DS, V_T, the analogue-mixed neural networks VLSI. transistor gate-source, threshold and drain-source voltage

The asynchronous spike based neural networks in 0.18μm respectively. To achieve a linear voltage multiplier (V _Gs-to-Vos), CMOS VLSI technology is proposed to provide the it is necessary to remove the second order term of V_Ds ² /2 in eq.1. advantage of analogue-mixed neural network VLSI with Though there are various methods for such controlled linearity, small power consumption and no need for a synchronous there are limitations in utilizing the most advanced VLSI operation. technology. An example is an issue of a synapse circuit of Fig. 1, as it suffers from the advanced CMOS process of low voltage operation such as 3.3V or less. Though the circuit of Fig. 1 has the accuracy for real-time packet switch control, it is necessary to improve analogue-mixed neural network VLSI, as the

I. INTRODUCTION fabrication technology advances towards 0.18μm or 0.13μm.

Biologically inspired neural networks, i.e. spike-based operation is widely investigated in various areas, from robots to regenerative medicine [1-5]. The pulse or spike based implementation of neural networks has advantages of VLSI neural networks, which is suitable for the large scale real-time or embedded requirement [6-7]. In the brain science, a large scale and general neural network VLSI has been also expected for cognition [8]. The advantage of analog VLSI is low power consumption or larger networks integration, though digital VLSI has advantages of design flexibility or leading-edge technology. Issues in analog neural network VLSI are the accuracy problem, or the complexity, in comparison with the digital. Hence the current demand in large capacity or biological performance

drives analog-mixed neural networks, improving the accuracy or Fig. 1. Spike-based synapse circuit with its measured linearity, flexibility by pulse/spike based operation. based on controlled conductance. AND SCALE

is added for as in Fig.3. A buffer stage current source with a flexible capability of current offset terms in eq (4), source I_MS. One of of inhibitive synaptic source. The synaptic flow of input. An an output

The drain-source current of M3 represents the summation of Fig. 3. A reference synapse circuit using the controlled

synapse, though one are as low power threshold

Fig.4. New synapse circuit of balanced structure. A current mirror stage (M9, MlO) is the only added one to a advantages based on H-H formalism, although the asynchronous reference synapse of Fig, 3. It enables the output of the neural spikes inspire all sorts of neuromorphic implementation. difference of two current components, i.e. the drain-source Asynchronous neuron spikes or pulses are also considered as a current of M3 and M3A. The output current I_Ouτ in Fig. 4 is key element in high level cognition [13]. Hence, asynchronous derived as in the following dynamics of the H-H formalism is adopted as a reference model for a neuron compatible to the synapse of Fig.4.

of a neuron. of eq (3) can implement the An empirical conductance

membrane potential. V_n, is a is well illustrated in Fig.7 [14].

in the H-H

Fig.5. Synapse operation over 300 Mega pulse/spike inputs per Sum- second.

Neurons: The biologically motivated neuron has been targeted as a neuron model for its advantages of integrating various applications [9-12]. The feature of asynchronous firing or spike dynamics emerges as a critical issue, as such electrical signals seem to be essential to the neural information processing in biological unit. The Hodgkin-Huxley(H-H) formalism is widely adopted for its biophysical characterization and dynamics. An electrical equivalent circuit model of Fig. 6 is

known as an empirical model by the H-H formalism, which describes quantitatively the dynamics of the voltage-dependent Fig. 8. A H-H based neuron block diagram by voltage-controlled conductance. Most of recognition tasks do not exhibit any major conductance. The synapse of Fig.4 has a property of sum-multiplier as shown required synaptic stimulus current for a single firing can be in eq (5). The block diagram of asynchronous neuron of Fig. 8 is decided by the desired behavior of neuron, where the 0 IpF Q_n is inspired by controlled conductance from H-H model and simulated in Fig. 11. The 2 Mega spike stimulus per second is sum-multiplier of spike-based synapse implementation simulated, though elements of synapse circuit for a neuron can operate in much higher speed as in Fig. 5. The HSPICE

The differential equation in eq (6) is implemented by 1 ^Λ order simulation result demonstrates the asynchronous behavior of low pass filter, which induces a delayed response. The neuron of integration-and-firing with a refractory period. There are the Fig. 8 models three components of ionic conductance and its advantages of asynchronous operation, removing reference simulation result in Fig. 9 exhibits matching behavior to H-H clocks, and low voltage operation compared to the previous model. pulse-based analogue-mixed neural networks VLSI [15-16].

circuit with neuron conductance spikes as input,

neuron scale, low VLSI, for its simpler based on the controlled its characteristics of speed, can be tailored by

wire-OR expansion. For a applied to a neuron as in Fig. 11. The value of capacitor or the higher speed, the accuracy or the fast settlement is observed with 13. J. Taylor, Private communication, 2004 increased current levels in a controlled conductance device.

14. M. Hausser, "The Hodgkin-Huxley theory of action

The core of proposed electronic synapse and neuron are potential", Nature neuroscience supplement, vol. 3, pp. 1165, estimated respectively as within the size of 4μmX4μm and Nov, 2000 lOμmXIOμm. The asynchronous spike based neural networks in 0.18μm CMOS VLSI technology provides the advantage of 15.1. Han and R. Webb, "Neural Network Switch Controller analogue-digital mixed neural networks VLSI with small power with Analogue-Digital Mixed Neural Network VLSI," consumption and no need for a synchronous operation. Proceedings ofEANN'97, pp. 299-302, 1997

16.1. Han, "Neural network VLSI implementation and its

REFERENCES applications," Korea Telecom Journal, Vol.2, No.l, pp. 12-20, 1997

1. R. Eckmiller, O. Baruth, and D. Neumann, 'TSfeural information processing efforts to restore vision in the blind", 17.1. Han, "Signal processing circuit," U.K patent pending, Proceedings of ICONIP 2004, LNCS 3316, pp. 10-18, 2004 P105608GB, 2004

2. N. Taylor, M Hartley, and J. Taylor, "The learning of insertions by the Cerebellum", Proceedings of BICS 2004, Sterling, CNS 2.2, 2004

3. T. Pearce et al, '"Silicon-based neuromorphic Olfactory pathway implementation", Proceedings of BICS 2004, Sterling, BIS6.3, 2004

4. D. Floreano and C. Mattiussi, 'Εvolution of spiking neural controllers for autonomous vision-based robots", ER2001, LNCS 2217, pp.38-61, 2001

5. G. Bugmann, "Biologically plausible neural computation", Biosystems, 40, pp. 11-19, 1997

6. 1. Han, "Mixed-signal neuron-synapse implementation for large scale neural network", Proceedings of BICS 2004, Sterling, 2004

7. C. Christodoulou, G. Bugmann, and T. Clarkson, "A spiking neuron model: applications and learning", Neural networks, 15, pp. 891-908, 2002

8. J. Taylor, "Paying attention to consciousness," Progress in Neurobiology, 71, pp.305-335, 2003

9. M. Simoni et al, "A multiconductance silicon neuron with biologically matched dynamics", IEEE Trans. Biomedical Eng, vol. 51, No. 2. pp. 342-354, 2004

10. S. Matinoia et al, "Cultured neurons coupled to micoelectrode arrays: circuit models, simulations and experimental data", IEEE Trans, on Biomedical Eng,\ol.51,No. 5, pp. 859-864, 2004

11. B. Linares-Barranco et al, "A CMOS implementation of FizHugh-Nagumo neuron model", IEEEJSSC, Vol.26, No.7, 1991

12. C. Bartolozzi and G. Indiveri, "A neuromorphic selective attention architecture with dynamic synapses and integrate-and fire neurons", Proceedings of BICS 2004, Sterling, BIS2.2, 2004 Mixed-signal neuron-synapse implementation for large scale neural network

Il Song Han

Dept. of Electronic and Electrical Eng. , University of Sheffield . Sheffield, Sl 3ID, U.K.

Abstract

This paper describes a mixed-signal neural networks VLSI for low power and asynchronous operation. The voltage-controlled transconductance produces the synaptic function of multiplication and summation of synaptic currents for neuron, by compensating the non-linearity of MOSFET resistance in the triode region.

The flexible configuration of synapse accommodates the spike-based neural networks, inspired by the biological plausibility and low power requirement. The neuron with a combination of synapses demonstrates asynchronous spikes of integration-and-firing with a refractory period. The speed of individual synapse is up to 300 Mega operations/sec with the power consumption of less than 33μW₅ using 0.18μm CMOS process.

Key words: Analogue-mixed VLSI neural network, pulse/spike-based neural computation, asynchronous operation, MOSFET resistance, voltage-controlled linear resistance.

1. Introduction

VLSI neural networks has been continuously developed either in digital or analogue, as both methods have different advantages. The advantage of analogue VLSI is low power consumption or larger networks integration, though digital VLSI has advantages of design flexibility or leading-edge technology. Issues in an analogue neural network VLSI can be the accuracy problem, or the complexity, in comparison with the digital. In some special applications, the analogue utilised or developed better its non-ideal characteristic or complex design [1-2].

Recently, biologically inspired neural networks, i.e. spike-based operation is widely investigated in various areas, from robots to regenerative medicine [3-7]. The pulse or spike based implementation of neural networks has advantages of VLSI neural networks, which is suitable for the large scale real-time or embedded requirement [8-9]. IQ the brain science, a large scale and general neural network VLSI has been also expected for cognition [10]. Hence the current demand in large capacity or biological performance drives analogue-mixed neural networks VLSI of pulse/spike based operation, because of its advantages in large scale implementation and low power consumption in comparison to the digital. There exists an issue of accuracy improvement in analogue-mixed implementation, with technology advancement towards 0.18 μm or 0.13μm. An example of analogue-mixed VLSI is in Fig.l, which was developed for real-time packet control [H]. Though the circuit exhibits the accuracy, it demands the complex supply voltages which limit the level of integration under low supply voltages of up-to-date VLSI technology.

This paper introduces the new development in analogue-mixed neural networks VLSI with advantages of up-to-date advanced technology and fully asynchronous spike operation.

2. Analogue-mixed neural network synapses for low power and asynchronous operation

The current-voltage (I-V) relationship of MOSFET in the triode region can be adopted to implement electronic synapses, as it provides the multiplication function.

The equation of interest is that the drain-source current fc_>s for a MOSFET in the linear or triode region:

IDS = α { (V_GS - V_T>V_DS-V_DS ²12} (1) Here, α is the MOSFET process parameter including the geometry, VGS, VD_S, V_T, the transistor gate-source, drain-source and threshold voltage respectively. To achieve a linear voltage (V_Gs)-to-voltage (V_DS) multiplier, the balanced circuit with an operational amplifier can be used to remove the second order term of V_DS ² /2 in eq.l . Though various methods for suppressing or controlling the nonlinearity have been developed, there are limitations in solving the nonlinear problems. The synapse circuit of Fig. 1 is an effective method to compensate such nonlinearity, but it still has the drawback of demanding bipolar supply voltages to remove the second order term in eq.l.

(a) (b)

Fig. 1. (a) Electronic synapse circuit with compensated linear MOSFET resistance in the triode region, and (b) its measured linearity of output synapse current vs weight voltage ( +/- 10 μA with +/-0.5 V).

The synapse circuit proposed in this paper makes also a simple use of the MOSFET in the triode region. It realises the high speed and a small size analogue multiplier without an amplifier or dual supply voltages. The synapse circuit of new MOSFET resistance-based analogue multiplier is shown in Fig. 2 (a). For an efficient analogue multiplication, two terms, Vr V_DS and V_DS ² /2, from eq. 1 should be eliminated from the output. Based on eq. 1 for the MOSFET in the triode region, the currents of transistor Ml and transistor M2 are;

IMI = α ( (V_2-- VT)-V₁-VI ² ^) (2) IM₂ = α ( (V₂₊- V_T)-V₁-V_I ² /2} (3)

Here, V₂₊ and V_2- produce one input of two variables for the multiplication and voltages of V₂₊ and V_2- keep transistors Ml and M2 in the triode region, i.e. both Ml and M2 are operated under the condition of V_GS-V_T>V_DS- VI is the other input of multiplication, which represents the drain-source voltage of Ml and M2. The source voltages of both Ml and M2 remain in common as M3 or M4 acts as a diode. With the help of current mirrors (M3-M8 and M4-M7-M5-M6), the synaptic output current is the difference of I_MI and I_M2 in equations of (2) and (3).

= Di ( V₂₊ -V_2-)V₁

= Ct VwEI_GH_jV_l (4) where V_WEIGHT is the difference of V2₊ and V_2-. One of V₂+ and V₂- can be a DC reference voltage while the other one is a synaptic weight plus DC reference. From eq. 4, the synaptic multiplication of synapse weight (VWE_IGHT) and effective neural input (V₁) is achieved by two MOSFETs operated in the triode region and pairs of current mirrors.

+VDD

INPUT

(a) (b)

Fig.2. (a) New synapse circuit by voltage-controlled linear resistance of two MOSFETs in the triode region, and (b) its representation as differential input and multiplying transconductor.

The summation of post-synaptic current is attained by common-output connection of synapses as each synapse can contribute individual synaptic output current. Both of MOSFET M6 and M8 act as current source by either sourcing or sinking the synaptic output current, and the summation of post-synaptic current is computed by a integration capacitor in each neuron.

The new synapse of Fig. 2 (a) is designed using 0.18μm standard CMOS technology and evaluated by HSPICE simulation. The power supply voltage is 3.3 V and both accurate operation and low power consumption are design objectives. The linearity of synapse circuit is shown in Fig.3 (a) and demonstrates the behaviour of multiplier in eq. (4). One of inputs to Ml and M2 is applied with sinusoidal signal with DC offset, while the other is applied with the same DC offset The output current illustrates the analogue multiplication with amplitude modulation by neural input of triangular signal, though the neural input signal of binary state is enough for general purpose pulse/spike neural networks. The functional description of synapse in Fig. 2 (b) presents the general output characteristic of Fig.3 (a), as a synapse can be used as an element for complex neural signal processing.

The speed of more than 300Mega connection per second inputs is tested as in Fig. 3 (b). The flexibility in power consumption can increase the operation speed further, as it accelerates charging or discharging rate of output current faster. The overall power is less than 33 μW per activated synapse cell and the new circuit is suitable for large scale VLSI neural network implementation. As observed in the circuit of Fig.2 (a), there is no current or power consumption without any active neural input. Without neural input, V₁, I_MI and I_M2 become null and there flows no current in Ml -M3 or M2-M4. Also, there flows no current in M5-M7 or M6-M8, as the source current of current mirror is null. Therefore the synapse cell does not consume any power if there is no activity. The operational principle in eq (4) does not require any timing characteristics of neural inputs, and the synapse circuit of Fig.2 (a) operates either asynchronously or synchronously, free from any synchronous constraints. The power consumption is dependent on the activity of neural input or pulse/spike firing rates of neural networks.

(a) (b)

Fig.3. (a) New synapse circuit's characteristics as a multiplier, a neural input voltage (triangular wave) and synaptic weight voltage (sinusoidal wave) for inputs and modulated current for synaptic output current, and (b) transient characteristics of pulses/spikes operation over 300 Mega connections per second for synapse cell

3. Analogue-mixed and bio-inspired neurons for VLSI

There are different requirements for the VLSI implementation of neurons depending on applications, i.e. asynchronous pulse/spike or synchronous ones. Recent developments of biologically inspired neural networks or neuromorphic solutions are largely based on asynchronous operation, while certain applications like 'real-time packet switch controller' are based on synchronous operation [11]. Either asynchronous or synchronous neuron is available to the synapse of Fig. 2 (a) for mixed-signal neuron-synapse implementation of large scale neural network.

Synchronous neuron: For analogue or analogue-mixed neural network operation, the neural state is represented as the voltage at a neuron capacitor connected to networked synapse outputs of Fig. 2 (a). Every synapse produces the current by multiplication of weight value and input value based on eq. (4), which results a voltage of neural state by the integration of synaptic current in a neuron capacitor.

The circuit in Fig.4 (a) shows the block diagram of a neuron with the linear ramp function and sigmoid-like continuality. At each sampling with CLKl 'high' in Fig. 4 (a), the summation of the synaptic computation as a buffered input voltage is fed to a capacitor for copying the neural state into the neuron. The CLK2 signal operates on that sampled voltage, which is then transformed into the new voltage level for a sigmoid-like function enabled by two MOSFETs in the diode configuration. Two MOSFET as in Fig.4 (a) are for defining the dynamic range of neural states, where one is for the upper bound conversion and the other is for the lower bound conversion. With the reduced gate-to-source voltage during the operation, the sub-threshold conduction delivers the overall conversion close to a continuous sigmoid function as shown in the transfer curve of Fig. 4 (b). The transformed voltage as a neural state is applied to the following part of neuron circuit while CLK3 is 'high' .

(a) (b)

Fig. 5. (a) Analogue-mixed neuron processing circuit and (b) its transfer characteristics

In order to provide the pulse/spike output, the discharging current is subtracted from the neural state of a sampling capacitor Cs when the comparator CON output of Fig. 6 (a) turns the switch MOSFET M2 'on'. The output pulse from a comparator is generated when the voltage at the sampling capacitor is higher than the provided reference level (REF). An AND gate in Fig. 6a encodes the pulse/spike output with system reference clock CLK and PWM output of comparator CON. A MOSFET switch Ml controls the synchronous operation of neuron output. The neuron circuit does not always include the transformation block in Fig. 5 (a)₅ because many applications demand only binary neuron output.

As a neuron circuit in Fig. 6 (a) is not based on any amplifying circuit but only a comparator, diodes, or switches, the total operation speed of neural networks does not degrade the overall performance based on analogue-digital mixed synapse operation of Fig. 2 (a). The complexity of neuron introduces unlikely any issue in synapse-neuron integration as illustrated in Fig. 6 (b) of neuron-synapse chip photograph, where the sampling capacitor Cs of holding neural state occupies larger portion of neuron area. The synchronised operation is required for real-time synchronous applications like a high speed telecommunication switch controller.

(a)

Fig. 6. (a) Pulse/spike based synchronous neuron, and (b) its chip photograph of a mixed-signal neuron-synapse VLSI. Asynchronous neuron: The biologically motivated neuron has been targeted as a neuron model for its advantages of integrating various applications [12-17]. The feature of asynchronous firing or spike dynamics emerges as a key aspect, because such electrical signals seem to be essential to the neural information processing in biological unit. The Hodgkin-Huxley(H-H) formalism is widely adopted for its biophysical characterization and dynamics. An electrical equivalent circuit model of Fig. 7 is known as an empirical model by the H-H formalism, which describes quantitatively the dynamics of the voltage-dependent conductance. The asynchronous neural spikes inspire all sorts of neuromorphic implementation, although most of particular recognition tasks do not exhibit any major advantages based on H-H formalism. Asynchronous neuron spikes or pulses are even considered as a key element in high level cognition [IS]. Hence, asynchronous dynamics of the H-H formalism is adopted as a reference model for asynchronous neuron compatible to the synapse of Fig.2 (a).

The voltage-current relationship of eq (4) can implement the voltage dependant conductance employed in Fig.7. An empirical mathematical formalism models dynamics of each conductance element as

(5) where b is sigmoidal function of the membrane potential. V_m is a membrane potential and the overall dynamic modelled by an action potential and related ionic conductance.

The block diagram of asynchronous neuron in Fig. 8 is inspired by controlled conductance from H-H model and sum-multiplier of spike-based synapse in Fig. 2. The differential equation in eq (5) is implemented by 1 ^ order low pass filter, which induces a delayed response. The neuron of Fig. 8 models three components of ionic conductance in Fig. 7, where KA, ENA, and EL represent EK, E_NA, E_leak respectively and G_L as Gieak in Fig.7. The capacitor of C_membrane is modelled as an integrator for its functional behaviour.

Fig. 8. A H-H based neuron block diagram based on functions of synapse in Fig.2 The MATLAB simulation of H-H neuron block diagram in Fig. 8 shows the result in Fig 9 (a), which exhibits matching behaviour to H-H model in Fig 9 (b).

(a) (b)

Fig. 9. (a) Simulation of Action potential, and (b) an experimental Action potential from ref 19.

The H-H formalism inspired neuron, which is based on the conductance controlled model of Fig. 8 is used to implement a asynchronous silicon neuron in Fig. 10. A further simplification is introduced by an approximation of sigmoidal function and a reduction of one differentiated conductance stage . A differential amplifier with buffer stages (Diff amp in Fig.10) acts as a sigmoidal transfer function for the steady state activation variable, where a reference voltage is introduced for the half-activation potential. Two synapses of SYNAPSE2 and SYNAPSE3 in Fig. 10 (a) realise the approximation of two voltage-controlled conductances for the smaller chip area without major drawbacks.

A low pass filter is implemented by one synapse of SYNAPSEl and a capacitor CL, as a synapse of Fig.2 is equal to an operational transconductance amplifier from eq (4). It is equivalent to a RC low pass filter, where the resistance R can be programmed by a control voltage (SETbias in Fig. 10). The control voltage (SETbias) corresponding to V_WEIGHT of eq. (4) can be used to control the bandwidth of a low pass Filter, otherwise remaining as a fixed one. The linearity of low pass filter is -45dB in harmonic distortion and tuneable up to 50% by a control voltage.

(a) (b)

Fig. 10. (a) Asynchronous spike firing neuron by three synapses of Fig. 2, inspired by H-H model, and (b) asynchronous behaviour of a neuron circuit with synaptic spike currents as inputs : (from top to down) neuron capacitor's potential as a membrane potential, synaptic current spikes as input, and firing pulses with the refractory period The neuron of Fig. 10 (a) shows a membrane dynamic behaviour consistent to the simulated result in Fig. 9 (a). Instead of single stimulus as a single firing in Fig.9, the current spike stream is applied to the simulation of asynchronous neuron as in Fig. 10 (b). The 2 Mega spike stimulus per second is simulated as in Fig. 10 (b), though elements of synapse circuit for a neuron can operate in much higher speed as in Fig. 3. The HSPICE simulation result demonstrates the asynchronous behaviour of integration-and-firing with a refractory period. There are advantages of asynchronous operation, removal of reference clocks, and low voltage operation compared to previous pulse-based analogue-mixed neural networks VLSI [20]. Another advantage of H-H formalism inspired silicon neuron in Fig. 10 (a) is its fast operation speed in Fig. 10 (b), which may demand substantial computing otherwise [21].

IV. CONCLUSION

The new spike-based synapses and asynchronous neuron demonstrate the feasibility of large scale, low power and asynchronous neuron-synapse VLSI implementation, for its simpler circuit, speed and biological plausibility based on the controlled conductance. It is flexible to adapt its characteristics of speed, power, accuracy on demand, as those can be tailored by controlling the operating synapse current and parallel wired-OR expansion. For an example, the higher operation speed, or the fast settlement is observed with increased operational current levels in a synapse element. The overall power consumption can be less in practice, as individual synapse only demands the power when there is an active neural input signal.

The core of proposed electronic synapse and neuron are estimated respectively as within the size of 2μmX2μm and 6μmX6μm in 0.18μm CMOS VLSI technology, though the overall neuron area is finalised by the desired average firing frequency. The asynchronous spike based neural networks in advanced CMOS VLSI technology provides the advantage of analogue-digital mixed neural networks VLSI with small power consumption and no need for a synchronous operation.

References

1. Y. Horio, K. Aihara, and O. Yamamoto, Neuron-Synapse IC Chip-Set for Large-Scale Chaotic Neural Networks, IEEE Trans on Neural Networks, vol. 14, no 5, Sept, 2003, ppl393-1404

2. M. Milev and M. Hristov, Analog Implementation of ANN With Inherent Quadratic Nonlinearity of the Synapses, IEEE Trans on Neural Networks, vol. 14, no 5, Sept, 2003, ppll87-1200

3. R. Eckmiller, O. Baruth, and D. Neumann, Neural information processing efforts to restore vision in the blind, Proceedings of ICONIP 2004, LNCS 3316, 2004, pp. 10-18

4. N. Taylor, M Hartley, and J. Taylor, The learning of insertions by the Cerebellum, Proceedings of BICS 2004, Sterling, 2004, CNS 2.2

5. T. Pearce et al, Silicon-based neuromorphic Olfactory pathway implementation, Proceedings of BICS 2004, Sterling, 2004, BIS6.3

6. D. Floreano and C. Mattiussi, Evolution of spiking neural controllers for autonomous vision-based robots, ER2001, LNCS 2217, pp .38-61, 2001

7. G. Bugmann, Biologically plausible neural computation, Biosystems, 40, pp. 11-19, 1997

8. 1. S. Han, Mixed-signal neuron-synapse implementation for large scale neural network, Proceedings of BICS Sterling, 2004, NC4.2 9. C. Christodoulou, G. Bugmann, and T. Clarkson, A spiking neuron model: applications and learning, Neural networks, 15, 2002, pp. 891-908

10. J. Taylor, Paying attention to consciousness, Progress in Neurobiology, 71, 2003, pp.305-335

IL L S. Han and R. Webb, Neural Network Switch Controller with Analogue-Digital Mixed Neural Network VLSI₃ Proc. ofEANN'97, 1997, pp.299-302

12. M. Simoni et al, A multiconductance silicon neuron with biologically matched dynamics, IEEE Trans. Biomedical Eng, vol. 51, No.2, 2004, pp. 342-354,

13. S. Matinoia et al, Cultured neurons coupled to micoelectrode arrays: circuit models, simulations and experimental data, IEEE Trans, on Biomedical Eng, vol. 51, No. 5, 2004, pp. 859-864

14. B. Linares-Barranco et al, A CMOS implementation of FizHugh-Nagumo neuron model, IEEE JSSC, Vol.26, No.7, 1991, pp956-965

15. C. Bartolozzi and G. Ltidiveri, A neuromorphic selective attention architecture with dynamic synapses and integrate-and fire neurons, Proceedings of BICS 2004, Sterling, 2004, BIS2.2

16. E. M. Izhikevich, Which model to use for cortical spiking neurons?, IEEE trans. Neural Networks, vol. 15, No. 5, 2004, ppl063-1070

17. T. Asai et al, A subthreshold MOS neuron circuit based on the Volterra system, IEEE trans Neural Networks, Vol. 14, No.5, 2003, pp.l308-1312

18. J. Taylor, Private communication, 2004

19. M. Hausser, The Hodgkin-Huxley theory of action potential, Nature neuroscience supplement, vol. 3, Nov, 2000, pp. 1165

20. 1. S. Han, Neural network VLSI implementation and its applications, Korea Telecom Journal, Vol.2, No.l, 1997, pp. 12-20

21. U. Seiffert, Artificial neural networks on massively parallel computer hardware, Neurocomputing 57, 2004, pρl35-150

A Novel Tunable Transconductance Amplifier Based on Voltage-controlled Resistance by MOS

Transistors

Il-Song Han , Member, IEEE

triode-region MOS transistors in Fig. l(a), the current of each half may be expressed as

Abstract — A new tunable transconductance amplifier is proposed for the programmable analog signal processing or Im = K [(VoS-Vm)V_ES- V_DS ²/2] low power filter applications. The transconductor I_D2 = K [(V_c -V^)VD_S- V_DS ²/2] linearization is based on the compensation of nonlinear behaviour by two MOS transistors. The transconductance amplifier in this paper exhibits the good common-mode where dynamic range and the voltage-controlled transconductance. K- U_efiC_0XW/(2L) parameter of MOS transistor HSPICE circuit simulation using O.lSμm standard CMOS V_GS ⁼Vi_αDc ± ΔVi, gate-source voltage of transistor Ml technology shows the ±50% tunable transconductance range V_DS ⁼ V_C-V_BIAS drain-source voltage of transistors with the ± 0.2V control voltage, and the linearity of less than Vc transconductance tuning voltage 6OdB in the total harmonic distortion for the 0.6 VPP input VM)_C DC offset of input voltage signal. the index of 1 and 2 refers to MOS transistor Ml and

Index Terms — Analog MOS integrated circuits, tunable M2. amplifiers, tunable filters, voltage-controlled conductance Generally, the DC voltage of V_BIA_S maintains the triode-region operation of M2, but if the circuit

I. INTRODUCTION implementation has the non-ideal characteristics of V_BIA_S,

ΠPHE transconductance based on MOS transistors has the V_DS will be sensitive to the variation of V^c or Vc- JL been widely used in the analog signal processing or Hence neglecting such dependence, the Gn? parameter of VLSI circuits such as programmable amplifiers, tunable the circuit in Fig.l(a) is determined by filters, oscillators, neural networks and others. Various methods were reported to improve the linearity of Gm=( 9I_D/ 6V_GS ) _ΔVill=o « K(V_C-V_B1AS) (1) transconductance, by utilizing MOS transistor's characteristics hi the triode region or the saturation region. where Recently new improvements on transconductance ID = Im +

amplifiers based on cascode types or degeneration types Therefore, the ratio of GWJZI_D is were presented hi [1], [2], for extending the linearity of transconductrors and low voltage operation in the Mode GWZID(AVm=_O)=IZ(VGS^VBIAS-SVTH) (2) region. The MOS transistors in the triode region can be exploited for transconductors as in [3], though it is The analysis of (1) and (2) shows that the circuit in Fig. necessary to eliminate the nonlinear characteristics. Many l(a) has the similar characteristics of general cascode cascode or source degeneration topologies were proposed transconductors, with typical parameters and to improve the distortion mechanism of 2^nd order effect or approximation of V_BIAS ⁼ Vn₅. The 2^nd order distortion can 1he mobility degradation. The triode-region transconductor be removed by the matched pair ofMl and M2 in the triode with regulated cascode circuit was well known for its region. The issues of the mobility degradation or variation linearity in. [1], [4], [5], [6], and its application to the of bias voltage can cause the weak nonlinearity for a tunable filter was introduced in [7]. For tunable filters, the tunable transconductance amplifier. The triode-region conductance based on triode-region MOS transistors can transconductor hi Fig. 1 (b) shows the simplified version of be utilized in other ways as in [8], [9]. As the idea of the CMOS transconductor using a saturation-region parallel operation of triode-region and saturation-region transistor M3 in diode connection as a voltage source of MOS transistors demonstrates the possibility of VBI_AS- In this paper, a new transconductance amplifier is transconductors using the cascode topology in [1], [3], [4], discussed for the linear characteristics and the tunable [5], the linear conductance by two parallel connection of transconducatnce, and is based on the parallel combination triode-region MOS transistors can be utilized for the of a basic structure of transconductor hi Fig. 1 (b). transconductance amplifier design. In the case of two π. TRANSCONDUCTANCE CIRCUIT DESCRIPTION The basic principle of proposed transconductor is to the simulation result of Fig.3 are Wj/Li = W₂TL₂ = Wi_A/Li_A~ output the differential current to improve the nonlinearity - W_2AZL_2A= 0.125, W₃/L₃ = W4/L4 = W_3A/L_3A = W_4A/L_4A = problem of the circuit in Fig. l(b), by connecting two of 10, W₅ZL₅ = W_5A/L_5A = 5. The input V_1n is 10 MHz them as in Fig. 2. The term of V_BIAS in (1) is implemented sinusoidal signal ( 1.25V + 0.2V) and the tuning voltage V_c as the gate-source voltage of M3, while the drain current of is 1.2V± 0.2V. To avoid possible problems with the tuning M3 is dependent on the channel resistance of Ml and M2 voltage as the whole supply voltage, the output buffer in and the square of its own gate-source voltage. The Fig. 4 is added for a complete transconductance amplifier. deviation of V_BIAS from the expected voltage is undesirable The transistors M6-M9 hi Fig.4 are added for the output but presents due to the mobility reduction effect [5] or the circuit to be independent of the tuning voltage V_c, and the nonlinear characteristic of M3 in diode connection. The output circuit can be flexible to the application transconductor in Fig. 2 compensates such variation by requirements such as the current level or dynamic range. introducing the differential mode operation for the output, An additional balanced output current can be available as the variation of V_BIA_S is almost common between both from the other transconductor element, by adding the elements of basic transconductor cell. equivalent output circuit. The transistor pairs of (M5, M6)

Two basic transconductors operate in the same condition, and (M7, M8) are introduced to copy the current in M3 or other than the small signal of differential input applied to M4 to the output with the supply voltage of V_DD. The the gate of Ml and MlA in Fig. 2. Therefore the current in M3A or M4A is mirrored to the transistor M9, gate-source voltage of M3 and M3A is almost same, which contributes to the differential output current I_Ouτ as in the Fig. 4 simulation are shown in Table I mismatch of gate circuit in Fig. V, and total the input of 0.6 can of be varied from IV to

gate-source voltage of transistor the maximum signal level hi the circuit like 1.5V. The

MlA overall power consumption is lOOμW and the noise

Gm=K(Vc-VBiAs) characteristics can be improved by increasing the operation Therefore, the output current I_Ouτ is current by the increased conductance, though the simulation result shows the total output noise voltage of

I_OUT = G/w ΔV_i]1 (3) 300μV over IKHz to lOOMHz.

The tunable transconductance range is about ± 50%, by where I_Ouτ = ID3 ~ID3A. applying the control voltage V_c of 1 V i 0.2V. The simulation result of the tunable function is illustrated hi

The analysis of (3) is also based on the triode-region Fig.5, by its application to a simple active RC filter in GmC operation of transistors Ml, M2, MlA, and M2A. For the implementation, as illustrated in Fig, 6. By changing the transistors Ml and MlA, the triode-region condition of control voltage V_c in the range of (0.8V to 1.2V), the ( V_GS - V_TH > V_DS ) yields the dynamic range of input cut-off frequency is programmable from 1 MHz to 2.4 signals. The V_BIAS; the gate-source voltage of M3 or M3A MHz, which demonstrates the tunable function hi the GmC may satisfy the triode-region of M2 or M2A, while M3 or filters. The power supply is 1.8 V throughout the HSPICE M3A operates hi the saturation region with the tuning simulations, though the performance is not much degraded voltage V_c. Hence, the circuit dynamic range is until reducing the supply voltage to the range of tuning approximately voltage.

In Table II for the comparison, the proposed tunable

V_jn> Vc-V_BIAS transconductance amplifier can meet the linearity requirement and the tunable function, though the where V_BIA_S is larger than Vn₁ of M3 and M3 A, V_c is larger advantages or constraints are dependent on the particular than VXH of M2 and M2A. The transconductor in Fig.2 was application or process parameters. simulated in HSPICE using typical 0.18 μm CMOS process parameter and a Level 49 model. The simulation m. CONCLUSION result in Fig. 3 shows either characteristics of tunable The transconductance amplifier based on triode-region operational transconductor or analog multiplication, with MOS conductance is proposed for the tunable analog the sinusoidal differential input signal Vj_n and the linearly circuit implementation, with the advantage of low power increasing tuning voltage Vc- The transistor sizes used for consumption and tuning flexibility. The twin configuration

of basic element in Fig. W₃/L₃ = W₄ZL₄ = = 5.

(a) Vk; 0.4 lour.

using vs. control MHz (1.2V).

Fig.4. Proposed tunable transconductance amplifier, W1/L1 = W2/L2 = MAIN PARAMETERS OF DIFFERENT TEANSCONDUCTANCE AMPLIFIERS WW/LIA= W₂AZL₂A = 0.12₅, W₃ZL₃ = W4/L4 = W_3A/L₃A = W₄A^L₄A = W7/L₇ = W₈ZL₈ = 10, W₅/L5 = W_SA/L_SA = Wβ/Ls = W9ΛL9 = 5.

TABLE I

SIMULATED LINEARITY OF PROPOSED CIRCUIT IN FIG.4.

Vin

(a) V₁ inDC [1] Parameters are based on the 3rd-order filter, i.e. the power consumption is divided by the number of transconductance amplifiers.

[2] The 2™¹ order harmonic distortion component, which is the main cause of nonlinearity at the middle point of tuning voltage.

[3] The total harmonic distortion of 1% is accepted as the nominal linearity.

[4] The tuning range of Butterworth 3^rd-order filter's cut-off frequency.

Claims

1. A method of analogue signal processing using non-linear devices operable in respective triode regions (operable with non-linear current-voltage relationship); the method comprising the steps of producing an input current

using respective currents of a pair of the non-linear devices, the input current having a linear current-voltage characteristic with respect to a selected parameter (VD_S) of the pair of non-linear devices; the input current comprising a first portion (α(Vc.Vχ)) representing an arithmetic operation between first (Vc) and second voltages (Vi_N= Vχ+Vχp) applied to first and second terminals associated with the non-linear devices to induce the flow of the respective currents.

2. A method as claimed in any preceding claim, in which a gate of a first device of the pair of non-linear devices forms the first terminal; and in which the step of producing the input current comprises the steps of applying the first voltage to the gate of the first device of the pair of non¬ linear devices.

3. A method as claimed in any preceding claim in which a drain of a third device of the non-linear devices forms the second terminal and in which the step of producing the input current comprises the step of applying the " second voltage

to the drain of the third non-linear device of the non-linear devices; the third non-linear device being connected to ground via a parallel combination of the pair of non-linear devices between a terminal.

4. A method as claimed in any preceding claim, further comprising the step of deriving an output current (I_OUT) associated with the input current (IIN).

5. A method of analogue signal processing as claimed in claim 4 in which the step of deriving the output current comprises the step of mirroring, using a fourth non-linear device (M4) of the non-linear devices, the input current.

6. A method of analogue signal processing as claimed in any preceding claim further comprising the step of influencing (excitation or inhibition, ie addition/subtraction) the output current using a bias current (I_BIAS) of a fifth non-linear device (M5).

7. A method of analogue signal processing as claimed in claim 6 in which the fourth (M4) non-linear device is connected in series with the fifth non- linear device between second (VIN) terminal and ground.

8. A method as claimed in either of claims 6 and 7 in which the step of influencing comprises the step of generating the bias current according to a second portion (α(V_OffSetVχ)) of the input current.

9. A method as claimed in claim 8 in which the step of generating the bias current is such that the bias current (IBIA_S) eliminates the second portion

(α(V_0ffsetVχ)).

10. A signal processing circuit comprising at least a pair of non-linear devices operable together to induce a current (II_N) varying linearly with respect to the parameter (VD_S); wherein respective currents of the pair of non-linear devices vary non-linearly with respect to the parameter (VD_S); at least a first device of the pair of non-linear devices comprising a first input terminal for receiving a first input signal.

11. A signal processing circuit as claimed in claim 10 further comprising a third non-linear device (M3) connected in series with a parallel arrangement of the pair of non-linear devices between a second input terminal, for receiving a second input signal, and ground.

12. A signal processing circuit as claimed in claim 11 further comprising means for deriving an output current from the induced current; the output current representing an arithmetic operation associated with the first and second input signals.

13. A signal processing element as claimed in claim 12 in which the means for deriving the output current comprises a further device operable as a current mirror to mirror the induced current in producing the output current.

14. A signal processing element as claimed in any preceding claim further comprising means to influence the output current in response to a third input signal (VBIAS)-

15. A signal processing element as claimed in claim 14 in which the means to influence comprises a further non-linear device operable, in response to the third input signal, to produce a biasing current (I_BIA_S) for influencing the output current (IOUT)-

16. A signal processing element as claimed in either of claims 14 and 15 in which the means to influence is operable to remove an offset current from the output current.

17. A signal processing circuit substantially as described herein with reference to and/or as illustrated in any of the accompanying drawings.

18. A signal processing method substantially as described herein with reference to and/or as illustrated in any of the accompanying drawings.