WO2023170540A1

WO2023170540A1 - Three-dimensional neural network

Info

Publication number: WO2023170540A1
Application number: PCT/IB2023/052065
Authority: WO
Inventors: Simeon Asher BAMFORD; Chiara BARTOLOZZI
Original assignee: Fondazione Istituto Italiano Di Tecnologia
Priority date: 2022-03-07
Filing date: 2023-03-06
Publication date: 2023-09-14
Also published as: IT202200004229A1

Abstract

Three-dimensional neural network (1000, 1000a, 1000b, 1000c) comprising at least one flexible printed circuit flex-PCB (100) made from a sheet (11) of electronically printable two-dimensional flexible material extending along a first (1) and a second (21) dimension, the flexible printed circuit flex-PCB (100) comprising a plurality of neural network elements (10) each comprising a plurality of synapse circuits (14) arranged to receive input signals (8) and to process them for transmission to a neuron circuit (20) of the network element (10), arranged in turn to produce a output signal (26), wherein said synapse circuits (14) are further arranged to connect different neuron circuits (20) of said plurality of neural network elements (10) to one another along said first (1) and second (21) dimension, and to process the output signals (26) of said neuron circuits (20) before sending them as input to other neuron circuits (20) of said neural network elements (10), said flexible printed circuit flex-PCB (100) being rolled or folded in a three- dimensional shape, thus forming a layered structure, wherein individual layers are located side by side along a third dimension (41) perpendicular to said first (1) and second (21) dimension, said output signals (26) of the neuron circuits (20) passing from one layer to another to make connections between said neuron circuits (20) along said third dimension (41), between different layers of the flexible printed circuit flex-PCB (100), so as to form said three-dimensional neural network (1000, 1000a, 1000b, 1000c).

Description

“Three-dimensional neural network”

*******

DESCRIPTION

The present invention relates to a three-dimensional neural network.

The brain, and animal nervous systems more in general, are networks of electrically active cells ("neurons") that pass a stimulatory electrical activity from one to the other through physical connections ("synapses"). Some connections occur between neurons set at a certain distance from one another by means of specialised "processes" or "axons", which are thread-like extensions. However, neurons preferably connect to other nearby or local neurons. It is possible to identify layered structures in the brain in which the neurons are mainly situated along two- dimensional collector surfaces, but the positioning of the neurons and their connectivity are fundamentally three-dimensional.

The volume of the nervous system and especially of the brain is dominated by the volume requirements of axons. Connectivity between neurons is typically directional and of the many-to-many type: communications from one neuron can spread and reach thousands or in some cases up to hundreds of thousands of other neurons and, similarly, a neuron can receive connections from a large number of other neurons.

Since neurons are continually active, processing in the nervous system takes place in parallel, and the speed of cognition and behaviour of animals derives from the ability of nervous systems to carry out this parallel perceptive and cognitive processing.

Artificial neural networks (ANNs) have become an important processing paradigm applied to many problems of the real world. An artificial neural network is a mathematical construct that can be simulated by a serial numerical computation in a central processing unit. However, its computational power derives from the parallelism of the computation and, consequently, dedicated graphic processing units are used to construct and train such networks, wherein specialised computational architectures parallelise the processing to a certain extent.

In the technical field presented above, the use of two-dimensional silicon chips to create integrated circuits that imitate aspects of a nervous system and the computations it performs is known. "Neuron circuits" have been created, each of which achieves a simplification of the processing performed by a biological neuron, and analogue neural networks have also been created with inputs transmitted along rows (in one dimension) that are densely connected to columns of connections in an orthogonal dimension. These "crossbar arrays" are a way to obtain parallel connections.

Analogue multiplication techniques have also been used to achieve synaptic functioning. Inputs produced in the form of voltages are multiplied with predetermined memorised “weights" to create current outputs that are then added together along a wire according to Kirchhoff’s law. Finally, a neuronal circuit applies a nonlinear function to this current value to produce an output (Holler M., Tam S., Castro H., Benson R., “An electrically trainable artificial neural network (ETANN) with 10240 floating gate synapses”, International Joint Conference on Neural Networks 1989 Jun 18 (Vol. 2, pp. 191-196)).

In simpler designs, a chip forms a single row of neurons, so the neural network is physically one-dimensional. However, the network topology can represent a greater dimensional connectivity.

Neurons can also be arranged in a two-dimensional array and in such a case it is possible to create connections between local neurons, thus imitating the mainly local connections that are formed between neurons in the brain, in two of their native three dimensions (Taba B., Boahen K., “Silicon growth cones map silicon retina”, Advances in Neural Information Processing Systems, 2005; 1 8:1329-36).

In another solution, neural circuits produce "spikes” modelled as digital pulses or events (Van Schaik A., “Building blocks for electronic spiking neural networks”, Neural networks, 2001 Jul 9; 14(6-7):617-28); thus, there is an “event-based” communication between the neural circuits and the resulting neural network is called a “spiking neural network' or SNN. The advantages of this approach include a possible lower energy cost for computation in the neural network. In some cases, physical wires provide individual connections between these neural circuits, but more generally the communications are transmitted over a shared routing infrastructure.

In yet another solution, a digitally encoded number is transmitted which represents an "address", whereby the neural circuit that has produced an event can be uniquely identified. Alternatively, these "address events" can carry the address or addresses of the neural circuits to which they must be delivered. This form of multiplexing exploits the difference in speed between the typical communication frequencies that may be reached in silicon chip-based electronics (1 GHz or higher) and the frequencies at which the neurons of a nervous system produce spikes (1 kHZ or lower). As long as the biological processing speeds for individual neurons are complied with, large savings can be obtained in the physical area of silicon chips that must be dedicated to wiring for communication between neural circuits. However, communication in these so-called "neuromorphic" computing architectures remains a limit, contributing decisively to the power budget of neuromorphic computing systems.

One approach that has been developed to overcome these limitations consists in creating "stacked” three-dimensional integrated circuits. Layers of active electronic devices are produced, and they are connected to one another with metal interconnections. These interconnections include arrays of passages through the silicon that permit parallel communications between the layers. Neural networks can thus be created in these stacked three-dimensional structures (Lin P., Li C., Wang Z., Li Y., Jiang H., Song W., Rao M., Zhuo Y., Upadhyay NK, Barnell M., Wu Q., “Three- dimensional memristor circuits as complex neural networks”, Nature Electronics, 2020 Apr; 3(4):225-32; Ham D., Park H., Hwang S., Kim K., “Neuromorphic electronics based on copying and pasting the brain”, Nature Electronics, 2021 Sep; 4(9):635-44).

An advantage of creating physically three-dimensional neural networks is the efficiency of the wiring, as the average distance between the neural circuits is shortened since the number of neural circuits within a given distance increases as n³ in a three-dimensional substrate, compared to the value of n² that is obtained in a two-dimensional substrate. The final compact form that is obtained can also be an advantage for some applications. A possible disadvantage of this solution, however, is the need to dissipate a higher density of heat.

Alongside stacked chip technology, additive manufacturing is a further promising field for the construction of three-dimensional structures that include wiring. It is thus possible to produce three-dimensional structures such as antennas or active devices such as transistors (Kwon J., Takeda Y., Shiwaku R., Tokito S., Cho K., Jung S., “Three-dimensional monolithic integration in flexible printed organic transistors”, Nature communications, 2019 Jan 3; 10(1 ): 1-0.). The ability to form connections between arbitrary places in a neural network is sometimes useful or necessary, but a predominant paradigm in the design of ANNs for practical applications emphasises convolutions, in which the neurons receive inputs from a localised region of space.

Thus, in image processing, where the input space is fundamentally two- dimensional, a neuron located in a first visual processing zone has a "receptive field", which is a contiguous two-dimensional region in the image plane from which the inputs reached the neuron itself.

Active electronic circuits can be integrated on thin flexible substrates (hereinafter flex-PCBs) using printing processes with techniques that include photolithography, inkjet printing, rotogravure printing, etc. (Sheng J., Jeong H.J., Han K.L., Hong T., Park J.S., 2017, “Review of recent advances in flexible oxide semiconductor thin-film transistors”, Journal of Information Display, 18:4, 159-172, DOI: 10.1080/15980316.2017.1385544).

In particular, thin-film transistors (TFTs) can be printed. Comparing cutting- edge technologies, TFTs are considered inferior to MOSFETs produced on silicon based on various factors, including increase in size, reduction in speed and in gain, lower reliability and greater variation between devices. However, production through a printing process allows potential advantages, including ease of access to the technology and design iteration speed.

In the meantime, a series of methods have been developed, also in the field of neural and neuromorphic processing, whereby the computation can be rendered insensitive to variation of the device.

Flex-PCBs have been used to construct analogue neural networks, for example by using two-dimensional crossbar arrays to achieve parallel synaptic connectivity and exploiting the effects of non-volatile memory to program the neural network parameters (Mativenga M., Geng D., Kim B., Jang J., “Fully transparent and rollable electronics”, ACS applied materials & interfaces, 2015 Jan 28; 7(3): 1578-85). Sensor elements such as pressure- or light-sensitive devices can be printed on flex- PCBs (Munzenrieder N., Cantarella G., Vogt C., Petti L., Buthe L., Salvatore G. A., Trdster G. (2015), “Stretchable and conformable oxide thin-film electronics”, Advanced Electronic Materials, 1 (3), 1400038).

Flex-PCBs with printed circuits can also be rolled or otherwise shaped in a third dimension, for example with an accordion-like design. They have been used to construct three-dimensional electronic systems using construction models derived from traditional paper folding arts (Yamaoka J., Dogan M.D., Bulovic K., Saito K., Kawahara Y., Kakehi Y., Mueller S., FoldTronics, “Creating 3D objects with integrated electronics using foldable honeycomb structures”, in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems 2019 May 2 (pp. 1- 14)). However, only a small part of this potentially large design space has been explored to date.

The object of the present invention is thus to propose an innovative three- dimensional neural network that overcomes the problems of the prior art.

This and other objects are achieved with a three-dimensional neural network whose main characteristics are defined in claim 1 .

Particular embodiments form the subject matter of the dependent claims, whose content is to be understood as an integral part of the present description.

Additional features and advantages of the invention will become apparent from the detailed description that follows, provided purely by way of non-limiting example with reference to the appended drawings, in which:

- Figure 1 shows a top view of an element of a three-dimensional neural network;

- Figure 2 shows a top view of a flex-PCB layer comprising a plurality of elements of a neural network;

- Figure 3 shows an enlargement of the second circuit in Figure 1 ;

- Figure 4 shows the circuit of a transmitting element;

- Figure 5 shows the circuit of a receiving element;

- Figure 6 shows a first and a second diffusion circuit;

- Figure 7 shows two flex-PCB layers attached to a cylindrical template;

- Figure 8 shows a three-dimensional view of a neural processor obtained by rotating the template in Figure 7 about a rotation axis thereof;

- Figure 9 shows a vertical cross-section of a transmitting circuit and of a receiving circuit stacked one upon the other in the third dimension;

- Figure 10 is an isometric view of a plurality of diffusion circuits stacked in a third dimension;

- Figure 11 shows a second three-dimensional view of the neural processor;

- Figures 12a-12c show sectional views of the neural processor in Figure 11 ;

- Figure 13 shows the layout of an alternative transmitting circuit; - Figure 14 shows the layout of an alternative receiving circuit;

- Figure 15 shows two-dimensional views of five different ways of composing transmitting and receiving elements to create diffusion circuits;

- Figure 16 shows the diffusion circuits in Figure 15 stacked along the third dimension;

- Figure 17 shows a further rolled flex-PCB layer;

- Figure 18 illustrates an example of a complete neural network;

- Figure 19 is a three-dimensional view of an element of a flexible printed circuit flex-PCB similar to the one in Figure 1 ;

- Figure 20 shows the flex-PCB layer in Figure 2 with the holes highlighted;

- Figure 21 shows a sectional view of the neural processor in Figure 11 ;

- Figure 22 shows an assembled neural processor;

- Figure 23 shows different embodiments of three-dimensional neural networks; and

- Figure 24 shows an alternative version of the neural network element.

In summary, the three-dimensional neural network according to the present invention is based on a flex-PCB sensor, made on a two-dimensional flexible substrate, which comprises two-dimensional matrixes of electronic circuits adapted to perform a neural function, said circuits being printed on one or more portions of the flexible two-dimensional flex-PCB substrate. The term neural function relates to a network of computational elements, each having a specific function and being characterised by preferably one-way connections, but possibly also two-way ones, of the many-to-many type, wherein the computational elements transfer signals having a low information content and wherein the computational power of the overall network derives from the parallelism of the functions.

The electronic circuits achieve, in a non-limited manner, the integration of inputs from a plurality of input neuron circuits, and the transmission of an output signal towards a plurality of second neuron circuits.

The circuits are connected together along a two-dimensional surface of the flex-PCB substrate by means of connection wires or waveguides, so as to create parallel connections, in order that the communications between neuron circuits can take place simultaneously. Furthermore, neighbouring neuron circuits can communicate via a local physical connection, with a consequent reduction in the energy cost of communication. One or more flex-PCB layers are then tightly rolled together, or else physically shaped into a mainly laminar structure, so that the electronic circuits applied on adjacent flex-PCB layers are in close physical vicinity. These neuron circuits thus communicate not only along the two dimensions of the flex-PCB layer on which they are built, but also in a third dimension between adjacent and otherwise near layers of different flex-PCB layers, so that connections can be made between neuron circuits in all three physical dimensions.

The rolled three-dimensional structure is produced in a shaping step following the printing of the flex-PCB substrate, but it is also possible to use already layered two-dimensional flex-PCB layers following a preliminary printing process, and subsequently roll them to create the three-dimensional structure.

Communication between adjacent flex-PCB layers does not necessarily require an ohmic connection between the layers and may also be obtained with capacitive, photonic, quantum mechanical, ionic, chemical or biological means.

Figure 1 shows a top view of an element of a three-dimensional neural network 10 made from a sheet of electronically printable flexible material 11 (flex- PCB substrate), for example PEN (polyethylene naphthalate), preferably with a thickness in the order of 1 pm. Alternatively, the sheet 11 is made of another thin, flexible material on which it is possible to print electronically, including paper.

The two-dimensional shape of the sheet 11 is prevalently rectangular, having a first side 11 a and a second side 11 b.

A first and a second axis 1 and 21 of a Cartesian reference system are included in Figure 1 to highlight that the first side 11a is aligned with the first axis 1 (x axis) whereas the second side 11 b is aligned with the second axis 21 (y axis) perpendicular to the first axis 1 .

In the following description, the term "first dimension" will thus refer to a horizontal dimension, as shown by the dashed-line arrow 1 , whilst "second dimension" will refer to a vertical dimension, as shown by the dashed-line arrow 21. Finally, the term "third dimension" will refer to a third axis 41 (axis z) perpendicular to the first axis 1 and the second axis 21 , oriented in a direction coming out of the plane defined by the first axis 1 and the second axis 21 .

Printed circuits such as active electronic devices like TFTs (thin film transistors) are present on the neural network element 10. The printing process advantageously comprises photolithography, ink jet printing, rotogravure, flexography, or any other similar process. Alternatively, the printed circuits are made separately and subsequently positioned and fixed onto the sheet 11 .

The neural network element 10 comprises at least two conductive layers, which can be joined together or separated by an insulating layer in various points. These layers will subsequently be indicated as metal layers, even though they can be formed from any conductive material, including conductive polymers such as PEDOT:PSS (poly(3,4-ethylenedioxythiophene) polystyrene sulfonate).

Figure 1 thus shows a neural network element 10 intended to form a neuron circuit for transmission (upwards in the second dimension) or reception (downwards in the second dimension).

The neural network element 10 comprises a first circuit 2 (input circuit) adapted to produce output signals towards an output wire 4, in the form of brief pulses of positive voltage. Advantageously, the first circuit 2 is a sensor element that is per se known and adapted to transform environmental inputs such as light, heat, pressure, chemical concentrations, etc., into electric signals. Alternatively, the first circuit 2 is a circuit external to the neural network element 10, controlled by a remote computing device not shown in Figure 1.

The first output wire 4 connects the first circuit 2 to a second circuit 6, in particular a spreader circuit, adapted to transmit and receive signals along the third dimension.

A first bus 8 comprising a plurality of second vertical output wires, preferably five wires, is present at the output of the second circuit 6. One of these second output wires is directly controlled by the first circuit 2, whilst the other four second output wires are controlled by signals received along the third dimension, as described in detail below.

The neural network element 10 further comprises a second bus 12 comprising a plurality of horizontal wires, preferably five wires, which pass horizontally through the sheet 11 of the neural network element 10. The horizontal wires of the second bus 12 cross with the vertical wires of the first bus 8. At every point of crossing between a vertical wire and a horizontal wire a synapse circuit 14 is present.

The synapse circuit 14 comprises a resistive element or a capacitive element or a memristive element, such as a thin layer of P(VDF-TrFE) between the two metal wires (following what is described, for example, in Wang H., Zhao Q., Ni Z., Li Q., Liu H., Yang Y., Wang L., Ran Y., Guo Y., Hu W., Liu Y., “A ferroelectric/electrochemical modulated organic synapse for ultraflexible, artificial visual-perception system”, Advanced Materials. 2018 Nov; 30(46): 1803961 , wherein the ferroelectric memristive properties of PVDF are exploited in a synaptic context).

Alternatively, the synapse circuit 14 is made as a circuit of arbitrary complexity containing active devices, possibly capable of implementing complex synaptic learning rules, such as the ones described, for example in Bamford S.A., Murray A.F., Willshaw D.J., “Spike-timing-dependent plasticity with weight dependence evoked from physical constraints”, IEEE Transactions on Biomedical Circuits and Systems, 2012 Feb 23;6(4):385-98).

Advantageously, the synapse circuit 14 is arranged so as to perform mathematical operations on the signals it receives via the first bus 8 before transmitting them to the neuron circuit 20 of the neural network element 10 or of subsequent elements of neural network 10, as described below in reference to Figure 2. Such mathematical operations include a transduction between a voltage signal and a current signal or vice versa, a scaling reduction or amplification of the signal, or a storage of a reduction/amplification value, and such mathematical operations can be modified by the synapse circuit 14 itself or by a remote circuit.

Going back to Figure 1 , the reference 16 indicates a first fold in each horizontal wire 12 which moves it, in the lower right along the second dimension, towards the level of an underlying horizontal wire 12, whilst the reference 18 indicates a second fold of the last horizontal wire 12 towards a third circuit 20, in particular a neuron circuit.

The neuron circuit 20 can be, for example, a circuit which is per se known, as proposed by Van Schaik A., “Building blocks for electronic spiking neural networks”, Neural networks, 2001 Jul 9;14(6-7):617-28.), or any of the numerous similar circuits that have been proposed in the literature.

The neuron circuit 20 integrates the signals arriving from the second fold 18 and produces output signals over a terminal wire 26, which is advantageously a single wire that carries a positive voltage pulse to represent an “event’ or “spike" originating from the neuron circuit 20, or else it is any type of bus or channel that carries signals of arbitrary complexity. The signal of the terminal wire 26 is intended to travel forward in a single direction or in multiple directions towards additional circuits external to the neural network element 10, not shown in Figure 1. Alternatively, the signal of the terminal wire 26 is transmitted to additional circuits present on a flex-PCB layer 100 that accommodates the neural network element 10, as described below in reference to Figure 2.

The neural network element 10 advantageously also comprises a third bus 22 to deliver a supply voltage for the various active circuits. The third bus 22 also carries other static polarisation voltages or control signals and is connected to a lateral power supply bus that runs laterally from 24a to 24b. The lateral power supply bus carries a power supply, polarisation voltages and optionally also data along the first dimension, for a possible connection to external devices.

Figure 2 shows a top view of a flex-PCB layer 100 arranged to form a neural network according to the present invention. Hereafter in the description the expression “flex-PCB layef’ will be used, but it can be understood alternatively as a “flex-PCB circuit’.

The flex-PCB layer 100 comprises a plurality of neural network elements 10a, 10b, ...10e each accommodating a respective neuron circuit 20a, ... , 20e, located side by side along the first dimension. The references 10' and 10" indicate a first and a last neural network element 10.

If a roller printing process such as rotogravure or flexography is used, the flex- PCB sensor 100 can be very long along the first dimension and contain a large number of flex-PCB circuit elements 10.

A second bus 12', highlighted in bold, travels across the plurality of neural network elements 10a, 10b, ...10e, moving progressively downwards along the second dimension, while it extends towards the right along the first dimension, until connecting to the neuron circuit 20e of the fifth neural network element 10e.

Only synapse circuits 14’ of one of the wires of the second bus 12' are shown in Figure 2, for each neural network element 10a, 10b, ... , 10e, even though synapse circuits 14’ are present at every point of crossing between the second bus 12’ and the respective first buses 8a, 8b, ... , 8e of each neural network element 10a, 10b, ... , 10e. It is possible to note five groups of five synapse circuits 14’ each, connected to one another starting from the first wire of the second bus 12', for a total of twenty-five incoming synapse circuits 14’ on the fifth neuron circuit 20e. The signals coming from the twenty-five synapse circuits 14’ are then processed, as indicated above, before being input to the fifth neuron circuit 20e.

Figure 3 shows an enlargement of the second circuit 6 in Figure 1 . The second circuit 6 comprises a plurality of communication elements 28, preferably eight, overlapping along the second dimension, as detailed below. The communication elements 28 are advantageously transmitting elements or receiving elements, described in detail below.

Figure 4 shows the circuit of a transmitting element 30, which comprises a metal plate 34 and is passed through by the first bus 8. The transmitting circuit 30 is configured to transmit signals coming from the input circuit 2 downwards, along the second dimension, via said first bus 8, and to form a capacitive connection in the third dimension by means of the metal plate 34, placed preferably to the left of the first bus 8 within the transmitting circuit 30.

The capacitive connection is understood as a directional connection along the third dimension, whereby signals are received which enter the plane defined by the first axis 1 and the second axis 21 , signals that move, therefore, in a positive direction along the third axis 41 towards the plane in which the transmitting circuit 30 lies. However, this connection is shielded on the output side with respect to the plane of the transmitting circuit 30 (negative direction along the third axis 41 ) by a lower metal shielding layer 31 , printed below the metal plate 34 and connected to the ground circuit GND. The metal shielding layer 31 is thus kept at the most negative voltage of the device, i.e. a ground voltage GND of 0 V.

The reference 81a indicates a wire of the first bus 8 that enters from above into the transmitting circuit 30, crosses another wire of the first bus 8, a wire 82, without connecting to it, and connects to the metal plate 34 before exiting from the transmitting circuit 30 in the form of an output wire 81 b.

The wire 81a corresponds to the first output wire 4 (in the event that the transmitting circuit 30 is the first communication element 28 situated at the top in Figure 3) and the crossing with the wire 82 creates the possibility of entering a subsequent transmitting circuit 30, as described below with reference to Figure 6.

Figure 5 shows the circuit of a receiving element 32, with some details represented abstractly as circuit symbols. As for the transmitting circuit 30, there are wiring connections on the right and a capacitive element on the left.

In this case, a downward-facing metal plate 36 is present; it is arranged so as to receive signals entering towards the plane defined by the first axis 1 and second axis 21 in a positive direction along the third axis 22, whilst the whole area of the receiving circuit 32 is covered by an upper metal layer 50 (see Figure 9), in order to shield the connection exiting from that plane. The upper metal layer 50 is connected to a predetermined voltage, advantageously a supply voltage VDD, and is thus kept at a more positive voltage than the device.

The metal plate 36 is connected to an amplifier circuit 38, per se known and adapted to amplify the signals (pulses) coming from the plate 36 and transmit them over an output amplifier wire 86 belonging to the first bus 8, said amplifier wire 86 preferably being the first wire of said first bus 8.

During static operation, the amplifier 38 carries the output over the amplifier wire 86 at the ground voltage GND.

In contrast, when, because of a capacitive coupling with the metal plate 34 of a transmitting circuit 30, the metal plate 36 receives an incoming signal and thus increases its voltage output to the amplifier 38, the amplifier 38 will carry the output over the amplifier wire 86 towards the supply voltage VDD until a decrease in the input voltage from the plate 36 is detected or until the input voltage from the plate 36 is initialised by the receiving circuit 32.

The receiving circuit 32 is likewise passed through by the first bus 8 and is arranged to transmit signals downwards along the second dimension via said first bus 8. The signal present on the amplifier wire 86 is also transmitted downwards, becoming one of the wires of the first bus 8 exiting at the bottom.

Figure 6 shows two embodiments wherein transmitting circuits 30 and receiving circuits 32 are combined to create, respectively, a first and a second diffusion circuit 6a, 6b (corresponding to the second circuit 6 in Figure 1 ).

In the first diffusion circuit 6a, the path of the signal coming from the first output wire 4, from its entry into the first diffusion circuit 6a to its exit at the bottom, is highlighted in bold. It may be seen that this signal guides the metal sheet of a first transmitting circuit 30 in order to pass its signal into the third dimension, before moving step-by-step towards the right, and then exit from the diffusion circuit 6a in the position farthest to the right of the first bus 8, i.e. on a second output wire 8_s.

In the second diffusion circuit 6b it may be seen that every receiving circuit 32_2, ... , 32_4 is connected by means of respective second output wire 8_2, ... , 8_s to a respective transmitting circuit 30_i, ... , 30_4 (except for the receiving circuit 32_i situated at the bottom, which is directly connected to the output side by means of the second output wire 8_i), before the second output wire 8_i, ... , 8_s exits towards the bottom in the second dimension. For example, the receiving circuit 32_4 guides the metal plate of the transmitting circuit 30_3 before exiting at the bottom on the second output wire 8_4. The signal coming from the first output wire 4 in the second diffusion circuit 6b, by contrast, connects a transmitting circuit 30_4 before exiting at the bottom over the second output wire 8_s.

Three signals that depart from the receiving circuits 32_4, 32_3, 32_2, are connected to respective transmitting circuits 30_3, 30_2, 30_i , and a last signal coming from the receiving circuit 32_i immediately exits from the diffusion circuit 6b without connecting to a transmitting circuit 30. In this manner, five signals come out from the bottom, one of which comes directly from above (over the second output wire 8_s) and four from the receiving circuits 32 (over the second output wires 8_i , 8_2, 8_3 and 8_4).

Figure 7 shows a first flex-PCB layer 100a (like the one shown in Figure 2) attached at one end to a cylindrical template 40, preferably with adhesive tape 42. A second flex-PCB layer 100b is likewise attached to the template 40. The first flex- PCB layer 100a and second flex-PCB layer 100b carry, respectively, the first and second diffusion circuits 6a, 6b in Figure 6. The reference 24 indicates a power supply bus, bias and global signal adapted to carry a power supply, polarisation voltages and optionally also data between neuron circuits 20 of different neural network elements 10 or for a connection to remote external devices.

Figure 8 shows a three-dimensional view of a neural processor 1000 (neural network) obtained by rotating the template 40 in Figure 7 around a rotation axis 44 thereof.

The neural processor 1000 comprises the two flex-PCB layers 100a and 100b. In reference to this structure, the third dimension is understood as the outward radial direction, represented by the dashed-line arrows 41 . The reference 42 indicates the end of the power supply bus, bias and global signal 24 of the second flex-PCB layer 100b, arranged to enable a connection to external circuits for the power supply and communication.

The neural processor 1000 thus comprises the first diffusion circuit 6a made on the first flex-PCB layer 100a and the second diffusion circuit 6b made on the second flex-PCB layer 100b. It may be noted that these circuits are parallel to one another, positioned closely together and overlapping in the third dimension. In general, these circuits can be more or less aligned in the second dimension, because of the fact that the circumference of the winding varies by a few multiples of the width of the neural network element 10 as winding proceeds and the winding radius increases.

Figure 9 shows a vertical cross section of a transmitting circuit 30 and a receiving circuit 32 stacked one on top of the other in the third dimension. The dashed-line arrow 41 , representative of the third dimension, is shown in Figure 9 from left to right, whilst the second dimension is represented by the vertical arrow 21.

The references 45 and 48 indicate a respective first and second substrate layer, on which other layers are printed along the third dimension.

Figure 9 shows the first lower metal layer 31 of the transmitting circuit 30, connected to the ground voltage GND, and the metal plate 34 of the transmitting circuit 30, surrounded by a first insulating layer 47.

Figure 9 further shows the metal plate 36 of the receiving circuit 32, surrounded by a second insulating layer 49, and the upper metal layer 50, connected to the power supply voltage VDD.

The first insulating layer 47 is shown adjacent to the second substrate layer 48. These two layers are placed in contact because of the lamination process. The metal plate 34 of the transmitting circuit 30 is coupled with the metal plate 36 of the receiving circuit 32. These two metal plates 34, 36 together form a capacitor 51 .

When the input circuit 2 (not shown in Figure 9) produces a signal which, via the first output wire 4, not shown in Figure 9, brings the metal plate 34 of the transmitting circuit 30 to a voltage as high as the power supply voltage VDD, an increase in voltage is induced on the metal plate 36 of the receiving circuit 32. The voltage pulse coming out of the metal plate 36 represents the input of the amplifier circuit 38 of the receiving circuit 32 (see Figure 5). The voltage pulse from the transmitting circuit 30 is thus transmitted, along the third dimension, to the receiving circuit 32. The amplifier circuit 38 can introduce a delay whose length will vary depending on the overall design of the device.

The capacitor 51 present between the transmitting circuit 30 and the receiving circuit 32 is shielded, in relation to signals coming from other layers of the neural processor 1000, respectively by the upper metal layer 50 of the receiving circuit 32 and the lower metal layer 31 of the transmitting circuit 30. These layers themselves form a capacitor configured to store energy and stabilise the power signal. Figure 10 shows five diffusion circuits 6a_1 , 6b_2, 6a_3, 6b_4 and 6a_5 in isometric projection, thus stacked one on top of the other along the third dimension 41.

The first dimension 1 and the second dimension 21 are likewise represented with dashed-line arrows. The distance between layers along the third dimension has been increased for clarity.

First output wires 4_i, 4_2, ... , 4_s lead into the respective diffusion circuits 6a_1 , 6b_2, 6a_3, 6b_4, 6a_5, providing input pulses (coming from respective first circuits 2 not shown in Figure 10).

Second output wires 8_i , 8_2, ... , 8_s lead out of the respective diffusion circuits 6a_1 , 6b_2, 6a_3, 6b_4, 6a_5 and represent the individual wires of the first bus 8 in Figure 1 .

The diffusion circuits 6a_1 , 6b_2, 6a_3, 6b_4, 6a_5 are of two different types as described with reference to Figure 6, and are alternating.

Capacitors C1a,...,n, C2_a,...,_n, C3_a,...,_n and C4_a,...,_n present between the layers are represented with dashed lines.

A voltage pulse which, through a first of the first output wires 4_i , arrives at the first diffusion circuit 6a_1 , is transmitted through the first capacitor C1 _a to the second diffusion circuit 6b_2. The pulse is amplified and activates the metal plate of a further transmitter circuit 30_ib, which is coupled to the receiving circuit 32_ of the third diffusion circuit 6a_3 via the capacitor C . In this manner, the voltage pulse, through a series of four passages based on the capacitive transmission from one layer to the other, arrives at the last diffusion circuit 6a_5, finally exiting through a first of the second output wires 8_i.

A voltage pulse coming from a second of the first output wires 4_2 travels along a similar path, completing only three passages to arrive at a second of the second output wires 8_2.

A voltage pulse coming from a fifth of the first output wires 4_s travels instead directly towards a fifth of the second output wires 8_s.

The general connectivity scheme enables every voltage pulse applied on the first output wires 4_i, 4_2, ... , 4_s to be transmitted over the five second output wires 8_i , 8_2, ... , 8_5 of different layers.

Figure 11 shows a second three-dimensional view of the neural processor 1000, in which only some of its circuits are shown. For the sake of simplicity, the output neuron circuit 20 receives input from only three inputs in the first dimension and only three inputs in the third dimension, unlike in the case of the five parameters used in the description above.

Furthermore, for the sake of simplicity, in Figure 11 the output neuron circuit 20 receives signals coming from circuits situated on its right, unlike what is shown in Figure 2, in which the fifth neuron circuit 20e instead receives signals coming from its left.

The neural processor 1000 comprises an upper surface region 52 below which there is a plurality of input circuits 2_1a, 2_1 b, 2_2a, 2_2b, 2_3a, 2_3b whose inputs are transmitted to the neuron circuit 20. In particular, said input circuits 2_1a, 2_1 b, 2_2a, 2_2b, 2_3a, 2_3b are on three different layers of the neural processor 1000, and the surface region 52 is thus a two-dimensional surface formed by the edges of said adjacent layers. The surface region 52 represents the receptive field of the neuron circuit 20.

The input circuits 2_1a, 2_1 b, 2_2a, 2_2b, 2_3a, 2_3b are located in all of the first, second and third dimensions of the neural processor 1000 with respect to the neuron circuit 20; the connectivity model thus exploits all three dimensions of the device and represents a key element of the present invention.

An input circuit 2_3a is on the same layer as the neuron circuit 20. A first output wire 4 leads out of the input circuit 2_3a and leads into a diffusion circuit 6_a, shown symbolically. A second output wire 8_1a leads out of the diffusion circuit 6_a and reaches a synapse circuit 14_3a. An input circuit 2_2a also transmits pulses downwards over a first output wire (not shown) and is transmitted to the diffusion circuit 6_a. The transmitted pulses then exit over a respective second output wire 8_2a and reach a synapse circuit 14_2a. Similarly, an input circuit 2_1a also transmits pulses to the neuron circuit 20.

It is interesting to note that the input circuit 2_1a is on the same printed substrate as the neuron circuit 20, but the transmission does not take place along the surface of said rolled layer, but rather along a shorter physical path in the third dimension.

The input circuit 2_3b is on the same substrate as the neuron circuit 20 and is placed in an adjacent cell.

The input circuit 2_3b transmits downwards towards the synapse circuit 14_3b and from there reaches the neuron circuit 20. The synapse circuits 14_2a, 14_3a, 14_3b provide an analogue electrical quantity, as a current or charge, which can be added as input to the neuron circuit 20. More in general, the encoding of these inputs can take on any form per se known, and the sum can be obtained with any known technique, including discrete computation or numerical simulation.

The effect of the misalignment along the first dimension due to the variation in circumference is that some transmitting-receiving capacitors will be misaligned and thus less effective. It can also occur, depending on the design details, that one-to-two or two-to-one couplings are formed between the transmitting and receiving circuits.

Figure 12a shows a sectional view of the neural processor in Figure 11 .

In particular, it shows an edge 1001 of a matrix of neural network elements wound into a spiral if one is looking down towards the neural processor 1000 in Figure 11 , i.e. along the second dimension. The central area of the winding of the neural processor 1000, where the radii of curvature are smaller, is occupied by the cylindrical template 40. In Figure 12 the boundaries of the flex-PCB layers 100a and 100b are also shown. Figure 12 also shows, as thicker portions along the lines of the spiral, the neural network elements 10, which have all been printed with an equal length and spacing along the first dimension. Due to the large radius of the neural processor 1000 relative to the width of the neural network elements 10, the flex-PCB layers 100a, 100b appear almost parallel to one another.

Figure 12b shows an enlargement of a first portion 94 in Figure 12a, in which a receptive field 52f of a neural network element 10f is illustrated. Figure 12b is based on receptive fields 52f of five times five inputs along the first and third dimensions. It should be noted that the inputs along the first dimension are carried to the neuron circuit 20 by means of the synapse circuits 14’ (see Figure 2) and then along the second bus 12’ (see Figure 2), whereas the inputs along the third dimension arrive through the second circuits 6 and then through the synapse circuits 14’ and the second bus 12’.

The reference 55 indicates the capacitive couplings, shown as dashed lines that unite the flex-PCB layers, and which can be of a one-to-one type or also of a two-to-one type, as shown by the reference 54, so that the neuron circuit 20 of the neural network element 10f receives input from a receptive field of twenty-six input circuits, and not only from the twenty-five provided for. Figure 12c shows an enlargement of a second portion 96 in Figure 12a, wherein two neural network elements (two cells) 10g, 10h, and the respective receptive fields 52g, 52h are represented. It may be noted that a portion of said receptive fields 52g, 52h is shared, so that the diffuser circuits 6 of the neural network elements 10g, 10h of that shared portion will serve to carry signals coming from input neurons both to the neural network element 10g and to the neural network element 10h.

The receptive fields 52f, 52g, 52h all have different shapes, because of the lateral shifting of the neural network elements 10f , 10g, 10h, which change with increases in the radius of the neural processor 1000. However, all the receptive fields 52f, 52g and 52h are contiguous areas.

These misalignment effects can be compensated for with a winding strategy that lengthens the substrate as winding proceeds; however, this is not necessary in order for the design to produce functional neural networks, since training compensates for the structural inhomogeneities of the neural network (Bamford S.A., Murray A.F., Willshaw D.J., “Silicon synapses self-correct for both mismatch and design inhomogeneities”, Electronics letters. 2012; 48(7):360-1 .).

As an alternative to coupling between layers using capacitive pulses, light pulses are transmitted and received.

An alternative design that uses printed light-emitting diodes and photodiodes will now be illustrated. In an alternative embodiment, the simultaneous transmission from one layer towards many layers is also demonstrated: since thin photodiodes can absorb only a part of the light that passes through them, it is possible for a stack of photodiodes on separate layers to each receive and amplify a pulse from a single pulse transmitter.

Figure 13 shows the layout of an alternative transmitting circuit, in particular a light transmitting circuit 30’ intended to be used in the place of the transmitting circuit 30 of a capacitive type in Figure 4.

The light transmitting circuit 30’ comprises a light-emitting diode 56, shown in a standard circuit configuration, configured to pass pulses through a second output wire 8a_i and convert them into light pulses. Even though Figure 13 presents the use of certain components in a certain configuration, a person skilled in the art will understand that in general any component or circuit that emits light can be used. The light transmitting circuit 30’ sends light only upwards in the third dimension (positive direction along the third axis 22) and prevents the passage of light downwards thanks to the presence of an opaque lower layer 31’. Other second output wires 8_a_2, 8_a_3, 8_a_4, 8_a_5, pass through the cell of the light transmitting circuit 30’ without interacting with other circuits.

Figure 14 shows the layout of an alternative receiving circuit, in particular a light receiving circuit 32’, intended to be used in the place of the receiving circuit 32 in Figure 5, which is of a capacitive type.

The light receiving circuit 32’ comprises a photodiode 58 configured to convert incoming light pulses into current pulses. The light receiving circuit 32’ further comprises an amplification circuit 60 adapted to transform the current pulses into voltage pulses over a second output wire 8b_i. Even though Figure 14 presents the use of certain components in a certain configuration, a person skilled in the art will understand that in general any light-detecting component or circuit can be used. This cell is designed to have the greatest translucency possible, so that light can pass through it freely. Other second output wires 8_a_i, 8_a_2, 8_a_3, 8_a_4, 8_a_s, pass through the cell of the light receiving circuit 32’ in the second dimension.

Figure 15 shows two-dimensional views of five different ways of composing the transmitting and receiving elements in Figures 13 and 14 to create respective diffusion circuits 6’a, 6’b, 6’c, 6’d, 6’e.

A light transmitting circuit 30’ and four light receiving circuits 32’ are present for every row. Five flex-PCB sensor strips like the one in Figure 2 are thus printed, each of which comprises light transmitting circuits 30’ and light receiving circuits 32’ organised according to what is shown in the columns of said diffusion circuits 6’a, 6’b, 6’c, 6’d, 6’e. These five strips of the flex-PCB layer are then wound together into a coil, in a manner similar to that illustrated with reference to Figure 7.

Figure 16 shows the diffusion circuits 6’a, 6’b, 6’c, 6’d, 6’e in Figure 15 stacked along the third dimension (axis 41 ) after winding.

Like Figure 10, the drawing is isometric, and the first, second and third dimensions are represented by the respective dashed-line arrows 1 , 21 , and 41. The layers accommodating the diffusion circuits 6’a, 6’b, 6’c, 6’d, 6’e are represented, in the third dimension, spaced apart by a great deal more than they are in a real spiral structure. For example, if the light transmitting circuits 30’ and light receiving circuits 32’ are of the order of 100um, in the first and second dimensions, the distance between layers along third dimension is for example 10 pm or less, thus creating compact three-dimensional zones in which light is transmitted and received.

First output wires 4_i, 4_2, 4_3, 4_4, 4_s carry respective signals coming from input circuits 2 not shown in Figure 16, which are transmitted by the diffusion circuits 6’a, 6’b, 6’c, 6’d, 6’e in order to arrive at respective second output wires 8_i , 8_2, ... , 8_5 and, finally, be output from the fifth diffusion circuit 6’e. A further copy of the first diffusion circuit 6’a is shown on the right in Figure 16.

When a pulse arrives over a first of the first output wires 4_i of the first diffusion circuit 6’a, it arrives at a respective light transmitter 3O’a1 and produces a short light pulse. This light cannot travel towards the left along the third dimension, because the light transmitting circuit 3O’a1 comprises the opaque lower layer 3T. The light thus travels upwards along the third dimension, i.e. towards the right in the drawing, as indicated by the dashed-line arrow 62, which represents a ray of light. The ray of light 62 passes through the transparent (or translucent) substrates of the second diffusion circuit 6’b to stimulate a first receiver element, and a substantial part of the light continues to travel through the subsequent diffusion circuits 6’c, 6’d and 6’e to stimulate the light receiving circuit 32’e1 , which amplifies the pulse received and sends it to a first of the second output wires 8_i.

A pulse that arrives instead over a second of the first output wires 4_2 stimulates the second light transmitting circuit 30’b and produces a respective ray of light 66. The ray of light 66 is propagated along the third dimension (towards the right in the figure), passing through various layers to stimulate different light receiving circuits, including the light receiving circuit 32’e2, which amplifies the pulse received and transmits it over a second of the second output wires 8_2. The ray of light continues to be propagated through the subsequent diffusion circuit 6’a.

In the meantime, any light that should arrive below the light transmitting circuit 30’b, such as a ray of light 64, will be blocked by the opaque lower layer 3T of the light transmitting circuit 30’b, and thus does not reach further light receiving circuits.

A pulse on a fifth of the first output wires 4_s, by contrast, travels directly downwards through the respective diffusion circuit 6’e, in order to be output over a fifth of the second output wires 8_s.

Transmission from one layer to others has been achieved using two or more flex-PCB layers printed with different patterns, in order to align the receiving circuits with the transmitting circuits. It is also possible to use one or more flex-PCB layers printed solely with a repeated pattern.

Figure 17 shows a further flex-PCB layer 100’e rolled into a spiral.

The flex-PCB layer 100’e comprises the diffusion circuits 6’e in Figure 15. At every complete rotation of the spiral, a downward offset 67 has been introduced. The offset 67 is equal to the pitch between the light transmitting circuits 30’ and the light receiving circuits 32’, in the second dimension, and is also the height, again in the second dimension, of such light transmitting circuits 30’ and light receiving circuits 32’, such as for example the circuits 3O’_1 and 32’_2.

An input circuit 2_2 is connected to a diffusion circuit 6’e_2, shown on the outermost coil in the third dimension. An input circuit 2_1 is exposed in the upper part of the coil but is connected to a diffusion circuit 6’e_1 , which, being lower in the second dimension, is covered by the layer on which the diffusion circuit 6’e_2 is printed. It can be seen that the light transmitting circuit 3O’_1 is aligned, in the second dimension, with the light receiving circuit 32’_2. In this manner, it is possible to form stacks of diffusion circuits 6’a, 6’b, 6’c, 6’d, 6’e in the third dimension which reach a connectivity topologically similar to the one shown in Figure 16.

Figure 18 illustrates an example of a complete neural network.

The spiral neural processor 1000 comprises a single flex-PCB layer 100, even though, alternatively, several flex-PCB layers 100 can be used. The end of the flex- PCB layer 100 in the first dimension is illustrated unrolled for the sake of clarity. The reference 72 indicates an optional connector for an external processing system, which is adapted to carry a communication bus. This bus is distributed by various multiplexer/demultiplexer blocks over an area printed to measure 78, separating the bus into various communication sub-buses 68, which then extend along the first dimension. In a preferred embodiment of the present invention, the communication sub-bus 68 is used to implement a guaranteed addressed delivery (including arbitration and queuing where necessary) of asynchronous digital events (Manohar R., “Reconfigurable asynchronous logic”, IEEE Custom Integrated Circuits Conference 2006, 2006 Sep 10 (pp. 13-20), IEEE.). An event can thus be sent via the connector 72, arrive at a router block 70 and trigger an input circuit 2 to produce a short output voltage pulse. Alternatively, the input circuit 2 is a sensor element, activated to produce pulses from an environmental stimulus. When the input circuit 2 produces a pulse, the latter is detected by an intermediate router 74a and is delivered, outside the chip, as an address event or by means of some other communication protocol. Alternatively, the pulse is routed inside the chip according to a predetermined algorithm into one or more different positions in the neural processor 1000. In this manner, arbitrarily complex network architectures can be produced. The pulse from the input circuit 2 can also be delivered directly to the diffusion circuit 6, where it is delivered to the other layers in the third dimension.

Circuits present on other layers in the third dimension deliver pulses, through respective diffusion circuits, to the layer in question, so that there is a bus of wires leading out from each diffusion circuit of each layer, each of which carries an output signal. These pulses stimulate the synapse circuit 14a and thus provide the input to a neuron circuit 20a.

The layers of an artificial neural network are made up of groups of neuron circuits, and in a typical neural network there are at least two, sometimes many, layers. A so-called “deep" neural network is a network that has many layers, and in a typical so-called “feedforward” neural network, the neuron circuits in one layer are connected to many neuron circuits in a subsequent layer, in progression, and so on. The input circuits are thus on the first layer, and the neuron circuits 20a, 20b etc. on the second layer.

The output of the bus from the input circuit 2 and the diffusion circuit 6 continues downward, in the second dimension, and provides pulses to the synapse circuit 14b, thus providing the input to a neuron circuit 80 of the third layer. There is thus a connection from the first layer towards the second layer and the third layer. This can be considered a “skip" connection. Alternatively, the second and third layers can be considered two parallel “kernels" of a single layer. The output of the neuron circuit 20a passes through a router 74b and a spreader circuit precisely like the output of the input circuit 2, and it likewise stimulates the neuron circuit 80 of the third layer.

The output of the neuron circuit 80 is transferred upwards, in the second dimension, reaching a synapse circuit 14c, which then provides the input to the neuron circuit 80. This is an example of a “recurrent’ connection. The output of the neuron circuit 80 also reaches a synapse circuit 14d, which provides an input to the neuron 20a located in the previous layer. This is an example of a so-called “feedback’ connection.

In Figure 18 the reference 76 indicates a grouping of two adjacent neural network elements 10. The structure of these neural network elements 10 differs slightly, so that one contains the neuron circuit 80 in the third layer whilst the other (the one with the reference 10) does not contain it.

This example illustrates the possibility for different layers to have different numbers, pitches or types of neuron circuits and different types of connectivity.

An additional router 74c can provide pulses to the neuron circuit 80 through the synapse circuit 14e. Additional inputs like these can advantageously be used to carry out “semi-supervised” training.

Although analogue neuromorphic computing, in particular of the spike-based type, is generally considered advantageous because of lower energy consumption compared to the arithmetical simulation of neural networks, in densely packed volumes of active electronic devices, some of which continuously functioning, the dissipation of heat can become a problem.

Figure 19 shows a three-dimensional view of an element of a flexible printed circuit flex-PCB 10 similar to the one in Figure 1 , which has been modified to include a cut 82 (hole), which can be made during the printing process and before the three- dimensional conformation.

Figure 20 shows the flex-PCB layer 100 in Figure 2; the holes 82 are indicated and the other details in Figure 2 are not illustrated for reasons of clarity. Between every pair of holes 82 in every row there is a portion 88 of the flex-PCB layer 100 which has not been removed. The width of the portions 88 along the second dimension is smaller than the width of the holes 82.

Figure 21 shows a sectional view of the neural processor 1000 in Figure 11 , in six flex-PCB layers 100 are present, namely layers 100_1 , 100_2 ... 100_6, shown laterally in Figure 21 , which represents the first dimension. The section is shown at the level of the holes 82 in the second dimension so that the portions 88 are present. For example, in a first flex-PCB layer 100_1 one sees the portions 88_1a, 88_1b, 88_1c, 88_1d, etc. Any curvature that may result from the rolling of the neural processor 1000 is not shown for the sake of simplicity. Even if the flex-PCB layers 100_1 , 100_2, ... , 100_6 are not aligned along the first dimension, empty passages are in any case formed along the third dimension (upwards or downwards in Figure 21 ) as shown by the arrow 90.

Figure 22 shows the assembled neural processor 1000.

The cut 82 represents a hole cut into the surface of the flex-PCB circuit element 10, whilst the reference 84 indicates the empty space that forms after the three-dimensional stacking of several cuts 82, i.e. the space corresponding to the empty passages that form where indicated by the arrow 90 in Figure 21. This empty space permits a fluid to pass through the three-dimensional volume, be it liquid or gas, to help the dissipation of heat. A further possibility that this design allows is that chemical or thermal sensors or sensors of another type can be included in the printed design to act as an input for the neural network and detect some physical properties of the fluid.

The power supply can be provided to the circuits by means of the neural network element 10 along the first dimension, but it can also be enhanced by the formation of ohmic connections along the third dimension. In this case, the cut 82 in Figure 19 represents an area of the flex-PCB substrate that has been designed to facilitate the formation of ohmic connectivity in the third dimension, and the shaded area 84 in Figure 22 represents a final conductive node through which power can be supplied throughout the third dimension.

Even though the present description has focused on spiking or pulsed neural networks, any form of neural computation or other parallel computation can be performed by a structure such as the one described in the present invention, wherein the key innovation is the use of electronic or photonic connectivity between printed layers in order to achieve a third dimension of connectivity between mainly local computing elements, which act in parallel. In particular, neural networks with a continuous value can be produced, whether they operate continuously or are governed by a clock, or the values are represented by analogue electronic quantities or are represented digitally. Though the design of the connections between layers has shown communication by pulses (or “events", or “peaks"), the person skilled in the art will understand that there are many methods whereby continuous values can also be transmitted. Even though the transmission of data packets, such as the use of an address event representation, has been described along the first dimension, such communication protocols can also be implemented to transmit data among different layers of the three-dimensional device. Although in the present description only coils have been illustrated as three- dimensional conformation steps, any three-dimensional configuration conformation that results in a number of layers of a printed circuit extending in a prevalently laminar structure is included within the scope of the present invention.

Figure 23 shows different embodiments of three-dimensional neural networks.

In particular, a first neural network 1000a, made by folding a single flex-PCB sheet of the previously described type in an accordion-like fashion, or a second neural network 1000b, made by folding two flex-PCB sheets in an accordion-like fashion, or a third neural network 1000c, made by simply stacking separate flex-PCB sheets.

Figure 24 shows an alternative version of the neural network element 10, incorporating an alternative optical diffusion circuit 6”a, which differs from the previously described diffusion circuit 6’a in that the amplification circuits 60 of each of the receiving circuits have been moved down into an area 92. The area 92 also contains the neuron circuit 20. The second fold 18 and the directionality of the second bus 12 have been adapted to this modification, with the signals output from the synapse circuits 14 propagating towards the left, towards their neuron circuits 20.

The grouping of active circuits which provides for transistors in the area 92 leaves open the possibility that such circuits are not created by direct printing onto the surface of the neural network element 10, but rather through a separate manufacturing process, which could include the production of molten silicon CMOS and thinning of the matrix, and subsequent positioning and gluing. The connection to the communication bus 24, not shown in Figure 24, and other functions can also be arranged in other areas such as the area 92, allowing most of the area of the flex- PCB layer to be used for printing communication elements, memristive synaptic elements and other possible elements, such as detection and implementation elements.

Naturally, without prejudice to the principle of the invention, the embodiments and details of execution can be broadly varied with respect to what has been described and illustrated purely by way of non-limiting example, without for this reason falling outside the scope of protection of the present invention defined by the appended claims.

Claims

1. Three-dimensional neural network (1000, 1000a, 1000b) comprising at least one flexible printed circuit flex-PCB (100) made from a sheet (11 ) of electronically printable two-dimensional flexible material extending along a first (1 ) and a second (21 ) dimension, the flexible printed circuit flex-PCB (100) comprising a plurality of neural network elements (10) each comprising a plurality of synapse circuits (14) arranged to receive input signals (8) and to process them for transmission to a neuron circuit (20) of the neural network element (10) arranged in turn to produce an output signal (26), wherein said synapse circuits (14) are further arranged to connect different neuron circuits (20) of said plurality of neural network elements (10) to each other, along said first (1 ) and second (21 ) dimension, and to process the output signals (26) of said neuron circuits (20) before sending them as input to other neuron circuits (20) of said neural network elements (10), said flexible printed circuit flex-PCB (100) being rolled or folded in a three- dimensional shape thus forming a layered structure, wherein individual layers of said layered structure are located side by side along a third dimension (41 ) locally perpendicular to said first (1 ) and second (21 ) dimension and wherein circuits placed on adjacent layers of said layered structure are in close proximity along said third dimension (41 ), said output signals (26) of the neuron circuits (20) passing from one layer to another to make connections between said neuron circuits (20) along said third dimension (41 ), between different layers of the flexible printed circuit flex-PCB (100), so as to form said three-dimensional neural network (1000, 1000a, 1000b, 1000c).

2. Three-dimensional neural network (1000, 1000a, 1000b) according to claim 1 , wherein said input signals (8) come from a sensory element (2) belonging to the neural network element (10) and capable of transforming environmental inputs into electrical signals, or they come from a remote circuit.

3. Three-dimensional neural network (1000, 1000a, 1000b) according to claim 1 or 2, wherein each output signal (26) is arranged to be transmitted to a circuit external to the neural network (1000, 1000a, 1000b, 1000c) or to a further circuit present on said flexible printed circuit flex-PCB (100) that accommodates said plurality of neural network elements (10).

4. Three-dimensional neural network (1000, 1000a, 1000b) according to any of the preceding claims, wherein said plurality of neural network elements (10) comprises circuits printed by a printing process comprising photolithography, inkjet printing, gravure printing, flexography or the printed circuits are made separately and subsequently positioned and fixed on the sheet (11 ) of flexible material.

5. Three-dimensional neural network (1000, 1000a, 1000b) according to any of the preceding claims, wherein, when the flexible printed circuit flex-PCB (100) is conformed into a three-dimensional shape, said plurality of neural network elements (10) is arranged to form multiple connections between neuron circuits (20) placed in physical proximity in a three-dimensional space.

6. Three-dimensional neural network (1000, 1000a, 1000b) according to any of the preceding claims, wherein each synapse circuit (14) is arranged to perform mathematical operations on the input signals (8) before transmitting them to the neuron circuit (20) of the respective neural network element (10) or other neural network elements (10) of said plurality of neural network elements (10), said mathematical operations including a transduction between a voltage input signal (8) to a current signal or vice versa, a scaling reduction or amplification of the input signal (8) or a memorisation of a reduction/amplification value, said mathematical operations being modifiable by the synapse circuit (14) itself or by a remote circuit.

7. Three-dimensional neural network (1000, 1000a, 1000b) according to any of the preceding claims, wherein said plurality of neural network elements (10) comprises a connection bus (24) for a data connection between different neuron circuits (20) or for a connection to remote devices.