Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/08—Learning methods
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/04—Architecture, e.g. interconnection topology
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/04—Architecture, e.g. interconnection topology
  • G06N3/0464—Convolutional networks [CNN, ConvNet]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/04—Architecture, e.g. interconnection topology
  • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/04—Architecture, e.g. interconnection topology
  • G06N3/0495—Quantised networks; Sparse networks; Compressed networks
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
  • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/08—Learning methods
  • G06N3/084—Backpropagation, e.g. using gradient descent
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/08—Learning methods
  • G06N3/09—Supervised learning
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks

Definitions

• a method of operating a neural network includes determining a complexity of separable filters
• a computer program product for operating a neural network.
• the computer program product includes a non- transitory computer readable medium having encoded thereon program code.
• the program code includes program code to determine a complexity of separable filters approximating a filter in the neural network.
• the program code further includes program code to selectively apply a decomposed convolution to the filter based on the determined number of separable filters.
• computational network neural system or neural network
• FIGURE 4 illustrates an example of a positive regime and a negative regime for defining behavior of a neuron model in accordance with certain aspects of the present disclosure.
• FIGURE 5 illustrates an example implementation of designing a neural network using a general-purpose processor in accordance with certain aspects of the present disclosure.
• FIGURE 7 illustrates an example implementation of designing a neural network based on distributed memories and distributed processing units in accordance with certain aspects of the present disclosure.
• FIGURE 8 illustrates an example implementation of a neural network in accordance with certain aspects of the present disclosure.
• an action potential In biological neurons, the output spike generated when a neuron fires is referred to as an action potential.
• This electrical signal is a relatively rapid, transient, nerve impulse, having an amplitude of roughly 100 mV and a duration of about 1 ms.
• every action potential has basically the same amplitude and duration, and thus, the information in the signal may be represented only by the frequency and number of spikes, or the time of spikes, rather than by the amplitude.
• the information carried by an action potential may be determined by the spike, the neuron that spiked, and the time of the spike relative to other spike or spikes. The importance of the spike may be determined by a weight applied to a connection between neurons, as explained below.
• the transfer of spikes from one level of neurons to another may be achieved through the network of synaptic connections (or simply "synapses") 104, as illustrated in FIGURE 1.
• neurons of level 102 may be considered presynaptic neurons and neurons of level 106 may be considered postsynaptic neurons.
• the synapses 104 may receive output signals (i.e., spikes) from the level 102 neurons and scale those signals according to adjustable synaptic weights where P is a total number of synaptic connections between the neurons of levels 102 and 106 and i is an indicator of the neuron level.
• i represents neuron level 102 and i+1 represents neuron level 106.
• Biological synapses can mediate either excitatory or inhibitory (hyperpolarizing) actions in postsynaptic neurons and can also serve to amplify neuronal signals.
• Excitatory signals depolarize the membrane potential (i.e., increase the membrane potential with respect to the resting potential). If enough excitatory signals are received within a certain time period to depolarize the membrane potential above a threshold, an action potential occurs in the postsynaptic neuron. In contrast, inhibitory signals generally hyperpolarize (i.e., lower) the membrane potential. Inhibitory signals, if strong enough, can counteract the sum of excitatory signals and prevent the membrane potential from reaching a threshold.
• the neural system 100 may be emulated by a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components, a software module executed by a processor, or any combination thereof.
• the neural system 100 may be utilized in a large range of applications, such as image and pattern recognition, machine learning, motor control, and alike.
• Each neuron in the neural system 100 may be implemented as a neuron circuit.
• the neuron membrane charged to the threshold value initiating the output spike may be implemented, for example, as a capacitor that integrates an electrical current flowing through it.
• the neuron 202 may combine the scaled input signals and use the combined scaled inputs to generate an output signal 208 (i.e., a signal Y).
• the output signal 208 may be a current, a conductance, a voltage, a real-valued and/or a complex-valued.
• the output signal may be a numerical value with a fixed-point or a floating-point representation.
• the output signal 208 may be then transferred as an input signal to other neurons of the same neural system, or as an input signal to the same neuron 202, or as an output of the neural system.
• the weights may settle or converge to one of two values (i.e., a bimodal distribution of weights). This effect can be utilized to reduce the number of bits for each synaptic weight, increase the speed of reading and writing from to a memory storing the synaptic weights, and to reduce power and/or processor consumption of the synaptic memory.
• STDP spike -timing-dependent plasticity
• BCM Bienenstock-Copper-Munro
• the weights may settle or converge to one of two values (i.e., a bimodal distribution of weights). This effect can be utilized to reduce the number of bits for each synaptic weight, increase the speed of reading and writing from to a memory storing the synaptic weights, and to reduce power and/or processor consumption of the synaptic memory.
• STDP is a learning process that adjusts the strength of synaptic connections between neurons. The connection strengths are adjusted based on the relative timing of a particular neuron's output and received input spikes (i.e., action potentials).
• LTP long-term potentiation
• LTD long-term depression
• FIGURE 3 illustrates an exemplary diagram 300 of a synaptic weight change as a function of relative timing of presynaptic and postsynaptic spikes in accordance with the STDP.
• a negative offset ⁇ may be applied to the LTP (causal) portion 302 of the STDP graph.
• the offset value ⁇ can be computed to reflect the frame boundary.
• a first input spike (pulse) in the frame may be considered to decay over time either as modeled by a postsynaptic potential directly or in terms of the effect on neural state.
• the computational complexity may be on the order of
• FIGURE 5 illustrates an example implementation 500 of the aforementioned decomposition using a general-purpose processor 502 in accordance with certain aspects of the present disclosure.
• Variables neural signals
• synaptic weights may be stored in a memory block 504
• instructions executed at the general-purpose processor 502 may be loaded from a program memory 506.
• the instructions loaded into the general-purpose processor 502 may comprise code for determining a number of separable filters to express a filter in the neural network and/or selectively applying a decomposed convolution to the filter.
• FIGURE 7 illustrates an example implementation 700 of the aforementioned decomposition.
• one memory bank 702 may be directly interfaced with one processing unit 704 of a computational network (neural network).
• Each memory bank 702 may store variables (neural signals), synaptic weights, and/or system parameters associated with a corresponding processing unit (neural processor) 704 delays, frequency bin information, regularization information and/or system metrics.
• the processing unit 704 may be configured to determine a number of separable filters to express a filter in the neural network and/or selectively apply a decomposed convolution to the filter.
• the process may also selectively apply a decomposed convolution to the filter, for example based on the complexity.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Health & Medical Sciences (AREA)
• Life Sciences & Earth Sciences (AREA)
• Biomedical Technology (AREA)
• Biophysics (AREA)
• General Health & Medical Sciences (AREA)
• General Physics & Mathematics (AREA)
• Evolutionary Computation (AREA)
• Computational Linguistics (AREA)
• Molecular Biology (AREA)
• Computing Systems (AREA)
• General Engineering & Computer Science (AREA)
• Data Mining & Analysis (AREA)
• Mathematical Physics (AREA)
• Software Systems (AREA)
• Artificial Intelligence (AREA)
• Neurology (AREA)
• Image Analysis (AREA)
• Electrically Operated Instructional Devices (AREA)

Priority Applications (2)

Applications Claiming Priority (4)

Publications (1)

Family

ID=55074837

Family Applications (2)

Family Applications After (1)

Country Status (8)

Cited By (2)

Families Citing this family (27)

Citations (2)

Family Cites Families (17)

• 2014
  • 2014-10-28 US US14/526,046 patent/US10402720B2/en active Active
  • 2014-10-28 US US14/526,018 patent/US10360497B2/en active Active
• 2015
  • 2015-07-13 CN CN201580038150.4A patent/CN106663222A/zh active Pending
  • 2015-07-13 KR KR1020177001193A patent/KR20170031695A/ko not_active Withdrawn
  • 2015-07-13 BR BR112017000229A patent/BR112017000229A2/pt not_active Application Discontinuation
  • 2015-07-13 WO PCT/US2015/040206 patent/WO2016010922A1/en not_active Ceased
  • 2015-07-13 EP EP15742477.1A patent/EP3170126A1/en not_active Ceased
  • 2015-07-13 CN CN201580038152.3A patent/CN106537421A/zh active Pending
  • 2015-07-13 AU AU2015289877A patent/AU2015289877A1/en not_active Abandoned
  • 2015-07-13 WO PCT/US2015/040221 patent/WO2016010930A1/en not_active Ceased
  • 2015-07-13 JP JP2017502110A patent/JP2017525038A/ja active Pending
  • 2015-07-13 EP EP15748340.5A patent/EP3170127A1/en not_active Ceased

Patent Citations (2)

Non-Patent Citations (4)

Also Published As

Similar Documents

Legal Events