CN111539522A - Construction method of large-scale NCS fault-tolerant framework based on fixed-size memristor array - Google Patents

Construction method of large-scale NCS fault-tolerant framework based on fixed-size memristor array Download PDF

Info

Publication number
CN111539522A
CN111539522A CN202010355449.5A CN202010355449A CN111539522A CN 111539522 A CN111539522 A CN 111539522A CN 202010355449 A CN202010355449 A CN 202010355449A CN 111539522 A CN111539522 A CN 111539522A
Authority
CN
China
Prior art keywords
sub
synaptic
connection
array
connection matrix
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010355449.5A
Other languages
Chinese (zh)
Inventor
陈松
吴雨婷
康一
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Science and Technology of China USTC
Original Assignee
University of Science and Technology of China USTC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Science and Technology of China USTC filed Critical University of Science and Technology of China USTC
Priority to CN202010355449.5A priority Critical patent/CN111539522A/en
Publication of CN111539522A publication Critical patent/CN111539522A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/061—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using biological neurons, e.g. biological neurons connected to an integrated circuit
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Neurology (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Algebra (AREA)
  • Databases & Information Systems (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Hardware Redundancy (AREA)

Abstract

The invention discloses a method for constructing a large-scale NCS fault-tolerant framework based on a memristor array with a fixed size, which not only improves the calculation reliability of a nerve morphology calculation system and accelerates the operation time of a framework algorithm, so that the scale of a neural network which can be processed by the system is greatly improved, but also reduces the resource consumption of the memristor array, and the adoption of the memristor array with the fixed size facilitates the subsequent general integrated design.

Description

Construction method of large-scale NCS fault-tolerant framework based on fixed-size memristor array
Technical Field
The invention relates to the field of neuromorphic computing systems, in particular to a method for constructing a large-scale NCS fault-tolerant framework based on a fixed-size memristor array.
Background
In recent years, artificial intelligence is rapidly developed, and the life style of human beings is greatly improved due to the advance of science and technology, so the real realization of brain-like intelligence is expected all over the world. But utilizing artificial intelligence for brain-like implementations mostly employ the traditional von neumann architecture, which has been the cornerstone of modern computer architectures since its birth in 1946. However, as the demand for high performance computing increases, the efficiency of the traditional von neumann computer architecture becomes lower and lower, and the structure with its arithmetic device separated from the memory causes the von neumann bottleneck, which limits the improvement of the computer performance. Correspondingly, the human brain is a typical non-von neumann framework, i.e., a parallel information processing mode integrating storage and calculation, and also has adaptive learning capability, fault-tolerant capability and anti-interference capability, so that the american scientist CarverMead proposed the concept of neuromorphic calculation in the last 80 s, and brain-like research implemented by hardware by referring to a neural network in bionics research was called neuromorphic calculation. The research of the neuromorphic calculation is still in the initial stage, and the emerging device which is used as the synapse simulation in the field and has the most potential is a memristor RRAM, and the resistance of the memristor RRAM changes under the action of voltage, so that the information storage and calculation can be realized. The memristor has nonlinearity and nonvolatility, has unique switching conversion mechanism, natural memory function, continuous input and output characteristics and nanoscale size, has huge application potential in the aspects of nonvolatile memories, large-scale integrated circuits, artificial neural networks, mode recognition, image processing and the like, is expected to improve the theory and application of the whole electronic circuit, and is an ideal device for realizing the synapse function.
In view of the great potential of memristors in the field of storage and the unique in-situ computing capability, the neural morphology computing system (NCS) based on the memristor array is widely researched and applied by the advantages of high computing speed, low design cost and the like. In the design of memristor-based neuromorphic computing systems, the weight of a connection is represented by the resistance of the memristor device. From a topological point of view, there are two methods of constructing neural networks: using discrete synapses and using memristive arrays: discrete synapses establish point-to-point connections between two neurons, while memristive arrays connect all of their input neurons to all of their output neurons. However, faults in memristors (SAFs) significantly degrade the computational accuracy of NCS. At present, fault-tolerant ideas of memristor arrays are mainly divided into the following categories:
1. redundant memristor arrays, by providing or dividing out independent extra regions, use their corresponding redundant parts when a memristor on a node fails.
2. When some part of devices have faults, the memristor with the faults is shielded through reconfiguration, and therefore the overall reliability is improved.
3. The fault tolerance is carried out by using the sum of the positive memristor array and the negative memristor array, and when a certain node of one array fails, the fault tolerance is carried out by tuning the corresponding node resistance value of the other array.
4. When a certain node fails, restoration can be performed in a software mode such as encoding and decoding, and although a failed device still exists, the reliability of the data layer is not affected.
Furthermore, since neural networks are large and sparse in most applications, this is contrary to the size constraints and high density of connections provided by memristive arrays. Meanwhile, the existing memristor array has different sizes and cannot meet the design of general integration. Therefore, how to propose a clustering mapping framework of a neuromorphic computing system based on memristive arrays, which tolerates more faults, with the hardware cost as low as possible, is very necessary.
Disclosure of Invention
The invention aims to provide a method for constructing a large-scale NCS fault-tolerant framework based on a fixed-size memristor array, which can realize universal integrated design, simultaneously considers hardware cost and mapping success rate, and reduces fault influence so as to improve the calculation reliability of a neuromorphic calculation system; in addition, the running time of the algorithm framework is accelerated, the mapping success rate and the utilization rate are improved, and the consumption of hardware resources is reduced.
The purpose of the invention is realized by the following technical scheme:
a method for constructing a large-scale NCS fault-tolerant framework based on a fixed-size memristor array comprises the following steps:
step 1, clustering synapse neurons in a synapse connection matrix by using an improved fault-tolerant clustering method for a synapse connection matrix of a given sparse neural network and a plurality of memristor arrays with given sizes to obtain a plurality of sub-synapse connection matrices;
step 2, calculating the probability of successful mapping of each sub-synaptic connection matrix, and thus judging whether each sub-synaptic connection matrix meets the size requirement of the memristor array; if the size requirement of the memristor array is not met, performing half-and-half transposition or dividing the memristor array into two new sub-synapse connection matrixes; then, calculating the utilization rate of each sub-synaptic connection matrix, comparing the utilization rate with a threshold value, and screening out the sub-synaptic connection matrix with the utilization rate larger than the threshold value; for sub-synaptic connection matrixes with the utilization rate smaller than the threshold, if the sum of the effective connection numbers in all the synaptic connection matrixes with the utilization rate smaller than the threshold accounts for the total connection proportion of the synaptic connection matrixes of the sparse neural network and exceeds a set proportion, merging the corresponding sub-synaptic connection matrixes, and then executing the step 2 again; repeating for multiple times to obtain a final clustering result;
step 3, deducing fault-tolerant memristor connection mapping based on two matching parts by adopting a heuristic algorithm based on matching according to a final clustering result;
and 4, carrying out plane planning on the NCS based on the memristor array with the given size, and estimating the area and the line length of the NCS.
According to the technical scheme provided by the invention, the calculation reliability of the neural morphology calculation system is improved, the frame algorithm operation time is accelerated, the scale of a neural network which can be processed by the system is greatly improved, the resource consumption of the memristor array is reduced, and the adoption of the memristor array with a fixed size facilitates the subsequent general integrated design.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
FIG. 1 is a schematic diagram of a neural network synapse structure based on a symmetric memristive array according to an embodiment of the present disclosure;
FIG. 2 is a flowchart of a method for constructing a large-scale NCS fault-tolerant framework based on a fixed-size memristor array according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a connection matrix of 4 × 4 incompletely connected neural networks according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a connection matrix of 5 × 9 incompletely connected neural networks according to an embodiment of the present invention;
fig. 5 is a schematic diagram of an extremely asymmetric matrix of 6 × 2 incompletely connected neural networks according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a mapped memristive array of FIG. 5 after binary partitioning according to an embodiment of the present disclosure;
fig. 7 is a schematic diagram of a mapped memristive array processed by the half-and-half transposition method according to the embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention are clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the invention provides a method for constructing a large-scale NCS fault-tolerant framework based on a memristor array with a fixed size.
Under the action of an external electric field, the memristor can generate transition between two or more resistance states, and the transition is nonvolatile, so that the memristor can realize continuous update of synaptic weights when being used for simulating synapses in an artificial neural network, and can also save the weights after power-off, thereby connectingThe weight of synapses may be represented by the resistance of memristors. The symmetric memristor array is easier to manufacture, and the computing power of the symmetric memristor array is better than that of the asymmetric memristor array, so that the symmetric memristor array is mainly considered in the embodiment of the invention. FIG. 1 is a neural network synapse structure based on a symmetric memristor array, with dimensions UCShowing that the structure is simulated by UCAn input sum UCU connected with output neuronC 2The intermediate synapse integrates the natural storage advantages of the memristor and the characteristics of cross array large-scale parallel processing, distributed information storage, self-organization, self-adaptation and the like, and has remarkable storage advantages including huge storage capacity, ultrahigh storage density, access speed and the like.
In a neural network, a presynaptic (i.e., input) neuron a sends a signal to the network, while a postsynaptic (i.e., output) neuron B receives information through a synapse. The synapse will apply different weights to the information during the transmission, which may be denoted as B ═ WA. Thus, W represents a matrix of synaptic connections (alternatively referred to as a matrix of synaptic weights), W is represented as a (1, 0) matrix, where a "1" indicates that there is a connection between two corresponding neurons, and a "0" is the opposite. Because the neural network with binary weights greatly reduces the memory size and replaces most arithmetic operations with bitwise operations, the input gives a binarized neural network W with n input neurons and m output neuronsn×mThe specific m and n values are determined by the imported neural network data set. The resistance values of memristors are large, it is difficult to program them to zero conductance corresponding to infinite resistance values, so when memristors are used, sparse matrices with a large number of zero values may result in large approximation errors. To solve the above problem, a region efficient design is achieved using a storage array structure that connects all input neurons to all its output neurons. In the embodiment of the invention, the synaptic connection matrix is decomposed into sub-blocks (clusters), the sub-blocks can be mapped to a group of memristor arrays with the same size, all zero rows and all zero rows are omitted, no computing resource is consumed, and only the rows with non-zero rows and the non-zero rows need to be mapped to the memristor arrays.
Due to the existing immature manufactureTechnical limitations, there are a wide variety of faults in memristors. The failure of a single memristor may be divided into soft and hard failures. A soft fault indicates that the actual resistance value deviates from the target value, but the resistance value is still adjustable. While the resistance of a hard fault tends to stick at the highest and lowest values, e.g., SA0 or SA 1: permanently open switch defects and open wordlines cause SA1 failure, leaving the memristor in a high-resistance state; the SA0 fault results in the memristor being in a low resistance state, which is not trimmable, due to excessive electrical formation defects, reset failures, and short defects. About 10% of memristors in a chip contain SAFS, and SA1 and SA0 have relatively high occurrence frequency in various hard faults, so the SA0 fault and the SA1 fault are mainly considered in the invention. The state of the memristive array is represented by memristive array matrix C, where the nonfaulty memristors are denoted as "0", and the SA1 and SA0 failures are denoted as "1" and "-1", respectively, and we assume that the failures are independent and evenly distributed. Defining sparsity of the neural network as 1 minus the number of actual connections in the input neural Network (NC)actualI.e. input neural network connection matrix Wn×m1) and the number of all possible connections (m × n), the utilization factor utilization being defined as the resulting number of available connections CC in all clustering matrices mapped onto memristive arrays (not including onto discrete synapses)actual(number of 1) and the total available connection number CC in all the corresponding memristor arraysall(number of memristor arrays × Uc 2) The ratio of (A) to (B); success mapping index (Succ) is defined as the probability of finding an effective mapping between a synaptic connection matrix and a memristive array.
sparsity＝1-NCactual/(m×n)
utilization＝CCactual/CCall
As shown in fig. 2, a method for constructing a large-scale NCS fault-tolerant framework based on a fixed-size memristor array according to an embodiment of the present invention mainly includes:
step 1, clustering synapse neurons in a given synapse connection matrix of a sparse neural network and a group of memristor arrays with given sizes by using a fault-tolerant clustering method to obtain a plurality of sub-synapse connection matrices.
In the embodiment of the invention, a given n × m input sparse neural network synapse connection matrix Wn×mGiven a plurality of sizes all being UC×UCThe memristive array of (1).
In the embodiment of the invention, the distance value between presynaptic neurons is used as the weight (calculated by adopting a formula 1), synaptic neurons in a synaptic connection matrix of an input neural network without all-zero rows and columns are clustered by using a modified METIS method, so that a plurality of sub-synaptic connection matrices (namely a plurality of clusters) are obtained, and all-zero rows and columns of the obtained sub-synaptic connection matrices also need to be removed.
METIS is a series of software packages for partitioning large grids, and has two key characteristics: METIS produces partitions 10% to 50% better than those produced by spectral clustering; METIS is one to two orders of magnitude faster than other commonly used segmentation algorithms. Because the clustering results of METIS are similar in size, in the embodiment of the invention, all memristive arrays are the same size (U)c) Similar sized clusters are needed for matching, which is contrary to METIS, and an improved METIS-based clustering method (MM) is proposed to divide large and sparse neural network connections into clusters.
In order to improve the fault tolerance of the NCS to the memristive array fault, presynaptic neurons connected to different postsynaptic neurons are grouped into one type in the embodiment of the invention; as shown in fig. 3, at A3And A4Between, XOR-3, A1And A2In the same way, so A3And A4Are grouped into a cluster, A1And A2Are grouped into a cluster. To achieve this, the weight definition of the edges in the algorithm needs to be changed, i.e. for two pre-synaptic neurons ApAnd AqThe distance measure between is defined as follows:
dist(Ap,Aq)＝XOR(Ap,Aq)/m (1)
where m is the number of postsynaptic neurons, and XOR represents an XOR operation.
As shown in fig. 4, 5 presynaptic neurons are set as 5 points, the distance value calculated by the distance measurement formula 1 is used as the side length of the corresponding presynaptic neuron to form a graph, and then the graph is converted into an input text file of METIS, so as to obtain a clustering result. The clustering obtained by the above method is superior to other clustering because it can be collocated with row/column reordering to reduce mapping errors.
In the embodiment of the present invention, the whole synaptic connection matrix Wn×mWill be initially divided into K1×K2Sub-synaptic connection matrix:
K1＝m/UC,K2＝n/UC(2)
step 2, calculating the probability that all the sub-synaptic connection matrixes are successfully mapped to the memristor array, and therefore judging whether each sub-synaptic connection matrix meets the size requirement of the memristor array or not; if the sub-synaptic connection matrix does not meet the size requirement of the memristor array, performing half-and-half transposition or dividing the sub-synaptic connection matrix into two new sub-synaptic connection matrices; then, calculating the utilization rate of each sub-synaptic connection matrix, comparing the utilization rate with a threshold value, and screening out the sub-synaptic connection matrix with the utilization rate larger than the threshold value; for sub-synaptic connection matrixes with the utilization rate smaller than the threshold, if the sum of the effective connection numbers in all the synaptic connection matrixes with the utilization rate smaller than the threshold accounts for the total connection proportion of the synaptic connection matrixes of the sparse neural network and exceeds a set proportion, merging the corresponding sub-synaptic connection matrixes, and then executing the step 2 again; and repeating for multiple times to obtain a final clustering result.
To improve fault tolerance, redundant rows and columns must be further preserved in the memristive array to better match the submatrices. Due to the process level limitation, the size of the existing memristive array capable of realizing reliability calculation cannot exceed 64 x 64, and meanwhile, in order to realize a general design, the invention requires that the sizes of all memristive arrays are the same, so that the size of a synaptic connection matrix is limited. The neural network needs to be divided into maximum connection matrixes corresponding to the memristor array, and the K obtained in the step 1 is1×K2Sub-synaptic connection matrix Wi(size M × N): i ═ 1, …, K1×K2M, N number of pre-synaptic and post-synaptic neurons, respectively; to determine whether it meets a given memristive array size requirement, first, a sub-synaptic connection matrix mapping is computed to have UC×UCProbability over memristive array of size:
Figure BDA0002473329700000061
wherein, PiIs a sub-synaptic connection matrix WiProbability of successful mapping to memristive array; p1And P0SA1 and SA0 failure rates, respectively; s1,S0Respectively a sub-synaptic connection matrix WiNumber of 1 and 0 in the middle r column, S1+S0N, 1 indicates that there is a connection between the two corresponding synaptic neurons, and 0 indicates that there is no connection between the two corresponding synaptic neurons.
For each sub-synaptic connection matrix WiP to be calculatediComparing with the set target probability Pt if PiLarger, it means the sub-synaptic connection matrix WiThe size of the memristor array is smaller than that of the memristor array, the size requirement of the memristor array is met at the moment, and good matching can be achieved subsequently; otherwise, the size requirement of the memristor array is not met, and the matching mapping standard cannot be reached. The target probability Pt referred to here can be set by itself according to the actual situation, and the present invention does not limit the value thereof.
If a sub-synaptic connection matrix Wi(M × N) does not meet the size requirement of the memristive array, half transposing is performed, or dividing into two new sub-synaptic connection matrices.
For the neural morphological network with the extremely asymmetric number of pre-synaptic neurons and post-synaptic neurons (such as 784 x 10), the utilization rate of the memristive array corresponding to the sub-synaptic connection matrix obtained through the steps is far lower than that of the neural network with the equal number of rows and columns in the synaptic connection matrix. For example, in the conventional scheme, a 6 × 2 neural network sub-connection matrix shown in fig. 5 is to be mapped to a memristive array with a fixed size of 5 × 5, and is divided into two matrices, namely 2 matrices of 3 × 2 and whiteColor filled circles, gray filled circles indicate the presence and absence of connections to other neurons, respectively. According to the scheme shown in fig. 6, two 3 × 2 matrices are respectively mapped and matched with one 5 × 5 memristor array, which greatly wastes the space of the memristor array. Therefore, the invention provides a half-and-half transposition method to improve the scale of the connection matrix which can be accommodated by the given memristor array and has the number of extremely asymmetric rows and columns, as shown in fig. 7, after transposition processing is performed on half of the extremely asymmetric matrix, the obtained new connection matrix can complete matching mapping only by one 5 × 5 memristor array, and the method greatly improves the utilization rate of the neural network of the extremely asymmetric connection matrix and greatly reduces the consumption of hardware resources. The sub-synaptic connection matrix W may be determined first byi(M × N) whether half transposition is possible:
(M/2+N)＜UC,N＜UC＜M
(N/2+M)＜UC,M＜UC＜N
if so, converting the half-and-half transposition of the original connection matrix into a new connection matrix, and then judging whether the new connection matrix meets the size requirement of the memristor array; and if the new connection matrix still does not meet the size requirement, performing binary division on the sub-synaptic connection matrix before half transposition to convert the sub-synaptic connection matrix into two small new connection matrices, and judging whether the new connection matrix meets the size requirement of the memristor array or not.
Then, the utilization rate of each sub-synaptic connection matrix is calculated, and the sub-synaptic connection matrix with the utilization rate larger than the threshold value is mapped onto the memristive array in the next step.
For sparse sub-synaptic connection matrixes with the utilization rate smaller than a threshold value, in order to minimize an outlier and maximize the scale of the synaptic connection matrixes matching the memristor array, calculating the sum of the numbers of effective connections 1 in the matrixes, calculating the proportion of the sum of the numbers of the effective connections to the total number of connections of the synaptic connection matrixes of a given input sparse neural network, and if the sum of the numbers of the effective connections is smaller than a set proportion (for example, the set proportion can be 5%), taking out the effective connections in the matrixes and mapping the effective connections to discrete synapses in a one-to-one manner; if the proportion is larger than the set proportion (for example, the set proportion can be 5%), arranging the sub-synaptic connection matrixes with too low utilization rate (namely, the utilization rate is smaller than the threshold value) from small to large according to the size, then combining the sub-synaptic connection matrixes pairwise from the small-size matrixes, and returning the new sub-synaptic connection matrixes after combination to the step 2 to start again; after iteration is repeated for a plurality of times until the proportion is smaller than the set proportion, the sub-synaptic connection matrixes with the proportion smaller than the set proportion are extremely sparse matrixes with very low utilization rate, and therefore the sub-synaptic connection matrixes are more suitable for one-to-one mapping to discrete synapses. The threshold and the setting ratio referred to herein may be set by themselves according to the actual situation, and the present invention does not limit the values thereof.
Through the above operations, K sub-synaptic connection matrixes, that is, final clustering results, are finally obtained, wherein K is included3A sub-synaptic connection matrix with a utilization rate greater than a threshold value, K3The matrices will be mapped into memristive arrays in the next step; and K4The utilization rate is less than the threshold value, the proportion of the sum of the effective connection numbers to the total connection number of the synaptic connection matrix of the given input sparse neural network is less than the sub-synaptic connection matrix with the set proportion, and K is4The connections in each connection matrix are directly mapped one-to-one onto discrete synapses.
And 3, independently generating faults in each memristor array according to the fault rates, wherein the fault rates of SA1 and SA0 are 9.04 percent and 1.75 percent respectively, and then deducing fault-tolerant memristor connection mapping based on two matching parts by adopting a matching-based heuristic algorithm according to a final clustering result.
According to the continuous execution of the steps 1-2, a final clustering result can be obtained, and in the embodiment of the invention, a matching-based heuristic algorithm is adopted to deduce the connection mapping of the fault-tolerant memristors based on two matching; specifically, the method comprises the following steps:
the sub-synaptic connection matrix W obtained by the above stepsi(M × N) and corresponding memristive array (U)C×UC) First, two decision vectors representing the mapping result are determined:
1) n × 1 input mapping vector Im: if the synapses of the sparse neural network are connected to the first matrixThe f column is assigned to the jth column of the memristive array, Im[f]J, i is presynaptic neuron, f is more than or equal to 1 and less than or equal to N, and j is more than or equal to 1 and less than or equal to UC2)1 × M output mapping vector Om: if the p-th row is assigned to the q-th row of the memristive array, Om[p]Q, the p-th behavioral postsynaptic neuron, p is more than or equal to 1 and less than or equal to M, and q is more than or equal to 1 and less than or equal to UC；
Note Wi：i＝1,…，K3For the ith sub-synaptic connection matrix, K, obtained by clustering3For the number of sub-synaptic connection matrices with utilization greater than the threshold, obtained in step 2, CiIs WiA memristive array mapped correspondingly; the purpose of the connection mapping is to establish a sub-synaptic connection matrix WiAnd memristor array CiThe row-column correspondence relationship is adopted to complete the mapping of synapses connected to memristors in the memristor array; first, to reduce the run time of the mapping method, the sub-synaptic connection matrix WiAnd memristor array CiThe columns of (a) are rearranged; since the "1" weight cannot be mapped to the SA0 failure, and the "0" weight cannot be mapped to the mapping rule of the SA1 failure, W tends to be mapped during the rearrangement processiColumns with more "1" values in them map to CiMiddle SA0 with fewer errors, and therefore, the sub-synapses are connected to the matrix WiRearranging according to the descending order of the number of 0 in each column, and memorizing the resistor array CiRearranged in ascending order of the number of 1's in the column, thereby obtaining the initial connection map Im(ii) a Then, a bipartite graph is constructed, the Kuhn Munkres algorithm is used to obtain the maximum bipartite matching, if the sub-synaptic connection matrix WiEach row of (A) has a corresponding memristive array CiMatch row, indicating that valid mapping was found, resulting in the final connection mapping OmOtherwise WiThe above process will be repeated again after performing the column random permutation. Because the clustering method is modified in the previous step and the weight calculation mode is redefined, so that the presynaptic neurons connected to different postsynaptic neurons are clustered into one class, from the matrix perspective, 0-1 pairs between rows and columns in the sub-synaptic connection matrix in the clustering result are as many as possible, if a certain memristor in the memristor array has a fault, and the fault memristor corresponds to the sub-synaptic connectionConnecting with the effective connection 1 in the matrix, obviously, the memristor cannot achieve effective matching mapping with the effective connection 1, so that the calculation result precision of the neuromorphic calculation system is influenced, and the W is randomly replaced by the columniThe columns are randomly rearranged, and as the modified clustering method enables 0-1 pairs in the matrix to be as many as possible, the column random replacement can replace the effective connection 1 with the surrounding connectionless 0 to match with the fault memristor, and the fault memristor is matched with the zero-connection neuron synapse without influencing the calculation result, so that the accuracy of the calculation system is maintained, and the fault-tolerant purpose is achieved.
And 4, according to the results, carrying out plane planning on the NCS based on the memristor array with the given size, and estimating the area and the line length of the NCS. The layout is represented herein by Sequence Pairs (SP), which are estimated using a semi-perimeter link length (HPWL) model for ground planning using IARFP.
In the embodiment of the invention, the sub-synaptic neuron connection matrix is connected with the memristor array through a line.
Through the method, a mapping solution and a layout plan of the synaptic connection matrix to a plurality of memristor arrays with fixed sizes can be obtained, and the method comprises the following steps:
1)K3a memristive array of the same size.
2) K capable of mapping to a fixed-size memristive array3Sub-synaptic connection matrix formed by clustering of synaptic connections of appropriate size, and K directly mapped to discrete synapses4A matrix of sub-synaptic connections.
3) Efficient mapping from the sub-connection matrix to the memristive array.
4) Layout planning based on memristive arrays NCS.
According to the technical scheme of the embodiment of the invention, the calculation reliability of the neural morphology calculation system is improved, the operation time of the frame algorithm is accelerated, the scale of the neural network which can be processed by the system is greatly improved, the resource consumption of the memristor array is reduced, and the adoption of the memristor array with a fixed size facilitates the subsequent general integrated design.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (6)

1. A method for constructing a large-scale NCS fault-tolerant framework based on a fixed-size memristor array is characterized by comprising the following steps of:
step 1, clustering synapse neurons in a synapse connection matrix by using an improved fault-tolerant clustering method for a synapse connection matrix of a given sparse neural network and a plurality of memristor arrays with given sizes to obtain a plurality of sub-synapse connection matrices;
step 2, calculating the probability of successful mapping of each sub-synaptic connection matrix, and thus judging whether each sub-synaptic connection matrix meets the size requirement of the memristor array; if the size requirement of the memristor array is not met, performing half-and-half transposition or dividing the memristor array into two new sub-synapse connection matrixes; then, calculating the utilization rate of each sub-synaptic connection matrix, comparing the utilization rate with a threshold value, and screening out the sub-synaptic connection matrix with the utilization rate larger than the threshold value; for sub-synaptic connection matrixes with the utilization rate smaller than the threshold, if the sum of the effective connection numbers in all the synaptic connection matrixes with the utilization rate smaller than the threshold accounts for the total connection proportion of the synaptic connection matrixes of the sparse neural network and exceeds a set proportion, merging the corresponding sub-synaptic connection matrixes, and then executing the step 2 again; repeating for multiple times to obtain a final clustering result;
step 3, deducing fault-tolerant memristor connection mapping based on two matching parts by adopting a heuristic algorithm based on matching according to a final clustering result;
and 4, carrying out plane planning on the NCS based on the memristor array with the given size, and estimating the area and the line length of the NCS.
2. The method for constructing the large-scale NCS fault-tolerant framework based on the fixed-size memristive array according to claim 1, wherein the clustering synaptic neurons in the synaptic connection matrix to obtain a plurality of sub-synaptic connection matrices comprises: removing all-zero rows and columns of a given m multiplied by n sparse neural network synaptic connection matrix W, clustering synaptic neurons by using a modified METIS method to obtain a plurality of sub-synaptic connection matrixes, and removing all-zero rows and columns of the sub-synaptic connection matrixes;
grouping presynaptic neurons connected to different postsynaptic neurons into one class; two presynaptic neurons ApAnd AqThe distance value between as a weight, the distance metric is defined as follows:
dist(Ap,Aq)＝XOR(Ap,Aq)/m
wherein m is the number of postsynaptic neurons, and XOR represents XOR operation;
taking the distance value calculated by the distance measurement formula as the side length of the corresponding presynaptic neuron to form a graph, and then converting the graph into an input text file of METIS, so as to obtain a clustering result; the entire synaptic connection matrix W will be initially divided into K1×K2Sub-synaptic connection matrix:
K1＝m/UC,K2＝n/UC
wherein, UC×UCThe size of the memristive array.
3. The method for constructing the large-scale NCS fault-tolerant framework based on the fixed-size memristive array according to claim 1, wherein the calculating the probability of successful mapping of each sub-synaptic connection matrix so as to determine whether each sub-synaptic connection matrix meets the size requirement of the memristive array comprises:
first, calculate the sub-synaptic connection matrix W of each M × NiMapping to having UC×UCProbability over memristive array of size:
Figure FDA0002473329690000021
wherein, PiIs a sub-synaptic connection matrix WiProbability of successful mapping to memristive array; p1And P0SA1 and SA0 failure rates, respectively; s1,S0Respectively a sub-synaptic connection matrix WiNumber of 1 and 0 in the middle r column, S1+S0N, 1 indicates that there is a connection between the respective two synaptic neurons, and 0 indicates that there is no connection between the respective two synaptic neurons;
p to be calculatediWith a set target probability PtMaking a comparison if PiLarger, it means the sub-synaptic connection matrix WiThe dimension of the memristor array is smaller than that of the memristor array, and the dimension requirement of the memristor array is met at the moment; otherwise, the size requirements of the memristive array are not met.
4. The method of claim 1, wherein if the size requirement of the memristive array is not met, performing a half-and-half transposition or dividing into two new sub-synaptic connection matrices comprises:
the size of the memory resistor array is UC×UCThe sub-synaptic connection matrix W with size M × N is determined byiWhether half transpose is possible:
(M/2+N)＜UC,N＜UC＜M
(N/2+M)＜UC,M＜UC＜N
if so, converting the half-and-half transposition of the original connection matrix into a new connection matrix, and then judging whether the new connection matrix meets the size requirement of the memristor array; and if the new connection matrix still does not meet the size requirement, performing binary division on the sub-synaptic connection matrix before half-and-half transposition to convert the sub-synaptic connection matrix into two small new connection matrices, and judging whether the new connection matrix meets the size requirement of the memristor array or not.
5. The method for constructing the large-scale NCS fault-tolerant framework based on the memristive array with fixed size according to claim 1,
calculating the proportion of the sum of the effective connections in all the sub-synaptic connection matrixes with the utilization rate smaller than the threshold value to the total connection number of the sub-synaptic connection matrixes of the given sparse neural network, and if the proportion is smaller than the set proportion, taking out the effective connections in the corresponding sub-synaptic connection matrixes and mapping the effective connections to the discrete synapses one by one; if the proportion is larger than the set proportion, arranging the corresponding sub-synapse connection matrixes from small to large according to the size, then combining the sub-synapse connection matrixes with small sizes in pairs, and then executing the step 2 again; repeating for multiple times until the proportion is smaller than the set proportion, and mapping the effective connections in the sub synapse connection matrix with the proportion smaller than the set proportion to discrete synapses in a one-to-one mode;
finally, K sub-synaptic connection matrixes are obtained, namely the final clustering result, wherein K is contained3A sub-synaptic connection matrix with a utilization greater than a threshold, and K4The sub-synaptic connection matrix has the utilization rate smaller than a threshold value and the proportion of the sum of the effective connection numbers to the total connection number of the synaptic connection matrix of the given input sparse neural network is smaller than a set proportion; k3The sub-synaptic connection matrix will be mapped into the memristive array; k4Connections in the sub-synaptic connection matrix are directly mapped one-to-one onto discrete synapses.
6. The method for constructing the large-scale NCS fault-tolerant framework based on the fixed-size memristor array according to claim 1, wherein the step of deriving the fault-tolerant memristor connection mapping based on the two-part matching by adopting the matching-based heuristic algorithm comprises the following steps:
sub-synaptic connection matrix W for M × N in final clustering resultiAnd corresponding size is UC×UCMemristive array, first, two decision vectors representing the mapping result are determined:
n × 1 input mapping vector Im: if the jth column of the synaptic connection matrix of the sparse neural network is assigned to the jth column of the memristive array, Im[f]J, i is presynaptic neuron, f is more than or equal to 1 and less than or equal to N, and j is more than or equal to 1 and less than or equal to UC1 × M output mapping vector Om: such asIf the p-th row is assigned to the q-th row of the memristive array, Om[p]Q, the p-th behavioral postsynaptic neuron, p is more than or equal to 1 and less than or equal to M, and q is more than or equal to 1 and less than or equal to UC；
Note Wi：i＝1,…，K3For the ith sub-synaptic connection matrix, K, obtained by clustering3For the number of sub-synaptic connection matrices with utilization greater than the threshold, obtained in step 2, CiIs WiA corresponding memristive array;
the purpose of the connection mapping is to establish a sub-synaptic connection matrix WiAnd memristor array CiThe row-column correspondence relationship is adopted to complete the mapping of synapses connected to memristors in the memristor array; first, the sub-synaptic connection matrix WiAnd memristor array CiAre rearranged to connect sub synapses in matrix WiRearranging according to the descending order of the number of 0 in each column, and memorizing the resistor array CiRearranged in ascending order of the number of 1's in the column, thereby obtaining the initial connection map Im(ii) a Then, a bipartite graph is constructed, the Kuhn Munkres algorithm is used to obtain the maximum bipartite matching, if the sub-synaptic connection matrix WiEach row of (A) has a corresponding memristive array CiMatch row, indicating that valid mapping was found, resulting in the final connection mapping OmOtherwise, the ith sub-synaptic connection matrix WiRepeating the connection mapping process again after the column random permutation is executed;
sub-synaptic connection matrix WiIs a matrix of (1, 0), wherein 0 indicates that there is no connection between two corresponding synaptic neurons, and 1 indicates that there is connection between two synaptic neurons; memristive array CiThe medium memristor comprises three states, 1 represents that the corresponding memristor has SA1 fault, 1 represents that the memristor has SA0 fault, and 0 represents no fault.
CN202010355449.5A 2020-04-29 2020-04-29 Construction method of large-scale NCS fault-tolerant framework based on fixed-size memristor array Pending CN111539522A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010355449.5A CN111539522A (en) 2020-04-29 2020-04-29 Construction method of large-scale NCS fault-tolerant framework based on fixed-size memristor array

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010355449.5A CN111539522A (en) 2020-04-29 2020-04-29 Construction method of large-scale NCS fault-tolerant framework based on fixed-size memristor array

Publications (1)

Publication Number Publication Date
CN111539522A true CN111539522A (en) 2020-08-14

Family

ID=71976146

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010355449.5A Pending CN111539522A (en) 2020-04-29 2020-04-29 Construction method of large-scale NCS fault-tolerant framework based on fixed-size memristor array

Country Status (1)

Country Link
CN (1) CN111539522A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112182495A (en) * 2020-09-14 2021-01-05 华中科技大学 Binary domain matrix operation circuit based on memristor

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
QI XU 等: "Fault tolerance in memristive crossbar-based neuromorphic computing systems", 《INTEGRATION》 *
YUTING WU 等: "Fault-Tolerant-Driven Clustering for Large Scale Neuromorphic Computing Systems", 《2020 IEEE INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE CIRCUITS AND SYSTEMS (AICAS)》 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112182495A (en) * 2020-09-14 2021-01-05 华中科技大学 Binary domain matrix operation circuit based on memristor
CN112182495B (en) * 2020-09-14 2024-04-19 华中科技大学 Binary domain matrix operation circuit based on memristor

Similar Documents

Publication Publication Date Title
Sun et al. Fully parallel RRAM synaptic array for implementing binary neural network with (+ 1,− 1) weights and (+ 1, 0) neurons
Xia et al. Fault-tolerant training with on-line fault detection for RRAM-based neural computing systems
US11544539B2 (en) Hardware neural network conversion method, computing device, compiling method and neural network software and hardware collaboration system
Chen et al. Mitigating effects of non-ideal synaptic device characteristics for on-chip learning
US9934463B2 (en) Neuromorphic computational system(s) using resistive synaptic devices
KR20220086694A (en) Memristor-based neural network training method and training device therefor
US7016886B2 (en) Artificial neurons including weights that define maximal projections
Schürmann et al. Edge of chaos computation in mixed-mode vlsi-a hard liquid
Schuman et al. Resilience and robustness of spiking neural networks for neuromorphic systems
CN111539522A (en) Construction method of large-scale NCS fault-tolerant framework based on fixed-size memristor array
Huang et al. Functional error correction for reliable neural networks
Müller et al. Randomized unregulated step descent for limited precision synaptic elements
Wang et al. Fault-tolerant deep neural networks for processing-in-memory based autonomous edge systems
de Lima et al. Quantization-aware in-situ training for reliable and accurate edge ai
Kazemi et al. A device non-ideality resilient approach for mapping neural networks to crossbar arrays
Akers et al. Limited interconnectivity in synthetic neural systems
Tanaka et al. Regularity and randomness in modular network structures for neural associative memories
Liu et al. Making the fault-tolerance of emerging neural network accelerators scalable
Ahmed et al. Online Fault-Tolerance for Memristive Neuromorphic Fabric Based on Local Approximation
Sugawara et al. Fault tolerant multi-layer neural networks with GA training
Huang et al. Improve robustness of deep neural networks by coding
Li et al. Zero-Space Cost Fault Tolerance for Transformer-based Language Models on ReRAM
Dong et al. Study on memristor-based 3-D convolutional neural network adapting to non-ideal effects
Rückert VLSI implementation of an associative memory based on distributed storage of information
Chandrachoodan et al. A Smoothed LASSO Based DNN Sparsification Technique

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20200814

RJ01 Rejection of invention patent application after publication