CN114527930A - Weight matrix data storage method, data acquisition method and device and electronic equipment - Google Patents
Weight matrix data storage method, data acquisition method and device and electronic equipment Download PDFInfo
- Publication number
- CN114527930A CN114527930A CN202110585872.9A CN202110585872A CN114527930A CN 114527930 A CN114527930 A CN 114527930A CN 202110585872 A CN202110585872 A CN 202110585872A CN 114527930 A CN114527930 A CN 114527930A
- Authority
- CN
- China
- Prior art keywords
- weight
- value
- matrix
- special
- connection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 239000011159 matrix material Substances 0.000 title claims abstract description 271
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000013500 data storage Methods 0.000 title claims abstract description 6
- 230000015654 memory Effects 0.000 claims description 129
- 238000004590 computer program Methods 0.000 claims description 6
- 230000003993 interaction Effects 0.000 claims description 6
- 239000000126 substance Substances 0.000 claims description 2
- 238000004364 calculation method Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 210000002569 neuron Anatomy 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 210000004556 brain Anatomy 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0658—Controller construction arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
Abstract
The present disclosure provides a method of data storage of a weight matrix, comprising: generating a connection matrix, a data type table and a special value table according to the weight matrix; storing a connection matrix, a data type table and a special value table; the connection matrix has the same structure as the weight matrix, and each element of the connection matrix is the connection type of the weight value of the corresponding element in the weight matrix; the connection types are divided into valid connections and invalid connections; the data type table comprises data types which are sequentially arranged and correspond to the weight values which are effectively connected in the weight matrix; the data types comprise a special value type and a plurality of preset value types, each preset value type corresponds to a preset weight value which is effectively connected, the special value type corresponds to a special weight value, and the other weight values which are effectively connected except the preset weight value are the special weight values; the special value table comprises all special weight values in the weight matrix which are sequentially arranged. The disclosure also provides a method and a device for acquiring data of the weight matrix, and electronic equipment.
Description
Technical Field
The present disclosure relates to the field of brain-like computing, and in particular, to a method for storing data of a weight matrix, a method and an apparatus for acquiring data of the weight matrix, and an electronic device.
Background
Brain-like computing techniques have been widely used in the fields of intelligence science, cognitive science, information processing, artificial intelligence, etc., such as image (video) recognition, image (video) processing, voice recognition, etc.
The human brain includes a large number of neurons (e.g., about 1000 billion), and the brain-like chip (neuromorphic chip) simulates the operation of the human brain by providing a large number of electrically simulated neurons. Many "weights (e.g., connection weights)" are also required in the operation of the neuron, and these weights may be in the form of a "weight matrix", i.e., each element of the weight matrix is a weight value (e.g., a connection weight value).
The weight matrix has a large number of weight values and high precision, so that the weight matrix occupies a large amount of storage space.
Disclosure of Invention
The disclosure provides a method for storing data of a weight matrix, a method and a device for acquiring data of the weight matrix and electronic equipment.
In a first aspect, an embodiment of the present disclosure provides a method for storing data of a weight matrix, including:
generating a connection matrix, a data type table and a special value table according to the weight matrix;
storing the connection matrix, the data type table and the special value table;
the connection matrix has the same structure as the weight matrix, and each element of the connection matrix is the connection type of the weight value of the corresponding element in the weight matrix; the data volume of a connection type is less than that of a weighted value, and the connection type is divided into an effective connection and an ineffective connection;
the data type table comprises data types which are sequentially arranged and correspond to the weight values which are effectively connected in the weight matrix; the data volume of one data type is less than that of one weight value, the data type comprises a special value type and a plurality of preset value types, each preset value type corresponds to one effectively connected preset weight value, the special value type corresponds to the special weight value, and the other effectively connected weight values except the preset weight value are the special weight values;
the special value table comprises all special weight values in the weight matrix which are sequentially arranged.
In some embodiments, before the generating the connection matrix, the data type table, and the special value table according to the weight matrix, the method further includes:
and determining a plurality of effectively connected weight values as preset weight values according to the weight matrix, and storing the preset weight values.
In some embodiments, the amount of data for each of the connection types is one bit.
In a second aspect, an embodiment of the present disclosure provides a method for acquiring data of a weight matrix, including:
determining the weight values of corresponding elements in the weight matrix according to a pre-stored connection matrix, a data type table and a special value table;
the connection matrix has the same structure as the weight matrix, and each element of the connection matrix is the connection type of the weight value of the corresponding element in the weight matrix; the data volume of a connection type is less than that of a weighted value, and the connection type is divided into an effective connection and an ineffective connection;
the data type table comprises data types which are sequentially arranged and correspond to the weight values which are effectively connected in the weight matrix; the data volume of one data type is less than that of one weight value, the data type comprises a special value type and a plurality of preset value types, each preset value type corresponds to one effectively connected preset weight value, the special value type corresponds to the special weight value, and the other effectively connected weight values except the preset weight value are the special weight values;
the special value table comprises all special weight values in the weight matrix which are sequentially arranged.
In some embodiments, the determining, according to the pre-stored connection matrix, the data type table, and the special value table, the weight value of the corresponding element in the weight matrix includes:
the address generator sequentially generates addresses of all elements in the connection matrix and inputs the addresses to the connection matrix memory;
the connection matrix memory inputs the connection type of the corresponding element of the connection matrix prestored in the connection matrix memory into the first gate and the data type memory according to the received address;
when the data type memory receives the effective connection, inputting the corresponding data type of the data type table prestored in the data type memory into the second gate and the special value judger;
the preset value memory synchronously inputs all preset weight values prestored in the preset value memory to the second gate;
the special value judger outputs an updating signal to the special value memory when the received data type is the special value type;
the special value memory inputs the current special weight value of the special value table prestored in the special value memory into the second gate, and updates the current special weight value to the next special weight value of the special value table when an updating signal is received;
the second gate selects one of the received current special weight value and each preset weight value to output according to the type of the received data;
the invalid value memory inputs the prestored invalid weight value into the first gate;
the first gate selects one output from the received invalid weight value and the output of the second gate according to the received connection type.
In some embodiments, after determining the weight values of the corresponding elements in the weight matrix according to the pre-stored connection matrix, the data type table, and the special value table, the method further includes:
and acquiring the determined weight value.
In some embodiments, after determining the weight values of the corresponding elements in the weight matrix according to the pre-stored connection matrix, the data type table, and the special value table, the method further includes:
restoring at least partial region of the weight matrix according to the determined weight value, and storing the restored region of the weight matrix into a temporary storage space;
and acquiring the weight value of the weight matrix from the temporary storage space.
In a third aspect, an embodiment of the present disclosure provides an apparatus for acquiring data of a weight matrix, including an address generator, a special value memory, an invalid value memory, a preset value memory, and at least one acquiring unit; wherein, the first and the second end of the pipe are connected with each other,
the address generator is used for sequentially generating addresses of all elements in the connection matrix and inputting the addresses to the connection matrix memory of the corresponding acquisition unit;
the preset value memory is used for storing all preset weight values and synchronously inputting all the preset weight values to the second gates of all the acquisition units;
the invalid value memory is used for storing invalid weight values and inputting the invalid weight values to the first strobers of all the acquisition units;
the special value memory is used for storing a special value table, inputting the current special weight value of the special value table to the second gate, and updating the current special weight value to the next special weight value of the special value table when an updating signal is received;
each of the acquisition units includes:
the connection matrix memory is used for storing the connection matrix and inputting the connection type of the corresponding element of the connection matrix to the first gate and the data type memory according to the received address;
the data type memory is used for inputting the corresponding data types of the data type table prestored in the data type memory into the second gate and the special value judger when the effective connection is received;
a special value judger for outputting an update signal to the special value storage when the received data type is a special value type;
the second gate is used for selecting one of the received current special weight value and each preset weight value to output according to the type of the received data;
a first gate for selecting one output from the received invalid weight value and the output of the second gate according to the received connection type;
the connection matrix has the same structure as the weight matrix, and each element of the connection matrix is the connection type of the weight value of the corresponding element in the weight matrix; the data volume of a connection type is less than that of a weighted value, and the connection type is divided into an effective connection and an ineffective connection;
the data type table comprises data types which are sequentially arranged and correspond to the weight values which are effectively connected in the weight matrix; the data volume of one data type is less than that of one weight value, the data type comprises a special value type and a plurality of preset value types, each preset value type corresponds to one effectively connected preset weight value, the special value type corresponds to the special weight value, and the other effectively connected weight values except the preset weight value are the special weight values;
the special value table comprises all special weight values in the weight matrix which are sequentially arranged.
In some embodiments, the number of the acquisition units is plural;
each of the acquisition units further includes: an output enabler connected to an output of the first gate;
the device further comprises: a logic controller connected between each special value judger and the special value memory and connected to each output enabler; the logic controller is used for sending the updating signals to the special value memory, sequentially inputting the updating signals to the special value memory when receiving the updating signals at the same time, and controlling the output enablers of the acquisition units corresponding to the updating signals to be sequentially conducted.
In a fourth aspect, an embodiment of the present disclosure provides an electronic device, including:
one or more processors;
a memory having one or more computer programs stored thereon:
one or more I/O interfaces connected between the processor and the memory and configured to realize information interaction between the processor and the memory;
the one or more computer programs, when executed by the one or more processors, enable:
a method of storing data of any of the above weight matrices;
and/or the presence of a gas in the gas,
any one of the above methods for obtaining data of the weight matrix.
In the embodiment of the disclosure, only the connection matrix, the data type table, the special value table and the preset weight value are actually stored, wherein the connection matrix only has the connection type of the weight value, the data type table only has the data type of the effectively connected weight value, and the data volume of each connection type and data type is obviously much smaller than that of the weight value, so the storage space occupied by the connection matrix and the data type table is much smaller than that occupied by the weight matrix; the special value table and the preset weight value are only few specific weight values in the weight matrix, so that the occupied storage space of the special value table and the preset weight value is also small.
Therefore, in the embodiment of the present disclosure, the storage space occupied by all actually stored data is much smaller than the storage space occupied by the weight matrix, but all information (data) of the weight matrix is actually included, so that the storage space is greatly saved.
Drawings
The accompanying drawings are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the detailed description serve to explain the disclosure and not limit the disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing in detail embodiments with reference to the attached drawings in which:
fig. 1 is a flowchart of a method for storing data of a weight matrix according to an embodiment of the present disclosure;
fig. 2 is a diagram of a corresponding relationship between a weight matrix and a connection matrix in a method for storing data of the weight matrix according to an embodiment of the present disclosure;
FIG. 3 is a flow chart of a method for storing data of another weight matrix provided by an embodiment of the present disclosure;
fig. 4 is a flowchart of a method for acquiring data of a weight matrix according to an embodiment of the present disclosure;
fig. 5 is a flowchart of another method for data acquisition of a weight matrix according to an embodiment of the present disclosure;
fig. 6 is a flowchart of another method for data acquisition of a weight matrix according to an embodiment of the present disclosure;
fig. 7 is a block diagram illustrating an apparatus for acquiring data of a weight matrix according to an embodiment of the present disclosure;
fig. 8 is a block diagram of another apparatus for acquiring data of a weight matrix according to an embodiment of the present disclosure;
fig. 9 is a block diagram of an electronic device according to an embodiment of the present disclosure.
Detailed Description
In order to make those skilled in the art better understand the technical solution of the present disclosure, the following describes in detail a method for storing data of a weight matrix, a method and apparatus for acquiring data of a weight matrix, and an electronic device provided in the present disclosure with reference to the accompanying drawings.
The present disclosure will be described more fully hereinafter with reference to the accompanying drawings, but the illustrated embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth in the disclosure. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
The present disclosure may be described with reference to plan and/or cross-sectional views by way of idealized schematic illustrations of the present disclosure. Accordingly, the example illustrations can be modified in accordance with manufacturing techniques and/or tolerances.
Embodiments of the present disclosure and features of embodiments may be combined with each other without conflict.
The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in this disclosure, the term "and/or" includes any and all combinations of one or more of the associated listed items. As used in this disclosure, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. The terms "comprises," "comprising," "made of … …," as used in this disclosure, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
Unless otherwise defined, all terms (including technical and scientific terms) used in this disclosure have the same meaning as commonly understood by one of ordinary skill in the art. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The present disclosure is not limited to the embodiments shown in the drawings, but includes modifications of configurations formed based on a manufacturing process. Thus, the regions illustrated in the figures have schematic properties, and the shapes of the regions shown in the figures illustrate specific shapes of regions of elements, but are not intended to be limiting.
In a first aspect, the embodiments of the present disclosure provide a method for storing data of a weight matrix, which is used to store data (information) of the weight matrix for a subsequent brain-like computing process.
Wherein the weight matrix is used for storing weights (such as weights of neurons) used in brain-like calculations. The weight matrix may be in two dimensions or in higher dimensions, and each element in the weight matrix is a specific weight value.
The precision of all weight values in the weight matrix is the same, for example, in the form of int8, fp16, and the like.
Referring to fig. 1, a method for storing data of a weight matrix according to an embodiment of the present disclosure includes:
and S101, generating a connection matrix, a data type table and a special value table according to the weight matrix.
S102, storing a connection matrix, a data type table and a special value table.
The connection matrix has the same structure as the weight matrix, and each element of the connection matrix is the connection type of the weight value of the corresponding element in the weight matrix; the data volume of a connection type is less than that of a weighted value, and the connection type is divided into an effective connection and an ineffective connection;
the data type table comprises data types which are sequentially arranged and correspond to the weight values which are effectively connected in the weight matrix; the data volume of one data type is less than that of one weight value, the data type comprises a special value type and a plurality of preset value types, each preset value type corresponds to one effectively connected preset weight value, the special value type corresponds to the special weight value, and the other effectively connected weight values except the preset weight value are the special weight values;
the special value table comprises all special weight values in the weight matrix which are sequentially arranged.
In the embodiment of the present disclosure, according to the content (data) of the weight matrix to be stored, the corresponding connection matrix, data type table, and special value table are generated, and the connection matrix, the data type table, and the special value table are stored to reflect the information (data) of the weight matrix for the subsequent brain-like calculation process.
Referring to fig. 2, the form and size of the connection matrix and the weight matrix are the same, so that each element of the connection matrix necessarily corresponds to an element in the same relative position in the weight matrix. Furthermore, the value of each element of the connection matrix is the "connection type" to which the weight value of its corresponding element in the weight matrix belongs.
In the weight matrix, a part of the weight values are "useful" or "effective" in the subsequent brain-like calculation process, which is called "weighted" or "effective weight value", and the connection type of the weight values is "effective connection"; meanwhile, in the weight matrix, another part of the weight values is actually "useless" or "invalid" in the subsequent brain-like calculation process, which is called as "no weight" or "invalid weight value", for example, the weight value is 0, and the connection type of such weight value is "invalid connection".
It can be seen that the weight values of the above invalid connections actually correspond to 0 elements in the weight matrix, and the weight values of the valid connections actually correspond to non-0 elements in the weight matrix.
Since the weight values of the above invalid connections do not play a practical role, it is not necessary to store their "specific values", but rather, the above connection matrix indicates which elements in the weight matrix are 0 elements (weight values of the invalid connections) and which are not 0 elements (weight values of the valid connections).
For example, referring to FIG. 2, a connection matrix may have a1 for an active connection (non-0 element) and a 0 for an inactive connection (0 element).
Further, each weight value of an active connection of the weight matrix has a certain "data type". Specifically, many of the weight values of the effective connection of the weight matrix are repeatedly appeared for many times, such as some excitement weight values, suppression weight values, null weight values, and the like, so that each weight value appearing for many times (the weight value of the effective connection) can be defined as a preset weight value, and the preset weight value belongs to a 'preset value type'; while the other weight values (effectively connected weight values) in the weight matrix appear less, they are all defined as special weight values and belong to a "special value type".
Thus, the data types corresponding to the weight values of the effective connection of the weight matrix can be stored in a predetermined order (for example, in a row-by-row arrangement order) as a data type table. Thus, a data type table is actually a series of ordered data (data types), which may also be considered a "one-dimensional matrix" or "sequence".
For example, referring to fig. 2 and tables 1 and 2 below, the preset weight values in the weight matrix may include a weight value a, a weight value B, and a weight value C, so that the data volume of each data type in the data type table may be 2bits, for example, three preset weight values of the weight value a, the weight value B, and the weight value C (preset value types) are respectively corresponding to 01, 10, and 00, and 11 corresponds to a special weight value (special value type).
TABLE 1 correspondence of data types and weight values
TABLE 2 data type Table
| 11 | 01 | 00 | 01 | 10 | 11 | 00 | 11 | … | … |
All the special weight values of the weight matrix (i.e. the weight values corresponding to all the special value types in the data type table) can also be stored in sequence (in the same sequence as the data type table) as the special value table. Thus, a special value table is actually a series of ordered data (special weight values), which can also be considered as a "one-dimensional matrix" or "sequence".
Referring to fig. 2 and table 3 below, the special value table may be in the form of:
TABLE 3 Special value Table
| 3.2364 | -0.1231 | 63.7089 | … | … | … | … | … |
Matrix-based format requirements, where the data precision is the same for all elements. Thus, for example, if the data size of each weight value is 16bits, the memory space occupied by the weight matrix having N elements is 16 × Nbits.
As before, each element of the connection matrix may be only 1 or 0, and occupies 1bit of storage space, so that the storage space occupied by the connection matrix corresponding to the above weight matrix is Nbits.
Assuming that 80% of the weights in the weight matrix belong to invalid weight values and only 20% of the weights belong to valid weight values, the data type table occupies 20% by 2Nbits as before.
Assuming that 5% of the weights in the weight matrix belong to the special weight values, the storage space occupied by the special value table is 5% by 16 Nbits.
Therefore, the total storage space occupied by the connection matrix, the data type table and the special value table is N + 20% by 2 + N + 5% by 16N-2.2 Nbits, even if the storage space occupied by each preset weight value is added, only 3 by 16-48 bits is added, and when N is very large, the storage space is far smaller than 16 by Nbits, so that the storage space can be greatly saved by the mode of the embodiment of the disclosure.
Obviously, the larger the proportion of the weight values (invalid weight values) of the invalid connections in the weight matrix is, and the larger the proportion of the preset weight values in the weight values of the valid connections in the weight matrix is, the better the effect of saving the storage space by the method of the embodiment of the present disclosure is. Therefore, the embodiment of the present disclosure is particularly suitable for sparse matrices with relatively large invalid weight values.
For example, the weight value (invalid weight value) for invalid connections may be a weight matrix of at least 40% (number ratio among ownership weight values), or at least 60%, or at least 80%.
For another example, embodiments of the present disclosure may be used to preset a weight matrix with weight values at least 40% (by number of the weight values in the active connections), or at least 60%, or at least 80%.
In the embodiment of the disclosure, only the connection matrix, the data type table, the special value table and the preset weight value are actually stored, wherein the connection matrix only has the connection type of the weight value, the data type table only has the data type of the effectively connected weight value, and the data volume of each connection type and data type is obviously much smaller than that of the weight value, so the storage space occupied by the connection matrix and the data type table is much smaller than that occupied by the weight matrix; the special value table and the preset weight value are only few specific weight values in the weight matrix, so that the occupied storage space of the special value table and the preset weight value is also small.
Therefore, in the embodiment of the present disclosure, the storage space occupied by all actually stored data is much smaller than the storage space occupied by the weight matrix, but all information (data) of the weight matrix is actually included, so that the storage space is greatly saved.
In some embodiments, the amount of data per connection type is one bit.
As before, since there are only two connection types, the data amount per connection type is one bit (1bit), such as 1 or 0 above.
Referring to fig. 3, in some embodiments, determining the weight values of the plurality of active connections therein as the preset weight values according to the weight matrix, and storing the preset weight values (S101), further includes:
and S100, determining a plurality of weight values as preset weight values according to the weight matrix, and storing the preset weight values.
As before, the selection of the preset weight value has an obvious influence on the effect of saving the storage space, and if the preset weight value is too small, the special weight value is too much, and the storage space occupied by the special value table is too large; if the types of the preset weight values are too many, the data size of each data type is too large, and the storage space occupied by the data type table is too large.
Therefore, according to the specific condition of the weight matrix to be stored currently, the weight values appearing for a plurality of times are selected as the preset weight values, and the selected preset weight values are stored, so that the effect of saving the storage space is better achieved.
Of course, instead of setting the preset weight values according to each weight matrix, a general preset weight value may be preset and stored according to the overall situation of a large number of weight matrices.
In a second aspect, the embodiments of the present disclosure provide a method for acquiring data of a weight matrix, which uses data of the weight matrix obtained by the above method to acquire a weight value required in the weight matrix for a subsequent brain-like calculation process.
Referring to fig. 4, a method for acquiring data of a weight matrix according to an embodiment of the present disclosure includes:
s201, determining the weight values of corresponding elements in the weight matrix according to a pre-stored connection matrix, a data type table and a special value table.
The connection matrix has the same structure as the weight matrix, and each element of the connection matrix is the connection type of the weight value of the corresponding element in the weight matrix; the data volume of a connection type is less than that of a weighted value, and the connection type is divided into an effective connection and an ineffective connection;
the data type table comprises data types which are sequentially arranged and correspond to the weight values which are effectively connected in the weight matrix; the data volume of one data type is less than that of one weight value, the data type comprises a special value type and a plurality of preset value types, each preset value type corresponds to one effectively connected preset weight value, the special value type corresponds to the special weight value, and the other effectively connected weight values except the preset weight value are the special weight values;
the special value table comprises all special weight values in the weight matrix which are sequentially arranged.
It can be seen that the data of the weight matrix stored in the above manner is not a "weight matrix", so when a weight value of an element in the weight matrix needs to be used, it needs to be determined whether the weight value is an effective connection or an ineffective connection according to a corresponding element (an element at the same relative position) in the connection matrix, when the weight value is an effective connection, the data type of the weight value is continuously determined according to the data type table, and then it is determined which preset weight value the weight value is or which special weight value in the special value table, that is, which specific weight value is obtained.
Referring to fig. 5, in some embodiments, after determining the weight values of the corresponding elements in the weight matrix according to the pre-stored connection matrix, the data type table, and the special value table (S201), the method further includes:
and S202, acquiring the determined weight value.
As a form of the embodiment of the present disclosure, when it is needed to use the weight values of the elements in the weight matrix, the corresponding weight values are directly obtained in the above manner, and are directly read out and utilized.
Referring to fig. 6, in some embodiments, after determining the weight values of the corresponding elements in the weight matrix according to the pre-stored connection matrix, the data type table, and the special value table (S201), the method further includes:
s203, restoring at least partial region of the weight matrix according to the determined weight value, and storing the region of the restored weight matrix into a temporary storage space;
and S204, acquiring the weight value of the weight matrix from the temporary storage space.
As another form of the embodiment of the present disclosure, a part or all of the weight values in the weight matrix may be obtained first, that is, the weight matrix or a part thereof is "restored", and the restored weight matrix or a part thereof is stored in the temporary storage space, so that the temporary storage space is equivalent to a "temporary weight matrix", and the weight values may be obtained from the "temporary weight matrix" when necessary.
Wherein, when the data in the temporary weight matrix is used up, the 'another part' of the weight matrix can be written into the temporary storage space again.
The following describes a specific manner of obtaining the weight values of the elements in the weight matrix.
It should be understood that there is no necessary relationship between how the weight values of the elements in the weight matrix are derived in particular and how the weight values are subsequently utilized. For example, the weight values obtained in the following manner may be directly obtained (e.g., S202), or may be used to form the above "temporary weight matrix (e.g., S203, S204)".
It should be understood that the following specific manner of deriving the elements in the weight matrix is exemplary only, and is not intended to limit the scope of the present disclosure.
Referring to fig. 7, in some embodiments, determining the weight values of the corresponding elements in the weight matrix according to the pre-stored connection matrix, the data type table, and the special value table (S201) includes:
a1, address generator generates the address of each element in the connection matrix in turn, and inputs the address to the connection matrix memory.
And A2, inputting the connection type of the corresponding element of the connection matrix pre-stored in the connection matrix memory to the first gate and the data type memory according to the received address.
A3, the data type memory inputs the corresponding data type of the data type table pre-stored in it to the second gate and the special value decider when receiving the effective connection.
A4, the preset value memory synchronously inputs each preset weight value prestored in the preset value memory to the second gate.
A5, the special value determiner outputs an update signal to the special value memory when the received data type is a special value type.
A6, the special value memory inputs the current special weight value of the special value table pre-stored therein to the second gate, and updates the current special weight value to the next special weight value of the special value table when receiving the update signal.
And A7, selecting one of the received current special weight value and each preset weight value to output by the second gate according to the type of the received data.
A8, the invalid value memory inputs the weight value prestored in it to the first gate.
A9, the first gate selects one output from the received invalid weight value and the output of the second gate according to the received connection type.
It should be understood that each step above describes only the work or function performed by the corresponding structure, and therefore the order of description of the above steps does not represent the necessary order of execution.
Referring to fig. 7, the address generator sequentially (corresponding to the storage order of the data type table, the special value table) generates and inputs addresses to the connection matrix memory.
Wherein, the generated address is used for indicating that the weight value of which element in the weight matrix needs to be determined currently; so for simplicity, the addresses of the corresponding elements in the weight matrix and the connection matrix may be the same.
The connection matrix memory inputs the connection type (e.g., 1 or 0 above) of the element to which the address refers to the first strobe and the data type memory according to the received address.
When the received connection type is "valid connection (e.g., 1)", the data type memory inputs the corresponding data type of the data type table stored therein, i.e., the data type (e.g., 01, 10, 00, 11 above) corresponding to the weight value of the address of the valid connection, to the second gate.
For example, the data type storage may "update" the "next" data type of the output data type table each time the "effective connection" is received, so as to ensure that the ordinal number of the data type output by the data type storage is equal to the ordinal number of the corresponding weight value in all the effectively connected weight values of the weight matrix.
If the connection type received by the data type memory is "invalid connection (e.g. 0)", it may or may not continue to output the previous data type, and will not be described in detail herein.
Meanwhile, the preset value memory (e.g., a register) inputs (e.g., continuously inputs) all the preset weight values (e.g., the above weight value a, weight value B, and weight value C) stored therein to the second gate synchronously; the special value memory also inputs (e.g., continuously inputs) the current special weight value stored therein to the second gate.
Therefore, the second gate receives all the preset weight values and the current special weight value at the same time.
Meanwhile, the second gate also receives the data type (from the data type memory) of the element needing to determine the weight value at present, so that the second gate can determine which weight value should be output according to the data type (for example, if the data type is 01, the weight value A is output, if the data type is 10, the weight value B is output, if the data type is 00, the current special weight value is output), that is, the input of which way is gated is determined; the other "way" weight values received by the second gate are not corresponding to the current element and are not output.
If the second gate does not receive the data type, it may not generate any output.
The data type memory also inputs the data type to the special value judger, and the special value judger outputs an update signal to the special value memory when the received data type is a special value type (such as 11), so that the special value memory can update the current special weight value, namely, the next special weight value in the special value table is selected as a new current special weight value and output to the gate.
It can be seen that the addresses generated by the address generator are "in sequence", the data types in the data type table are also "in sequence" in the same manner, and the special weight values in the special value table in the special value storage are also "in sequence" in the same manner, so that the addresses and the current special weight values can be ensured to be "synchronous" by updating the current special weight values when the data types are special value types.
Further, the invalid value memory also inputs (e.g., continuously inputs) an invalid weight value (no weight, e.g., weight value of 0) prestored therein to the first gate, i.e., the first gate receives both the invalid weight value and an output (a preset weight value or a current special weight value) of the second gate.
Therefore, the first gate can select one output from the received invalid weight value and the output of the second gate according to the received connection type (from the connection matrix memory), specifically, the invalid weight value is output when invalid connection is received, and the output of the second gate (the preset weight value or the current special weight value, which both belong to the valid weight value) is output when valid connection is received, so that the invalid weight value is accurately output.
Referring to fig. 8, in some embodiments, the apparatus for data acquisition of a weight matrix includes acquisition units, each of which includes the above connection matrix memory, a data type memory, a special value determiner, a first gate, and a second gate.
When the number of the acquisition units is plural, each acquisition unit further includes: an output enabler connected to an output of the first gate;
the apparatus of the disclosed embodiment further comprises: a logic controller connected between each special value judger and the special value memory and connected to each output enabler; the logic controller is used for sending the updating signals to the special value memory, sequentially inputting the updating signals to the special value memory when receiving the updating signals at the same time, and controlling the output enablers of the acquisition units corresponding to the updating signals to be sequentially conducted.
As mentioned above, each obtaining unit can determine the weight value of one element, so if the weight values of a plurality of elements (such as the weight values of a plurality of adjacent elements in a row of the weight matrix) are to be determined simultaneously in a "parallel" manner, a plurality of (two in fig. 8 for example) obtaining units need to be used simultaneously, and each obtaining unit operates synchronously, and each obtaining unit determines one weight value.
When there are multiple acquiring units, if two structures belonging to the acquiring units are involved in interaction, it shall mean that two structures inside the same acquiring unit interact without special description.
Referring to fig. 8, when there are a plurality of fetch units, the special value memory, the preset value memory, and the invalid value memory may be "common", that is, they synchronously output the current special weight value and the preset weight value to the second gates of all fetch units, and output the invalid weight value to the first gates of all fetch units, thereby simplifying the hardware structure.
The address generator is also "common", and the difference is that the address of the element to be processed by each acquisition unit is respectively input to the connection matrix memories of the corresponding acquisition units (i.e. the addresses input to the connection matrix memories at the same time are different but are arranged in sequence), and then the connection matrix memories of the acquisition units input the corresponding connection types to the first gate and the data type memory of the acquisition unit where the first gate is located, so that the first gate outputs the weight values of the corresponding elements.
When the weight values of a plurality of elements output in parallel have a plurality of special weight values, the data type received by the second gating devices of a plurality of acquisition units is a special value type, and the current special weight values are respectively output to the first gating devices; however, the special weight values that should be output by the plurality of different fetch units (output by the first gate) are actually different, i.e., the special value memory needs to be updated a plurality of times for the current special weight value,
for this reason, referring to fig. 8, a logic controller and an output enabler are also required.
If the logic controller receives only one update signal at the same time, the update signal is directly forwarded to the special value memory.
When there are multiple special weight values in the weight values of multiple elements output in parallel, the logic controller must receive multiple update signals (e.g., two update signals) at the same time, so that it needs to input the update signals to the special value memory in sequence (e.g., first input the first update signal to the special value memory, and then input the second update signal to the special value memory after a certain time). Furthermore, the special value memory updates the current special weight value for a plurality of times and outputs different current special weight values at different times (for example, after receiving the first update signal and before receiving the second update signal, the first current special weight value is output, and after receiving the second update signal, the second current special weight value is output).
Accordingly, when a plurality of update signals are received simultaneously, the logic controller needs to control the output enablers of the acquiring units corresponding to the update signals to be sequentially turned on in a polling manner (for example, when the special value memory outputs a first current special weight value, the output enabler of the acquiring unit corresponding to the first update signal is turned on, and the output enabler of the acquiring unit corresponding to the second update signal is turned off, and when the special value memory outputs a second current special weight value, the output enabler of the acquiring unit corresponding to the second update signal is turned on, and the output enabler of the acquiring unit corresponding to the first update signal is turned off).
In this way, the plurality of acquiring units corresponding to the respective update signals can output (in parallel, but not completely "simultaneously") the special weight values of the respective corresponding elements.
When receiving a plurality of updating signals at the same time, the logic controller controls the output enablers of other acquisition units which do not correspond to the updating signals to keep a conducting state; when only one updating signal is received or the updating signal is not received at the same time, the logic controller should control the output enablers of all the acquisition units to keep the conducting state, and the detailed description is not repeated here
The specific implementation manner of the logic controller is various, for example, the logic controller may implement the adjustment of the operating state in a metronomic manner through the multiplier-adder, and will not be described in detail herein.
In a third aspect, referring to fig. 7, an embodiment of the present disclosure provides an apparatus for acquiring data of a weight matrix, which includes an address generator, a special value memory, an invalid value memory, a preset value memory, and at least one acquiring unit; wherein the content of the first and second substances,
the address generator is used for sequentially generating addresses of all elements in the connection matrix and inputting the addresses to the connection matrix memory of the corresponding acquisition unit;
the preset value memory is used for storing all preset weight values and synchronously inputting all the preset weight values to the second gates of all the acquisition units;
the invalid value memory is used for storing invalid weight values and inputting the invalid weight values to the first strobers of all the acquisition units;
the special value memory is used for storing the special value table, inputting the current special weight value of the special value table to the second gate, and updating the current special weight value to the next special weight value of the special value table when an updating signal is received;
each acquisition unit includes:
the connection matrix memory is used for storing the connection matrix and inputting the connection type of the corresponding element of the connection matrix to the first gate and the data type memory according to the received address;
the data type memory is used for inputting the corresponding data type of the data type table prestored in the data type memory into the second gate and the special value judger when the effective connection is received;
a special value judger for outputting an update signal to the special value storage when the received data type is a special value type;
the second gate is used for selecting one of the received current special weight value and each preset weight value to output according to the type of the received data;
a first gate for selecting one output from the received invalid weight value and the output of the second gate according to the received connection type;
the connection matrix has the same structure as the weight matrix, and each element of the connection matrix is the connection type of the weight value of the corresponding element in the weight matrix; the data volume of a connection type is less than that of a weighted value, and the connection type is divided into an effective connection and an ineffective connection;
the data type table comprises data types which are sequentially arranged and correspond to the weight values which are effectively connected in the weight matrix; the data volume of one data type is less than that of one weighted value, the data type comprises a special value type and a plurality of preset value types, each preset value type corresponds to one effectively connected preset weighted value, the special value type corresponds to the special weighted value, and the other effectively connected weighted values except the preset weighted value are all special weighted values;
the special value table comprises all special weight values in the weight matrix which are sequentially arranged.
Referring to fig. 8, in some embodiments, the number of acquisition units is plural;
each acquisition unit further comprises: an output enabler connected to an output of the first gate;
the device still includes: a logic controller connected between each special value judger and the special value memory and connected to each output enabler; the logic controller is used for sending the updating signals to the special value memory, sequentially inputting the updating signals to the special value memory when receiving the updating signals at the same time, and controlling the output enablers of the acquisition units corresponding to the updating signals to be sequentially conducted.
The apparatus of the embodiment of the present disclosure may implement the above method for acquiring data of the weight matrix through the above hardware structure, and is not described in detail here.
When there are multiple acquiring units, if two structures belonging to the acquiring units are involved in interaction, it shall mean that two structures inside the same acquiring unit interact without special description.
The structures in the above configurations may be multiple independent physical devices, or multiple structures may be integrated together to be implemented by one physical device, and are not described in detail here.
In a fourth aspect, referring to fig. 9, an embodiment of the present disclosure provides an electronic device, including:
one or more processors;
a memory having one or more computer programs stored thereon:
one or more I/O interfaces connected between the processor and the memory and configured to realize information interaction between the processor and the memory;
the one or more computer programs, when executed by the one or more processors, enable:
a method of storing data of any of the above weight matrices;
and/or the presence of a gas in the gas,
any one of the above methods for obtaining data of the weight matrix.
Wherein, the processor is a device with data processing capability, which includes but is not limited to a Central Processing Unit (CPU) and the like; memory is a device with data storage capabilities including, but not limited to, random access memory (RAM, more specifically SDRAM, DDR, etc.), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), FLASH memory (FLASH); the I/O interface (read/write interface) is connected between the processor and the memory, and can realize information interaction between the memory and the processor, including but not limited to a data Bus (Bus) and the like.
The electronic device of the embodiment of the disclosure can implement the method, so that the electronic device can be used for brain-like computing technology.
For example, the above electronic device may be integrated in a brain-like computing device such as a brain-like chip (neuromorphic chip). Alternatively, the above electronic device may also provide the weight value for the brain-like computing device independently of the brain-like computing device.
One of ordinary skill in the art will appreciate that all or some of the steps, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof.
In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation.
Some or all of the physical components may be implemented as software executed by a processor, such as a Central Processing Unit (CPU), digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, random access memory (RAM, more specifically SDRAM, DDR, etc.), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), FLASH memory (FLASH), or other disk storage; compact disk read only memory (CD-ROM), Digital Versatile Disk (DVD), or other optical disk storage; magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage; any other medium which can be used to store the desired information and which can be accessed by the computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.
The present disclosure has disclosed example embodiments and, although specific terms are employed, they are used and should be interpreted in a generic and descriptive sense only and not for purposes of limitation. In some instances, features, characteristics and/or elements described in connection with a particular embodiment may be used alone or in combination with features, characteristics and/or elements described in connection with other embodiments, unless expressly stated otherwise, as would be apparent to one skilled in the art. Accordingly, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the disclosure as set forth in the appended claims.
Claims (10)
1. A method of data storage of a weight matrix, comprising:
generating a connection matrix, a data type table and a special value table according to the weight matrix;
storing the connection matrix, the data type table and the special value table;
the connection matrix has the same structure as the weight matrix, and each element of the connection matrix is the connection type of the weight value of the corresponding element in the weight matrix; the data volume of a connection type is less than that of a weighted value, and the connection type is divided into an effective connection and an ineffective connection;
the data type table comprises data types which are sequentially arranged and correspond to the weight values which are effectively connected in the weight matrix; the data volume of one data type is less than that of one weight value, the data type comprises a special value type and a plurality of preset value types, each preset value type corresponds to one effectively connected preset weight value, the special value type corresponds to the special weight value, and the other effectively connected weight values except the preset weight value are the special weight values;
the special value table comprises all special weight values in the weight matrix which are sequentially arranged.
2. The method of claim 1, wherein before the generating a connection matrix, a data type table, and a special value table according to a weight matrix, further comprising:
and determining a plurality of effectively connected weight values as preset weight values according to the weight matrix, and storing the preset weight values.
3. The method of claim 1, wherein,
the amount of data per said connection type is one bit.
4. A method of data acquisition of a weight matrix, comprising:
determining the weight values of corresponding elements in the weight matrix according to a pre-stored connection matrix, a data type table and a special value table;
the connection matrix has the same structure as the weight matrix, and each element of the connection matrix is the connection type of the weight value of the corresponding element in the weight matrix; the data volume of a connection type is less than that of a weighted value, and the connection type is divided into an effective connection and an ineffective connection;
the data type table comprises data types which are sequentially arranged and correspond to the weight values which are effectively connected in the weight matrix; the data volume of one data type is less than that of one weight value, the data type comprises a special value type and a plurality of preset value types, each preset value type corresponds to one effectively connected preset weight value, the special value type corresponds to the special weight value, and the other effectively connected weight values except the preset weight value are the special weight values;
the special value table comprises all special weight values in the weight matrix which are sequentially arranged.
5. The method according to claim 4, wherein the determining the weight values of the corresponding elements in the weight matrix according to the pre-stored connection matrix, the data type table and the special value table comprises:
the address generator sequentially generates addresses of all elements in the connection matrix and inputs the addresses to the connection matrix memory;
the connection matrix memory inputs the connection type of the corresponding element of the connection matrix prestored in the connection matrix memory into the first gate and the data type memory according to the received address;
when the data type memory receives the effective connection, inputting the corresponding data type of the data type table prestored in the data type memory into a second gate and a special value judger;
the preset value memory synchronously inputs all preset weight values prestored in the preset value memory to the second gate;
the special value judger outputs an updating signal to the special value memory when the received data type is the special value type;
the special value memory inputs the current special weight value of the special value table prestored in the special value memory into the second gate, and updates the current special weight value to the next special weight value of the special value table when an updating signal is received;
the second gate selects one of the received current special weight value and each preset weight value to output according to the type of the received data;
the invalid value memory inputs an invalid weight value prestored in the invalid value memory into the first gate;
the first gate selects one output from the received invalid weight value and the output of the second gate according to the received connection type.
6. The method according to claim 4, wherein after determining the weight values of the corresponding elements in the weight matrix according to the pre-stored connection matrix, the data type table, and the special value table, the method further comprises:
and acquiring the determined weight value.
7. The method according to claim 4, wherein after determining the weight values of the corresponding elements in the weight matrix according to the pre-stored connection matrix, the data type table, and the special value table, the method further comprises:
restoring at least partial region of the weight matrix according to the determined weight value, and storing the restored region of the weight matrix into a temporary storage space;
and acquiring the weight value of the weight matrix from the temporary storage space.
8. A device for acquiring data of a weight matrix comprises an address generator, a special value memory, an invalid value memory, a preset value memory and at least one acquisition unit; wherein the content of the first and second substances,
the address generator is used for sequentially generating addresses of all elements in the connection matrix and inputting the addresses to the connection matrix memory of the corresponding acquisition unit;
the preset value memory is used for storing all preset weight values and synchronously inputting all the preset weight values to the second gates of all the acquisition units;
the invalid value memory is used for storing invalid weight values and inputting the invalid weight values to the first gates of all the acquisition units;
the special value memory is used for storing a special value table, inputting the current special weight value of the special value table to the second gate, and updating the current special weight value to the next special weight value of the special value table when an updating signal is received;
each of the acquisition units includes:
the connection matrix memory is used for storing the connection matrix and inputting the connection type of the corresponding element of the connection matrix to the first gate and the data type memory according to the received address;
the data type memory is used for inputting the corresponding data types of the data type table prestored in the data type memory into the second gate and the special value judger when the effective connection is received;
a special value judger for outputting an update signal to the special value storage when the received data type is a special value type;
the second gate is used for selecting one of the received current special weight value and each preset weight value to output according to the type of the received data;
a first gate for selecting one output from the received invalid weight value and the output of the second gate according to the received connection type;
the connection matrix has the same structure as the weight matrix, and each element of the connection matrix is the connection type of the weight value of the corresponding element in the weight matrix; the data volume of a connection type is less than that of a weighted value, and the connection type is divided into an effective connection and an ineffective connection;
the data type table comprises data types which are sequentially arranged and correspond to the weight values which are effectively connected in the weight matrix; the data volume of one data type is less than that of one weight value, the data type comprises a special value type and a plurality of preset value types, each preset value type corresponds to one effectively connected preset weight value, the special value type corresponds to the special weight value, and the other effectively connected weight values except the preset weight value are the special weight values;
the special value table comprises all special weight values in the weight matrix which are sequentially arranged.
9. The apparatus of claim 8, wherein the number of the acquisition units is plural;
each of the acquisition units further includes: an output enabler connected to an output of the first gate;
the device further comprises: a logic controller connected between each special value judger and the special value memory and connected to each output enabler; the logic controller is used for sending the updating signals to the special value memory, sequentially inputting the updating signals to the special value memory when receiving the updating signals at the same time, and controlling the output enablers of the acquisition units corresponding to the updating signals to be sequentially conducted.
10. An electronic device, comprising:
one or more processors;
a memory having one or more computer programs stored thereon:
one or more I/O interfaces connected between the processor and the memory and configured to realize information interaction between the processor and the memory;
the one or more computer programs, when executed by the one or more processors, enable:
a method of data storage of a weight matrix as claimed in any one of claims 1 to 3;
and/or the presence of a gas in the atmosphere,
a method of data acquisition of a weight matrix as claimed in any one of claims 4 to 7.
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110585872.9A CN114527930B (en) | 2021-05-27 | 2021-05-27 | Weight matrix data storage method, data acquisition method and device and electronic equipment |
| PCT/CN2022/095238 WO2022247908A1 (en) | 2021-05-27 | 2022-05-26 | Data storage method and apparatus for weight matrix, data acquisition method and apparatus for weight matrix, and device |
| US18/266,658 US20240046113A1 (en) | 2021-05-27 | 2022-05-26 | Data storage method, data acquisition method, data acquisition apparatus for a weight matrix, and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110585872.9A CN114527930B (en) | 2021-05-27 | 2021-05-27 | Weight matrix data storage method, data acquisition method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114527930A true CN114527930A (en) | 2022-05-24 |
| CN114527930B CN114527930B (en) | 2024-01-30 |
Family
ID=81619672
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110585872.9A Active CN114527930B (en) | 2021-05-27 | 2021-05-27 | Weight matrix data storage method, data acquisition method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114527930B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022247908A1 (en) * | 2021-05-27 | 2022-12-01 | 北京灵汐科技有限公司 | Data storage method and apparatus for weight matrix, data acquisition method and apparatus for weight matrix, and device |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150242484A1 (en) * | 2014-02-27 | 2015-08-27 | Sas Institute Inc. | Sparse Matrix Storage in a Database |
| US20170293659A1 (en) * | 2016-04-12 | 2017-10-12 | Hsilin Huang | Method, System and Program Product for Mask-Based Compression of a Sparse Matrix |
| CN107977704A (en) * | 2017-11-10 | 2018-05-01 | 中国科学院计算技术研究所 | Weighted data storage method and the neural network processor based on this method |
| CN110443359A (en) * | 2019-07-03 | 2019-11-12 | 中国石油大学(华东) | Neural network compression algorithm based on adaptive combined beta pruning-quantization |
| CN111010883A (en) * | 2018-08-06 | 2020-04-14 | 华为技术有限公司 | Matrix processing method and device and logic circuit |
| CN111008698A (en) * | 2019-11-23 | 2020-04-14 | 复旦大学 | Sparse matrix multiplication accelerator for hybrid compressed recurrent neural networks |
| CN111240744A (en) * | 2020-01-03 | 2020-06-05 | 支付宝(杭州)信息技术有限公司 | Method and system for improving parallel computing efficiency related to sparse matrix |
| CN111626415A (en) * | 2019-02-27 | 2020-09-04 | 辉达公司 | Efficient matrix data format suitable for artificial neural networks |
| CN112668689A (en) * | 2019-10-16 | 2021-04-16 | 三星电子株式会社 | Method and apparatus for multimedia data processing |
-
2021
- 2021-05-27 CN CN202110585872.9A patent/CN114527930B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150242484A1 (en) * | 2014-02-27 | 2015-08-27 | Sas Institute Inc. | Sparse Matrix Storage in a Database |
| US20170293659A1 (en) * | 2016-04-12 | 2017-10-12 | Hsilin Huang | Method, System and Program Product for Mask-Based Compression of a Sparse Matrix |
| CN109416702A (en) * | 2016-04-12 | 2019-03-01 | 黄锡霖 | Method, system and the program product of sparse matrix compression based on mask |
| CN107977704A (en) * | 2017-11-10 | 2018-05-01 | 中国科学院计算技术研究所 | Weighted data storage method and the neural network processor based on this method |
| CN111010883A (en) * | 2018-08-06 | 2020-04-14 | 华为技术有限公司 | Matrix processing method and device and logic circuit |
| CN111626415A (en) * | 2019-02-27 | 2020-09-04 | 辉达公司 | Efficient matrix data format suitable for artificial neural networks |
| CN110443359A (en) * | 2019-07-03 | 2019-11-12 | 中国石油大学(华东) | Neural network compression algorithm based on adaptive combined beta pruning-quantization |
| CN112668689A (en) * | 2019-10-16 | 2021-04-16 | 三星电子株式会社 | Method and apparatus for multimedia data processing |
| CN111008698A (en) * | 2019-11-23 | 2020-04-14 | 复旦大学 | Sparse matrix multiplication accelerator for hybrid compressed recurrent neural networks |
| CN111240744A (en) * | 2020-01-03 | 2020-06-05 | 支付宝(杭州)信息技术有限公司 | Method and system for improving parallel computing efficiency related to sparse matrix |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022247908A1 (en) * | 2021-05-27 | 2022-12-01 | 北京灵汐科技有限公司 | Data storage method and apparatus for weight matrix, data acquisition method and apparatus for weight matrix, and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114527930B (en) | 2024-01-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102081799B1 (en) | Apparatuses and methods for storing and writing multiple parameter codes for memory operating parameters | |
| US20190325305A1 (en) | Machine learning inference engine scalability | |
| US10372338B2 (en) | Memory controller and data processing circuit with improved system efficiency | |
| US5572692A (en) | Memory configuration decoding system having automatic row base address generation mechanism for variable memory devices with row access interleaving | |
| US20230205453A1 (en) | Memory systems including examples of calculating hamming distances for neural network and data center applications | |
| US11609853B2 (en) | Memory controllers including examples of calculating hamming distances for neural network and data center applications | |
| US11636285B2 (en) | Memory including examples of calculating hamming distances for neural network and data center applications | |
| CN114527930A (en) | Weight matrix data storage method, data acquisition method and device and electronic equipment | |
| CN109727187B (en) | Method and device for adjusting storage position of multiple region of interest data | |
| US11275632B2 (en) | Broadcast command and response | |
| CN116679887B (en) | Universal control module and method for NAND Flash | |
| US11782622B2 (en) | Memory apparatus embedded with computing function and operation method thereof | |
| US11367498B2 (en) | Multi-level memory hierarchy | |
| CN114546251A (en) | Weight matrix data storage method, data acquisition method and device and electronic equipment | |
| US20240046113A1 (en) | Data storage method, data acquisition method, data acquisition apparatus for a weight matrix, and device | |
| WO2005010890A1 (en) | Programmable chip select | |
| CN112396072B (en) | Image classification acceleration method and device based on ASIC (application specific integrated circuit) and VGG16 | |
| US20230315448A1 (en) | Flexible support for device emulation and bank swapping | |
| US8812810B2 (en) | Memory control apparatus | |
| US20240078036A1 (en) | Hybrid memory management systems and methods with in-storage processing and attribute data management | |
| WO2022022231A1 (en) | Writing method, reading method, processor chip, storage medium and electronic device | |
| CN114791786A (en) | Task mapping method, task control method, task processing method, processing core and electronic equipment | |
| US20200272585A1 (en) | Memory controller and memory system including the memory controller | |
| CN113721979A (en) | Register circuit capable of configuring bit width, method, decoding circuit and chip | |
| CN112580285A (en) | Embedded server subsystem and configuration method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |