CN110674931A - Weight importance-based full-connection neural network optimization method and device - Google Patents
Weight importance-based full-connection neural network optimization method and device Download PDFInfo
- Publication number
- CN110674931A CN110674931A CN201910935582.5A CN201910935582A CN110674931A CN 110674931 A CN110674931 A CN 110674931A CN 201910935582 A CN201910935582 A CN 201910935582A CN 110674931 A CN110674931 A CN 110674931A
- Authority
- CN
- China
- Prior art keywords
- neuron
- layer
- degree
- expression
- input
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Feedback Control In General (AREA)
Abstract
The invention discloses a method and a device for optimizing a fully-connected neural network, wherein the method comprises the following steps: acquiring structural data of a neural network to be optimized and an input sample data set thereof; calculating input and output expressions of each neuron in the neural network; calculating an expression of the influence degree of each neuron in the hidden layer on each neuron in the next layer; calculating a change degree expression of the influence degree caused by the input change of the previous layer based on the influence degree expression of the neuron on each neuron in the next layer; calculating an association degree expression of each neuron and each neuron in the next layer and associating degree values of the base sample data; and finally, for the smaller association degree value, carrying out approximate processing on the weight value between the corresponding neurons. The invention considers the influence of input change on the association degree while observing the association degree among the neurons, reduces the power consumption overhead and improves the reliability of the neural network.
Description
Technical Field
The invention relates to the technical field of neural network calculation, in particular to a weight importance-based fully-connected neural network optimization method and device.
Background
Artificial neural networks have raised a tremendous research boon in academia since the 21 st century. With the gradual improvement of the deep learning theory, the artificial neural network is continuously developed, and the good performance of the artificial neural network is shown in many fields. Artificial neural networks are also used extensively in outer space exploration in the near future.
At present, a large number of large neural networks occupy a large amount of memory space during work, and bring large power consumption overhead. Therefore, more and more people are concerned about how to optimize the neural network, and reduce the memory occupation ratio and the power consumption overhead.
Common neural network optimization methods are implemented based on approximate calculations. The influence degree of the input of the neurons or the weight between the neurons on the output is approximated to a numerical value which is easy to calculate, and after the values are sorted, approximation processing is performed on the neurons or the weight corresponding to a smaller value. But generally, once the inputs to the neural network change slightly, the ranked list of the influence of the neurons or weights on the outputs is disturbed. Therefore, when the influence degrees are sequenced, the input of the neural network is comprehensively considered, and the working precision of the optimized neural network is guaranteed. In addition, when the neural network is applied to severe radiation environments such as outer space, the working accuracy of the neural network is reduced due to the high-energy particle impact. Therefore, how to improve the reliability as much as possible while reducing the memory overhead of the neural network is also an urgent problem to be solved at the present stage.
Disclosure of Invention
The invention conception of the invention is as follows: in view of the fact that a part of the information in a neural network, including input data of neurons and weight data between neurons, has a small effect on the output. Therefore, when the importance of the information in the neural network is considered, the influence of the input on the information should be comprehensively considered, the importance of the information in the neural network is ranked, and then the relatively unimportant information in the ranking table is approximately stored, so that the memory occupation ratio and the power consumption overhead are reduced.
The invention aims to provide a weight importance-based fully-connected neural network optimization method and device, which can guarantee the reliability and the working precision of a neural network on the basis of reducing the memory occupation ratio and the power consumption overhead of the neural network.
The technical scheme adopted by the invention is as follows: a fully-connected neural network optimization method, comprising:
acquiring trained neural network structural data to be optimized and an input sample data set thereof;
calculating to obtain input and output expressions of each neuron in the neural network by taking the input of the input layer as a variable;
for each hidden layer, respectively calculating an influence degree expression of each neuron in the current hidden layer on each neuron in the next layer;
calculating a change degree expression of the influence degree caused by the input change of the previous layer of the current hidden layer based on the influence degree expression of each neuron on each neuron in the next layer;
calculating the association degree expression of each neuron in each hidden layer and each neuron in the next layer based on the influence degree expression and the change degree expression thereof;
calculating the association degree value of each neuron in each hidden layer and each neuron in the next layer based on each sample data in the input sample data set and the association degree expression;
and for the smaller association degree value, carrying out approximate processing on the weight value between the corresponding current hidden layer neuron and the next layer neuron.
The invention realizes the weight importance judgment between the neurons of the front layer and the back layer by calculating the association degree value, and the weight value between the neurons with smaller association degree value is approximately processed, so that the memory occupation ratio and the power consumption expense can be effectively reduced.
Optionally, the method of the present invention further comprises: and for the larger association degree value, reinforcing the weight value between the corresponding current hidden layer neuron and the next layer neuron. The existing triple-modular redundancy reinforcement mode can be adopted, and the reliability of the neural network can be further improved through reinforcement treatment.
Optionally, the neural network structure to be optimized includes 1 input layer, 1 output layer, and at least 1 hidden layer, each layer including a plurality of neurons respectively; the neural network structure data to be optimized comprises the number of neurons contained in each layer of the neural network, the activation function of each neuron and the weight value between neurons in adjacent layers.
Optionally, the expression for calculating the influence degree of any neuron in the current hidden layer on each neuron in the next layer is as follows: and calculating partial derivatives of the input of each neuron in the later layer to the input of the current neuron.
Optionally, for any neuron in the current hidden layer, the degree-of-change expression for calculating the degree of influence caused by the change of the previous layer input is as follows:
and calculating an expression of the influence degree of the current neuron on each neuron in the next layer, and calculating the total partial derivative input to all neurons in the previous layer of the current hidden layer.
Optionally, the expression for calculating the association degree between each neuron in each hidden layer and each neuron in the next layer is as follows: and dividing the calculated influence degree expression corresponding to each neuron in the later layer by the corresponding change degree expression respectively.
Optionally, for any neuron in each hidden layer and any neuron in the next layer, respectively substituting each group of sample data in the sample data set into the association degree expression corresponding to the two neurons, and calculating to obtain an association degree value; and adding the correlation degree values corresponding to each group of sample data, and dividing the added correlation degree values by the sample capacity of the input sample data set to obtain a result serving as the correlation degree value between two corresponding neurons.
Optionally, for any neuron in any hidden layer, sorting the calculated association degree values of the neuron and each neuron in the next layer in size, and then performing approximate processing on weight values between the current neuron and the next layer neuron corresponding to m smaller association degree values preset in the sorting. Similarly, the weight values between the neurons in the next layer and the current neuron corresponding to n larger preset association degree values in the sorting can be reinforced. Of course, the sorting operation may be performed from small to large, in which case, n with a larger association degree value are followed, and m with a smaller association degree value are followed. The number of m and n can be adjusted according to the total number of neurons in the later layer.
The selection of the larger association degree value or the smaller association degree value can be selected by sorting and presetting the selection number, and also can be selected by presetting an association degree value threshold.
Optionally, the approximating process is to perform bit width reduction on the weight value.
The invention also provides a fully-connected neural network optimization device, which comprises:
the data input module is used for acquiring trained neural network structural data to be optimized and an input sample data set thereof;
the input expression calculation module is used for calculating to obtain input and output expressions of each neuron in the neural network by taking the input of the input layer as a variable;
the influence degree expression calculation module is used for calculating the influence degree expression of each neuron in the current hidden layer on each neuron in the next layer for each hidden layer;
the change degree expression calculation module is used for calculating a change degree expression of the influence degree caused by the input change of the previous layer of the current hidden layer based on the influence degree expression of each neuron on each neuron in the next layer;
the association degree expression calculation module is used for calculating the association degree expression of each neuron in each hidden layer and each neuron in the next layer based on the influence degree expression and the change degree expression thereof;
the correlation degree value calculation module is used for calculating the correlation degree value of each neuron in each hidden layer and each neuron in the next layer based on each sample data in the input sample data set and the correlation degree expression;
and the weight value processing module is used for carrying out approximate processing on the weight value between the corresponding current hidden layer neuron and the next layer neuron according to the smaller association degree value.
Advantageous effects
When the relevance degree between each neuron is inspected, the influence of the input change of the neural network on the relevance degree is considered at the same time, the weight values between the neurons with lower relevance degree are approximately stored, and the memory occupation ratio and the power consumption overhead are reduced; and the data with higher correlation degree is reinforced, so that the reliability of the neural network is further improved, namely the purpose of optimizing the neural network is achieved.
Drawings
FIG. 1 is a schematic structural diagram of a neural network to be optimized according to an embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating the association of neuron B2 with the anteroposterior layers in the neural network of FIG. 1;
FIG. 3 is a schematic diagram of the input and output of a neuron in the neural network of FIG. 1;
FIG. 4 is a flowchart illustrating a method according to an embodiment of the present invention.
Detailed Description
The following further description is made in conjunction with the accompanying drawings and the specific embodiments.
Example 1
A fully-connected neural network optimization method, comprising:
acquiring trained neural network structural data to be optimized and an input sample data set thereof;
calculating to obtain input and output expressions of each neuron in the neural network by taking the input of the input layer as a variable;
for each hidden layer, respectively calculating an influence degree expression of each neuron in the current hidden layer on each neuron in the next layer;
calculating a change degree expression of the influence degree caused by the input change of the previous layer of the current hidden layer based on the influence degree expression of each neuron on each neuron in the next layer;
calculating the association degree expression of each neuron in each hidden layer and each neuron in the next layer based on the influence degree expression and the change degree expression thereof;
calculating the association degree value of each neuron in each hidden layer and each neuron in the next layer based on each sample data in the input sample data set and the association degree expression;
and for the smaller association degree value, carrying out approximate processing on the weight value between the corresponding current hidden layer neuron and the next layer neuron.
In the embodiment, the weight importance judgment between the neurons in the front layer and the neurons in the rear layer is realized by calculating the association degree value, and the weight value between the neurons with the smaller association degree value is approximately processed, so that the memory occupation ratio and the power consumption overhead can be effectively reduced, and the reliability of the neural network is improved.
Examples 1 to 1
As shown in fig. 4, based on embodiment 1, the method of this embodiment further includes: and for the larger association degree value, reinforcing the weight value between the corresponding current hidden layer neuron and the next layer neuron. The existing triple-modular redundancy reinforcement mode can be adopted, and the reliability of the neural network can be further improved through reinforcement treatment.
In this embodiment, referring to fig. 1, the neural network to be optimized includes one Input layer (Input), three Hidden layers (Hidden _ a, Hidden _ B, and Hidden _ C), and one Output layer (Output), and each open circle represents one neuron. The directional connection lines between layers represent the weight and direction of data transfer. The neural network structure data to be optimized comprises the number of neurons contained in each layer of the neural network, the activation function of each neuron and the weight value between neurons in adjacent layers. It is assumed here that the weight values on the connecting lines after neural network training are respectively:
ωIArepresenting the weight value of each input of the input layer to each neuron in the Hidden layer Hidden _ A, wherein omegaijRepresenting the weight of the jth neuron in the input-layer inputs INi through Hidden _ A, e.g. ω12Weights for the 2 nd neuron IN IN1 through Hidden _ A.
ωABRepresenting the weight values of each neuron in Hidden layer Hidden _ A to each neuron in Hidden _ B, the elements in the matrix being interpreted with reference to ωIAFor the matrix elements.
Taking the neuron B2 in the Hidden layer Hidden _ B layer in fig. 1 as an example, when optimizing the neural network, the method specifically includes the following steps:
and S1, acquiring the neural network information and the input sample data set during the training of the neural network.
S2, referring to fig. 2 and 3, taking inputs IN1, IN2, IN3 of the input layer as variables, calculating the input expressions of each neuron IN each hidden layer and each neuron IN the output layer IN the neural network as follows:
the ith neuron in Hidden layer Hidden _ a:
output of the neuron: y isAi＝F(XAi)，F(XAi) I.e. a transfer function of a neuron Ai, where b0Constant, representing the bias of the Hidden _ a layer neurons;
the ith neuron in Hidden layer Hidden _ B:output of the neuron: y isBi＝F(XBi) Where B1 is a constant, representing a neuron in layer Hidden _ BBiasing;
the ith neuron in Hidden layer Hidden _ C:
output of the neuron: y isCi＝F(XCi) Wherein b is2Constant, representing the bias of the Hidden _ C layer neurons;
the ith neuron in Output layer Output:
output of the neuron: y isOi＝F(XOi) Wherein b is3Is a constant, representing the bias of Output layer neurons.
S3, calculating the influence degree expression of the neurons in the hidden layer on each neuron in the next layer as follows: calculating partial derivatives of the input of each neuron in the later layer to the input of the current neuron, namely:
degree of Effect of neuron B2 on Hidden _ C layer neuron Ci
The expression of (a) is:
s4, for any neuron in the current hidden layer, calculating a degree of change expression of the degree of influence caused by the change of the previous layer input, wherein the degree of change expression is as follows: and calculating an expression of the influence degree of the current neuron on each neuron in the next layer, and calculating the total partial derivative input to all neurons in the previous layer of the current hidden layer. Namely:
if there is a slight change in Hidden _ A layer input, B2 affects Ci to some extent
Degree of change of
The expression of (a) is:
s5, calculating the relevance degree expression of each neuron in each hidden layer and each neuron in the next layer as follows: and dividing the calculated influence degree expression corresponding to each neuron in the later layer by the corresponding change degree expression respectively. Namely:
will be provided withIs divided by
The degree of association of B2 with Ci is obtained:
s6, substituting each group of sample data in the sample data set into the association degree expression respectively, and calculating to obtain an association degree value; and then adding the correlation degree values corresponding to each group of sample data, and dividing the sum by the sample capacity of the input sample data set to obtain a result serving as the correlation degree value between two corresponding neurons. Namely:
respectively substituting input sample data sets
And obtaining the correlation degree value of B2 and Ci under each input sample, and accumulating and dividing the correlation degree values by the sample capacity to obtain a correlation degree value ranking table of B2 and each neuron in the Hidden _ C layer.
S7, for any neuron in any hidden layer, sorting the calculated association degree values of the neuron and each neuron in the next layer in size, and then performing approximate processing on the weight values between the neuron in the next layer and the current neuron, which correspond to m smaller association degree values preset in the sorting; and reinforcing the weight values between the neurons in the next layer corresponding to the preset n larger association degree values in the sorting and the current neurons. Of course, the sorting operation may be performed from small to large, in which case, n with a larger association degree value are followed, and m with a smaller association degree value are followed. The number of m and n can be adjusted according to the total number of neurons in the later layer.
The method can be specifically implemented as follows: and selecting the Hidden _ C layer neuron corresponding to the smaller value in the sorting order, and carrying out bit width reduction on the neuron and the weight value between the neuron and the B2. Selecting the Hidden _ C layer neurons corresponding to the larger values in the sorting table, and performing triple-modular redundancy reinforcement on the weight values between the selected Hidden _ C layer neurons and B2.
The calculation and the approximate operation are carried out on all the neurons of each hidden layer, so that the memory occupation ratio and the power consumption overhead of the neural network can be effectively reduced, and the reliability of the neural network is improved.
Example 2
The present embodiment is a fully-connected neural network optimization apparatus based on the same inventive concept as embodiment 1, including:
the data input module is used for acquiring trained neural network structural data to be optimized and an input sample data set thereof;
the input expression calculation module is used for calculating to obtain input and output expressions of each neuron in the neural network by taking the input of the input layer as a variable;
the influence degree expression calculation module is used for calculating the influence degree expression of each neuron in the current hidden layer on each neuron in the next layer for each hidden layer;
the change degree expression calculation module is used for calculating a change degree expression of the influence degree caused by the input change of the previous layer of the current hidden layer based on the influence degree expression of each neuron on each neuron in the next layer;
the association degree expression calculation module is used for calculating the association degree expression of each neuron in each hidden layer and each neuron in the next layer based on the influence degree expression and the change degree expression thereof;
the correlation degree value calculation module is used for calculating the correlation degree value of each neuron in each hidden layer and each neuron in the next layer based on each sample data in the input sample data set and the correlation degree expression;
and the weight value processing module is used for carrying out approximate processing on the weight value between the corresponding current hidden layer neuron and the next layer neuron according to the smaller association degree value.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (10)
1. A full-connection neural network optimization method is characterized by comprising the following steps:
acquiring trained neural network structural data to be optimized and an input sample data set thereof;
calculating to obtain input and output expressions of each neuron in the neural network by taking the input of the input layer as a variable;
for each hidden layer, respectively calculating an influence degree expression of each neuron in the current hidden layer on each neuron in the next layer;
calculating a change degree expression of the influence degree caused by the input change of the previous layer of the current hidden layer based on the influence degree expression of each neuron on each neuron in the next layer;
calculating the association degree expression of each neuron in each hidden layer and each neuron in the next layer based on the influence degree expression and the change degree expression thereof;
calculating the association degree value of each neuron in each hidden layer and each neuron in the next layer based on each sample data in the input sample data set and the association degree expression;
and for the smaller association degree value, carrying out approximate processing on the weight value between the corresponding current hidden layer neuron and the next layer neuron.
2. The method of claim 1, further comprising: and for the larger association degree value, reinforcing the weight value between the corresponding current hidden layer neuron and the next layer neuron.
3. The method of claim 1, wherein the neural network structure to be optimized comprises 1 input layer, 1 output layer and at least 1 hidden layer, each layer comprising a plurality of neurons; the neural network structure data to be optimized comprises the number of neurons contained in each layer of the neural network, the activation function of each neuron and the weight value between neurons in adjacent layers.
4. The method of claim 1, wherein the expression for calculating the influence degree of any neuron in the current hidden layer on each neuron in the next layer is: and calculating partial derivatives of the input of each neuron in the later layer to the input of the current neuron.
5. The method of claim 1, wherein the degree of change expression for calculating the degree of influence caused by the change of the previous layer input for any neuron in the current hidden layer is:
and calculating an expression of the influence degree of the current neuron on each neuron in the next layer, and calculating the total partial derivative input to all neurons in the previous layer of the current hidden layer.
6. The method of claim 1, wherein the expression of the degree of association between each neuron in each hidden layer and each neuron in the next layer is calculated as: and dividing the calculated influence degree expression corresponding to each neuron in the later layer by the corresponding change degree expression respectively.
7. The method of claim 1, wherein for any neuron in each hidden layer and any neuron in the next layer, each group of sample data in the sample data set is respectively substituted into the association degree expressions corresponding to the two neurons, and the association degree value is calculated; and adding the correlation degree values corresponding to each group of sample data, and dividing the added correlation degree values by the sample capacity of the input sample data set to obtain a result serving as the correlation degree value between two corresponding neurons.
8. The method as claimed in claim 1, wherein for any neuron in any hidden layer, the calculated association degree values of the neuron and the neurons in the next layer are sorted according to their sizes, and then the weighted values between the neurons in the next layer and the current neurons corresponding to m smaller association degree values preset in the sorting are approximated.
9. The method of claim 1, wherein said approximating is bit width reduction of weight values.
10. A fully-connected neural network optimization device, comprising:
the data input module is used for acquiring trained neural network structural data to be optimized and an input sample data set thereof;
the input expression calculation module is used for calculating to obtain input and output expressions of each neuron in the neural network by taking the input of the input layer as a variable;
the influence degree expression calculation module is used for calculating the influence degree expression of each neuron in the current hidden layer on each neuron in the next layer for each hidden layer;
the change degree expression calculation module is used for calculating a change degree expression of the influence degree caused by the input change of the previous layer of the current hidden layer based on the influence degree expression of each neuron on each neuron in the next layer;
the association degree expression calculation module is used for calculating the association degree expression of each neuron in each hidden layer and each neuron in the next layer based on the influence degree expression and the change degree expression thereof;
the correlation degree value calculation module is used for calculating the correlation degree value of each neuron in each hidden layer and each neuron in the next layer based on each sample data in the input sample data set and the correlation degree expression;
and the weight value processing module is used for carrying out approximate processing on the weight value between the corresponding current hidden layer neuron and the next layer neuron according to the smaller association degree value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910935582.5A CN110674931B (en) | 2019-09-29 | 2019-09-29 | Weight importance-based full-connection neural network optimization method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910935582.5A CN110674931B (en) | 2019-09-29 | 2019-09-29 | Weight importance-based full-connection neural network optimization method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110674931A true CN110674931A (en) | 2020-01-10 |
| CN110674931B CN110674931B (en) | 2022-07-15 |
Family
ID=69080216
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910935582.5A Active CN110674931B (en) | 2019-09-29 | 2019-09-29 | Weight importance-based full-connection neural network optimization method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110674931B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115294411A (en) * | 2022-10-08 | 2022-11-04 | 国网浙江省电力有限公司 | Power grid power transmission and transformation image data processing method based on neural network |
| WO2023208243A1 (en) * | 2022-04-29 | 2023-11-02 | 北京灵汐科技有限公司 | Weight storage method, apparatus and system, weight transmission method, apparatus and system, weight calculation method, apparatus and system, and device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5596681A (en) * | 1993-10-22 | 1997-01-21 | Nippondenso Co., Ltd. | Method of determining an optimal number of neurons contained in hidden layers of a neural network |
| CN106650918A (en) * | 2016-11-25 | 2017-05-10 | 东软集团股份有限公司 | System model building method and apparatus |
-
2019
- 2019-09-29 CN CN201910935582.5A patent/CN110674931B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5596681A (en) * | 1993-10-22 | 1997-01-21 | Nippondenso Co., Ltd. | Method of determining an optimal number of neurons contained in hidden layers of a neural network |
| CN106650918A (en) * | 2016-11-25 | 2017-05-10 | 东软集团股份有限公司 | System model building method and apparatus |
Non-Patent Citations (1)
| Title |
|---|
| SONG HAN ET AL.: "DEEP COMPRESSION: COMPRESSING DEEP NEURAL", 《ARXIV》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023208243A1 (en) * | 2022-04-29 | 2023-11-02 | 北京灵汐科技有限公司 | Weight storage method, apparatus and system, weight transmission method, apparatus and system, weight calculation method, apparatus and system, and device |
| CN115294411A (en) * | 2022-10-08 | 2022-11-04 | 国网浙江省电力有限公司 | Power grid power transmission and transformation image data processing method based on neural network |
| CN115294411B (en) * | 2022-10-08 | 2022-12-30 | 国网浙江省电力有限公司 | Power grid power transmission and transformation image data processing method based on neural network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110674931B (en) | 2022-07-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Chen et al. | An evolutionary multitasking-based feature selection method for high-dimensional classification | |
| CN105446979B (en) | Data digging method and node | |
| Venkataramanaiah et al. | Automatic compiler based FPGA accelerator for CNN training | |
| CN107657316A (en) | The cooperative system of general processor and neural network processor designs | |
| CN105929690B (en) | A kind of Flexible Workshop Robust Scheduling method based on decomposition multi-objective Evolutionary Algorithm | |
| CN110674931B (en) | Weight importance-based full-connection neural network optimization method and device | |
| CN105701120A (en) | Method and apparatus for determining semantic matching degree | |
| CN112288087A (en) | Neural network pruning method and device, electronic equipment and storage medium | |
| CN105224577B (en) | Multi-label text classification method and system | |
| Ettaouil et al. | Architecture optimization model for the multilayer perceptron and clustering. | |
| Gorjestani et al. | A hybrid COA-DEA method for solving multi-objective problems | |
| EP4053739A1 (en) | Neural network model for image segmentation and image segmentation method therefor | |
| CN115437795A (en) | Video memory recalculation optimization method and system for heterogeneous GPU cluster load perception | |
| CN109583659A (en) | User's operation behavior prediction method and system based on deep learning | |
| CN112906865A (en) | Neural network architecture searching method and device, electronic equipment and storage medium | |
| KR20210039921A (en) | Operation method of deep neural network system configured to optimize neural network model | |
| CN114519425A (en) | Convolution neural network acceleration system with expandable scale | |
| CN108364030B (en) | A kind of multi-categorizer model building method based on three layers of dynamic particles group's algorithm | |
| Wen et al. | MapReduce-based BP neural network classification of aquaculture water quality | |
| US20080052496A1 (en) | Method And Apparatus For Priority Based Data Processing | |
| CN109816103A (en) | A kind of PSO-BFGS neural network BP training algorithm | |
| CN117114250A (en) | Intelligent decision-making system based on large model | |
| CN112733301A (en) | Six-dimensional torque sensor gravity compensation method and system based on neural network | |
| Ibrokhimov et al. | Effective node selection technique towards sparse learning | |
| Park et al. | Training deep neural network in limited precision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |