CN108304930A - Network pruning method, apparatus and computer readable storage medium - Google Patents
Network pruning method, apparatus and computer readable storage medium Download PDFInfo
- Publication number
- CN108304930A CN108304930A CN201810116366.3A CN201810116366A CN108304930A CN 108304930 A CN108304930 A CN 108304930A CN 201810116366 A CN201810116366 A CN 201810116366A CN 108304930 A CN108304930 A CN 108304930A
- Authority
- CN
- China
- Prior art keywords
- value
- channel
- reserve channel
- network
- characteristic value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F18/2135—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Complex Calculations (AREA)
Abstract
The invention discloses a kind of network pruning methods, including：Obtain the output data of each layer of network；It is cut into row of channels according to the output data and Principal Component Analysis Algorithm, obtains reserve channel；According to the reserve channel re -training model, new model is obtained.The invention also discloses a kind of network pruning device, computer readable storage mediums.The present invention can obtain the principal component channel of each layer of network, and according to principal component channel re -training model, disposable to obtain the high prototype network of network performance.
Description
Technical field
The present invention relates to deep learning field more particularly to a kind of network pruning method, apparatus and computer-readable storages
Medium.
Background technology
The concept of deep learning is derived from the research of artificial neural network.Multilayer perceptron containing more hidden layers is exactly a kind of depth
Learning structure.Deep learning forms more abstract high-rise expression attribute classification or feature by combining low-level feature, to find
The distributed nature of data indicates.Depth model is usually associated with high memory space requirements while obtaining superior function
And computation complexity, and existing universal computing platform (such as CPU or GPU) is difficult to realize the neural computing of high energy efficiency.
In order to meet calculation power and efficiency demand of the deep neural network under different application scenarios (such as high in the clouds and terminal), need calculating
Method level carries out model compression with the methods of quantization, beta pruning.It is generally direct to network by setup parameter threshold mode at present
Network is cut, this mode is unable to ensure the network performance quality after cutting, and needs repeatedly to be cut or trained
Iteration can just obtain suitable network.
The above is only used to facilitate the understanding of the technical scheme, and is not represented and is recognized that the above is existing skill
Art.
Invention content
The main purpose of the present invention is to provide a kind of network pruning method, apparatus and computer readable storage medium, purports
Solving, the network performance that the prior art is unable to ensure after cutting is fine or not, needs the iteration for repeatedly being cut or being trained that could obtain
The problem of to suitable network.
To achieve the above object, the present invention provides a kind of subject network method of cutting out, and the network pruning method includes：
Obtain the output data of each layer of network；
It is cut into row of channels according to the output data and Principal Component Analysis Algorithm, obtains reserve channel；
According to the reserve channel re -training model, new model is obtained.
Preferably, described to be cut into row of channels according to the output data and Principal Component Analysis Algorithm, obtain reserve channel
The step of include：
The covariance matrix of data set is determined according to the output data；
Feature decomposition is carried out to the covariance matrix, obtains the diagonal matrix of characteristic value；
Reserve channel is determined according to the diagonal matrix and preset cumulative energy arithmetic.
Preferably, described the step of determining reserve channel according to the diagonal matrix and preset cumulative energy arithmetic, includes：
Characteristic value in the diagonal matrix is arranged according to numerical values recited；
Added up one by one to characteristic value according to putting in order, and divided by characteristic value summation, it is cumulative to obtain different characteristic value
The corresponding cumulative energy value of quantity；
Reserve channel quantity is determined according to the cumulative energy value and preset energy threshold value；
Reserve channel is determined according to the convolution weight value parameter in the reserve channel quantity and channel.
Preferably, the step of characteristic value by the diagonal matrix is arranged according to numerical values recited include：
By the characteristic value in the diagonal matrix according to being ranked sequentially from big to small；
Described the step of determining reserve channel quantity according to the cumulative energy value and preset energy threshold value includes：
Determine that cumulative energy value is greater than or equal to the cumulative quantity of minimal eigenvalue of the preset energy threshold value, and will be described
Minimal eigenvalue adds up quantity as reserve channel quantity；
The step of convolution weight value parameter according to the reserve channel quantity and channel determines reserve channel include：
Channel is arranged according to convolution weight value parameter size；
Reserve channel is obtained according to the reserve channel quantity according to sequence from big to small.
Preferably, described that feature decomposition carried out to the covariance matrix, the step of obtaining the diagonal matrix of characteristic value it
After further include：
Judge whether the energy value of maximum eigenvalue in the diagonal matrix is greater than or equal to the preset energy threshold value；
When the energy value of the maximum eigenvalue is greater than or equal to the preset energy threshold value, with convolution weight value parameter
Maximum channel is as reserve channel.
To achieve the above object, the present invention also provides a kind of spot net Scissoring device, the spot net Scissoring device includes：
The spot net that memory, processor and being stored in can be run on the memory and on the processor cuts program, described
Spot net cuts when program is executed by the processor and realizes following steps：
The covariance matrix of data set is determined according to the output data；
Feature decomposition is carried out to the covariance matrix, obtains the diagonal matrix of characteristic value；
Reserve channel is determined according to the diagonal matrix and preset cumulative energy arithmetic.
Preferably, described to be cut into row of channels according to the output data and Principal Component Analysis Algorithm, obtain reserve channel
The step of include：
The covariance matrix of data set is determined according to the output data；
Feature decomposition is carried out to the covariance matrix, obtains the diagonal matrix of characteristic value；
Reserve channel is determined according to the diagonal matrix and preset cumulative energy arithmetic.
Preferably, described the step of determining reserve channel according to the diagonal matrix and preset cumulative energy arithmetic, includes：
Characteristic value in the diagonal matrix is arranged according to numerical values recited；
Added up one by one to characteristic value according to putting in order, and divided by characteristic value summation, it is cumulative to obtain different characteristic value
The corresponding cumulative energy value of quantity；
Reserve channel quantity is determined according to the cumulative energy value and preset energy threshold value；
Reserve channel is determined according to the convolution weight value parameter in the reserve channel quantity and channel.
Preferably, the step of characteristic value by the diagonal matrix is arranged according to numerical values recited include：
By the characteristic value in the diagonal matrix according to being ranked sequentially from big to small；
Described the step of determining reserve channel quantity according to the cumulative energy value and preset energy threshold value includes：
Determine that cumulative energy value is greater than or equal to the cumulative quantity of minimal eigenvalue of the preset energy threshold value, and will be described
Minimal eigenvalue adds up quantity as reserve channel quantity；
The step of convolution weight value parameter according to the reserve channel quantity and channel determines reserve channel include：
Channel is arranged according to convolution weight value parameter size；
Reserve channel is obtained according to the reserve channel quantity according to sequence from big to small.
Preferably, described that feature decomposition carried out to the covariance matrix, the step of obtaining the diagonal matrix of characteristic value it
After further include：
Judge whether the energy value of maximum eigenvalue in the diagonal matrix is greater than or equal to the preset energy threshold value；
When the energy value of the maximum eigenvalue is greater than or equal to the preset energy threshold value, with convolution weight value parameter
Maximum channel is as reserve channel.
In addition, to achieve the above object, it is described computer-readable the present invention also provides a kind of computer readable storage medium
Network pruning program is stored on storage medium, the network pruning program realizes network as described above when being executed by processor
The step of method of cutting out.
The present invention provides a kind of network pruning method, device and computer readable storage medium.In the method, net is obtained
The output data of each layer of network；It is cut into row of channels according to the output data and Principal Component Analysis Algorithm, obtains reserve channel；Root
According to the reserve channel re -training model, new model is obtained.By the above-mentioned means, being based on number using Principal Component Analysis Algorithm
It learns statistics to be cut into row of channels, the principal component channel of each layer of network can be obtained, and according to principal component channel re -training mould
Type, it is disposable to obtain the high prototype network of network performance.
Description of the drawings
Fig. 1 is the affiliated terminal structure schematic diagram of device for the hardware running environment that the embodiment of the present invention is related to；
Fig. 2 is the flow signal of inventive network method of cutting out first embodiment and network pruning device first embodiment
Figure；
Fig. 3 is the flow signal of inventive network method of cutting out second embodiment and network pruning device second embodiment
Figure；
Fig. 4 is the flow signal of inventive network method of cutting out 3rd embodiment and network pruning device 3rd embodiment
Figure；
Fig. 5 is the flow signal of inventive network method of cutting out fourth embodiment and network pruning device fourth embodiment
Figure；
Fig. 6 is the flow diagram of the 5th embodiment of inventive network method of cutting out.
Specific implementation mode
It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, it is not intended to limit the present invention.
Existing network pruning method generally directly cuts network on network by setup parameter threshold mode, this
Kind mode is unable to ensure the network performance quality after cutting, and needing the iteration for repeatedly being cut or being trained just can obtain suitably
Network.
In order to solve the above technical problem, the present invention provides a kind of network pruning methods, by the method, first obtaining
The output data of each layer of network is cut further according to the output data and Principal Component Analysis Algorithm into row of channels, is obtained to retain and be led to
Road obtains new model then according to the reserve channel re -training model.To disposably obtain the high mould of network performance
Type network.
As shown in Figure 1, the terminal structure schematic diagram for the hardware running environment that Fig. 1, which is the embodiment of the present invention, to be related to.
Terminal of the embodiment of the present invention can be PC, can also be smart mobile phone, tablet computer, E-book reader, MP3
(Moving Picture Experts Group Audio Layer III, dynamic image expert's compression standard audio level 3)
Player, MP4 (Moving Picture Experts Group Audio Layer IV, dynamic image expert's compression standard sound
Frequency level 4) the packaged type terminal device with display function such as player, pocket computer.
As shown in Figure 1, the terminal may include：Processor 1001, such as CPU, network interface 1004, user interface
1003, memory 1005, communication bus 1002.Wherein, communication bus 1002 is for realizing the connection communication between these components.
User interface 1003 may include display screen (Display), input unit such as keyboard (Keyboard), optional user interface
1003 can also include standard wireline interface and wireless interface.Network interface 1004 may include optionally that the wired of standard connects
Mouth, wireless interface (such as WI-FI interfaces).Memory 1005 can be high-speed RAM memory, can also be stable memory
(non-volatile memory), such as magnetic disk storage.Memory 1005 optionally can also be independently of aforementioned processor
1001 storage device.
Optionally, terminal can also include camera, RF (Radio Frequency, radio frequency) circuit, sensor, audio
Circuit, WiFi module etc..Wherein, sensor such as optical sensor, motion sensor and other sensors.Specifically, light
Sensor may include ambient light sensor and proximity sensor, wherein ambient light sensor can according to the light and shade of ambient light come
The brightness of display screen is adjusted, proximity sensor can close display screen and/or backlight when mobile terminal is moved in one's ear.As
One kind of motion sensor, gravity accelerometer can detect in all directions the size of (generally three axis) acceleration, quiet
Size and the direction that can detect that gravity when only, the application that can be used to identify mobile terminal posture are (such as horizontal/vertical screen switching, related
Game, magnetometer pose calibrating), Vibration identification correlation function (such as pedometer, tap) etc.；Certainly, mobile terminal can also match
The other sensors such as gyroscope, barometer, hygrometer, thermometer, infrared sensor are set, details are not described herein.
It will be understood by those skilled in the art that the restriction of the not structure paired terminal of terminal structure shown in Fig. 1, can wrap
It includes than illustrating more or fewer components, either combines certain components or different components arrangement.
As shown in Figure 1, as may include that operating system, network are logical in a kind of memory 1005 of computer storage media
Believe module, Subscriber Interface Module SIM and network pruning program.
In terminal shown in Fig. 1, network interface 1004 is mainly used for connecting background server, is carried out with background server
Data communicate；User interface 1003 is mainly used for connecting client (user terminal), with client into row data communication；And processor
1001 can be used for calling the network pruning program stored in memory 1005, and execute following operation：
Obtain the output data of each layer of network；
It is cut into row of channels according to the output data and Principal Component Analysis Algorithm, obtains reserve channel；
According to the reserve channel re -training model, new model is obtained
Further, processor 1001 can call the network pruning program stored in memory 1005, also execute following
Operation：
The covariance matrix of data set is determined according to the output data；
Feature decomposition is carried out to the covariance matrix, obtains the diagonal matrix of characteristic value；
Reserve channel is determined according to the diagonal matrix and preset cumulative energy arithmetic.
Further, processor 1001 can call the network pruning program stored in memory 1005, also execute following
Operation：
Characteristic value in the diagonal matrix is arranged according to numerical values recited；
Added up one by one to characteristic value according to putting in order, and divided by characteristic value summation, it is cumulative to obtain different characteristic value
The corresponding cumulative energy value of quantity；
Reserve channel quantity is determined according to the cumulative energy value and preset energy threshold value；
Reserve channel is determined according to the convolution weight value parameter in the reserve channel quantity and channel.
Further, processor 1001 can call the network pruning program stored in memory 1005, also execute following
Operation：
By the characteristic value in the diagonal matrix according to being ranked sequentially from big to small；
Added up one by one to characteristic value according to putting in order, and divided by characteristic value summation, it is cumulative to obtain different characteristic value
The corresponding cumulative energy value of quantity；
Determine that cumulative energy value is greater than or equal to the cumulative quantity of minimal eigenvalue of the preset energy threshold value, and will be described
Minimal eigenvalue adds up quantity as reserve channel quantity；
Channel is arranged according to convolution weight value parameter size；
Reserve channel is obtained according to the reserve channel quantity according to sequence from big to small.
Further, processor 1001 can call the network pruning program stored in memory 1005, also execute following
Operation：
Judge whether the energy value of maximum eigenvalue in the diagonal matrix is greater than or equal to the preset energy threshold value；
When the energy value of the maximum eigenvalue is greater than or equal to the preset energy threshold value, with convolution weight value parameter
Maximum channel is as reserve channel.
Based on above-mentioned hardware configuration, the embodiment of inventive network method of cutting out is proposed.
It is inventive network method of cutting out first embodiment flow diagram with reference to Fig. 2, Fig. 2.
Originally it practices in deep learning field.The concept of deep learning is derived from the research of artificial neural network.Containing how hidden
The multilayer perceptron of layer is exactly a kind of deep learning structure.Deep learning forms more abstract high level by combining low-level feature
Attribute classification or feature are indicated, to find that the distributed nature of data indicates.Depth model is past while obtaining superior function
Toward along with high memory space requirements and computation complexity, and existing universal computing platform (such as CPU or GPU) is difficult
Realize the neural computing of high energy efficiency.In order to meet deep neural network under different application scenarios (such as high in the clouds and terminal)
Calculation power and efficiency demand, need to carry out model compression with quantization, the methods of beta pruning in algorithm level.At present generally by setting
To determine parameter threshold mode directly to cut network on network, this mode is unable to ensure the network performance quality after cutting,
Need the iteration for repeatedly being cut or being trained that can just obtain suitable network.The present embodiment provides a kind of based on mathematical statistics
Network pruning method cuts channel based on Principal Component Analysis Algorithm, and according to reserve channel re -training model, can be with
It is disposable to obtain the network for meeting particular network performance requirement.The realization process of the present embodiment includes the following steps.
Step S10 obtains the output data of each layer of network；
Deep learning refers to solving image, the various problems such as text with various machine learning algorithms on multilayer neural network
Algorithm set.The present embodiment is illustrated by taking LeNet deep learning neural networks as an example.LeNet by convolutional layer, pond layer,
Full articulamentum composition.Wherein convolutional layer coordinates with pond layer, forms multiple convolution groups, feature is successively extracted, eventually by several
A full articulamentum completes classification.Convolutional layer is characterized extract layer, the input of each neuron and the local receptor field phase of preceding layer
Even, and the feature of the part is extracted, after the local feature is extracted, its position relationship between other features is also true therewith
It decides.In the present embodiment, by taking convolutional layer as an example, for specific data set, all samples of the data set are first sent into net
Network carries out convolution algorithm to sample data in convolutional layer and exports the data of convolution algorithm.Convolution process is equivalent to using one
Filter, i.e. convolution kernel, to filter each zonule of image, to obtain the convolution numerical value of these zonules.Specifically answering
In, often there are multiple convolution kernels, it is believed that each convolution kernel represents a kind of image model, a corresponding channel.If
Some image block and the value that this convolution nuclear convolution goes out are big, then it is assumed that this image block is in close proximity to this convolution kernel.Each sample number
An output data can be all generated according in this channel.If port number is d, sample number n, then it is d*n to export number.
Step S20 is cut according to the output data and Principal Component Analysis Algorithm into row of channels, obtains reserve channel；
Based on above-mentioned steps, for the output data of each layer of each network, according to Principal Component Analysis Algorithm to output data
It is handled, carries out cutting channel, obtain reserve channel.The cutting channel of the present embodiment is referred to one through statistical analysis
Subchannel is given up, and retains remaining channel (reserve channel i.e. in the present embodiment), according to remaining channel re -training
Model, to realize the cutting to network.Principal Component Analysis is a kind of statistical method of dimensionality reduction, it is by means of a positive alternation
Change, the relevant former random vector of its component be converted to the incoherent new random vector of its component, this shown as on algebraically by
The covariance matrix of former random vector is transformed into diagonal form battle array, is geometrically showing as the orthogonal coordinates of former coordinate system transformation Cheng Xin
System is allowed to be directed toward several orthogonal directions that sample point distribution is most opened, then carries out dimension-reduction treatment to multidimensional variable system, make it
It is converted into low-dimensional variable system with a higher precision, then by constructing cost function appropriate, further low-dimensional system
It is converted to unidimensional system.Its principle is to try primal variable being reassembled into one group of new several comprehensive change being independent of each other
Amount.Principal component analysis be try by it is original it is numerous there is certain correlation (such as P index), be reassembled into one group of newly mutual
Mutually unrelated overall target replaces original index.Processing usually mathematically is exactly that original P index is made linear combination,
As new overall target.The general process step of Principal Component Analysis Algorithm includes：1) average value of each column is sought respectively, it is then right
In all samples, corresponding average value is all subtracted；2) Eigen Covariance matrix is sought；3) ask covariance characteristic value and feature to
Amount；4) characteristic value is sorted according to sequence from big to small, selects maximum k, then by its corresponding k feature to
Amount is respectively as Column vector groups at eigenvectors matrix.Wherein, average value refers to the output number according to each data value of picture
According to the average value acquired.
In the present embodiment, when being cut into row of channels according to Principal Component Analysis Algorithm, first by the output data of each layer of network
It subtracts and corresponding is averagely worth to the two-dimensional matrix that mean value zero setting matrix Y, Y are d*n, wherein d is port number, and n is sample number.
Mean value zero setting matrix Y is multiplied again to obtain covariance matrix G (wherein G=YY with its transposed matrixT), then G is subjected to feature point
Solution can obtain the diagonal matrix S of the characteristic value of covariance matrix.Then by the characteristic value in diagonal matrix according to size order
Arranged, according to sequence from big to small to characteristic value carry out one by one add up and divided by characteristic value summation, obtain different numbers
The cumulative cumlative energy of measure feature value.Specifically, assume characteristic value in diagonal matrix S according to from big to small be ranked sequentially for
S1, S2……Sd, characteristic value summation is Ssum, then can (S1+S2)/SsumAs first cumulative energy value, by (S1+S2+S3)/
SsumAs second cumulative energy value, and so on.Then by cumulative energy value and preset energy threshold value (such as 95%) ratio
Compared with, the determining minimum cumulative energy value more than preset energy threshold value, and according to the number of the minimum cumulative energy value character pair value
According to reserve channel data are determined, by the absolute value of the weight in each channel and as the channel convolution weight value parameter, so
Afterwards by channel according to the big minispread of convolution weight value parameter, according to reserve channel number since convolution weight value parameter maximum value
Amount obtains reserve channel.Certainly, other than carrying out adding up one by one to characteristic value according to sequence from big to small and obtaining cumlative energy,
The present embodiment can also determine cumlative energy in the way of from small to large, then with preset energy threshold value (such as 5%), determine
Less than the cumulative maximum energy of preset energy threshold value, and according to the data and characteristic value of the minimum cumulative energy value character pair value
Sum determines reserve channel quantity.In the present embodiment, it other than calculating cumlative energy threshold value and determining reserve channel, can also incite somebody to action
Maximum eigenvalue divided by characteristic value summation, determine the energy of this feature value, and it is default to judge whether the energy of maximum eigenvalue is more than
Reserve channel quantity is determined as 1 by energy threshold when the energy of maximum eigenvalue is more than preset energy threshold value, and by convolution
The maximum channel of convolution weight value parameter is determined as reserve channel.
Step S30 obtains new model according to the reserve channel re -training model.
Based on above-mentioned steps, in the present embodiment, after determining reserve channel, data acquisition system is inputted into reserve channel group again
At network, re-start training, determine the new parameter of model, obtain new model.
In the present embodiment, the output data of each layer of network is obtained；According to the output data and Principal Component Analysis Algorithm
It is cut into row of channels, obtains reserve channel；According to the reserve channel re -training model, new model is obtained.By above-mentioned
Mode is cut based on mathematical statistics into row of channels using Principal Component Analysis Algorithm, and the principal component that can obtain each layer of network is logical
Road, and according to principal component channel re -training model, it is disposable to obtain the high prototype network of network performance.
Further, it is inventive network method of cutting out second embodiment flow diagram with reference to Fig. 3, Fig. 3, based on above-mentioned
The network pruning method that the present invention prompts proposes the second embodiment of the present invention.
In the present embodiment, step S20 includes：
Step S40 determines the covariance matrix of data set according to the output data；
Step S50 carries out feature decomposition to the covariance matrix, obtains the diagonal matrix of characteristic value；
Step S60 determines reserve channel according to the diagonal matrix and preset cumulative energy arithmetic.
Based on above-described embodiment, in the present embodiment, the output data of each layer of network is first subtracted into corresponding average value and is obtained
It is the two-dimensional matrix of d*n to mean value zero setting matrix Y, Y, wherein d is port number, and n is sample number.Again by mean value zero setting matrix Y with
Its transposed matrix is multiplied to obtain covariance matrix G (wherein G=YYT), then G progress feature decomposition can be obtained into covariance matrix
Characteristic value diagonal matrix S.When carrying out feature decomposition to G, following form, G=USU are resolved intoT, wherein U is the feature of G
Vector matrix is orthogonal matrix, and S is the eigenvalue matrix of U, is diagonal matrix.Then by the characteristic value in diagonal matrix according to
Size order is arranged, according to sequence from big to small to characteristic value carry out one by one add up and divided by characteristic value summation, obtain
The cumlative energy to add up to Different quantitative specificity value.Specifically, assuming that characteristic value in diagonal matrix S is suitable according to from big to small
Sequence is arranged as S1, S2……Sd, characteristic value summation is Ssum, then can (S1+S2)/SsumAs first cumulative energy value, by (S1
+S2+S3)/SsumAs second cumulative energy value, and so on.Then by cumulative energy value and preset energy threshold value (such as
95%) compare, determine the minimum cumulative energy value more than preset energy threshold value, and according to the minimum cumulative energy value character pair
The data of value determine reserve channel data, and the absolute value of the weight in each channel and as the channel convolution weighted value is joined
Number, then by channel according to the big minispread of convolution weight value parameter, according to reservation since convolution weight value parameter maximum value
Number of channels obtains reserve channel.Certainly, in addition to being accumulated one by one to characteristic value progress according to sequence from big to small
Outside energy, the present embodiment can also determine cumlative energy in the way of from small to large, then with preset energy threshold value (such as
5%) the cumulative maximum energy less than preset energy threshold value, is determined, and according to the number of the cumulative maximum energy value character pair value
Determine reserve channel quantity according to characteristic value sum, specifically, according to cumulative maximum energy determine it is corresponding give up number of channels,
Further according to total number of channels amount and gives up number of channels and determine reserve channel quantity.It is of course also possible to do not put in order to characteristic value,
It directly calculates the cumlative energy of all Different quantitative specificity value combinations and carries out comparing determining reserve channel with preset energy threshold value.
After determining reserve channel in the manner described above, again by the network of data acquisition system input reserve channel composition, instruction is re-started
Practice, determines the new parameter of model, obtain new model.
In the present embodiment, the covariance matrix of data set is determined according to the output data；To the covariance matrix
Feature decomposition is carried out, the diagonal matrix of characteristic value is obtained；It is determined and is retained according to the diagonal matrix and preset cumulative energy arithmetic
Channel.Reserve channel is determined according to preset cumulative energy arithmetic and characteristic value by the above-mentioned means, realizing, it is ensured that re -training
Network meets specific performance index requirements.
Further, the network pruning method based on aforementioned present invention prompt proposes that the third of the present invention is real with reference to Fig. 4
Apply example.
Based on embodiment shown in above-mentioned, in the present embodiment, step S60 includes：
Step S70 arranges the characteristic value in the diagonal matrix according to numerical values recited；
Step S80 adds up one by one to characteristic value according to putting in order, and divided by characteristic value summation, obtain different spies
The corresponding cumulative energy value of the cumulative quantity of value indicative；
Step S90 determines reserve channel quantity according to the cumulative energy value and preset energy threshold value；
Step S100 determines reserve channel according to the convolution weight value parameter in the reserve channel quantity and channel.
Based on above-described embodiment, in the present embodiment, after the diagonal matrix for obtaining characteristic value, by the spy in diagonal matrix
Value indicative is arranged according to size order, to characteristic value add up one by one according to sequence from big to small and divided by characteristic value
Summation obtains the cumulative cumlative energy of Different quantitative specificity value.Specifically, assuming characteristic value in diagonal matrix S according to from big
It is ranked sequentially to small as S1, S2……Sd, characteristic value summation is Ssum, then can (S1+S2)/SsumAs first cumlative energy
Value, by (S1+S2+S3)/SsumAs second cumulative energy value, and so on.Then by cumulative energy value and preset energy threshold
Value (such as 95%) compares, and determines the minimum cumulative energy value more than preset energy threshold value, and according to the minimum cumulative energy value
The data of character pair value determine reserve channel data, by the absolute value of the weight in each channel and as the channel convolution
Weight value parameter, then by channel according to the big minispread of convolution weight value parameter, since convolution weight value parameter maximum value
Reserve channel is obtained according to reserve channel quantity.In the present embodiment, can calculate every time after a cumulative energy value with it is default
Energy threshold is compared, and is carried out again with preset energy threshold value after the cumlative energy of whole quantative attribute values can also have been calculated into
Compare.Certainly, other than carrying out adding up one by one to characteristic value according to sequence from big to small and obtaining cumlative energy, the present embodiment is also
Cumlative energy can be determined in the way of from small to large, then with preset energy threshold value (such as 5%), determined and be less than default energy
The cumulative maximum energy of threshold value is measured, and is determined and is protected according to the data of the cumulative maximum energy value character pair value and characteristic value sum
Stay number of channels, specifically, according to cumulative maximum energy determine it is corresponding give up number of channels, further according to total number of channels amount and house
It abandons number of channels and determines reserve channel quantity.After determining reserve channel in the manner described above, data acquisition system is inputted again and is protected
The network for staying channel to form, re-starts training, determines the new parameter of model, obtain new model.
In the present embodiment, the characteristic value in the diagonal matrix is arranged according to numerical values recited；It is suitable according to arranging
Ordered pair characteristic value is added up one by one, and divided by characteristic value summation, obtain different characteristic value and add up quantity corresponding cumlative energy
Value；Reserve channel quantity is determined according to the cumulative energy value and preset energy threshold value；According to the reserve channel quantity and lead to
The convolution weight value parameter in road determines reserve channel.By the above-mentioned means, by being arranged according to numerical values recited characteristic value,
And add up one by one and determine reserve channel with the mode accurate quick of preset energy threshold value, reduce operand.
Further, with reference to Fig. 5, the fourth embodiment of inventive network method of cutting out is proposed.In the present embodiment, step
S70 includes：
Step S110, by the characteristic value in the diagonal matrix according to being ranked sequentially from big to small；
Step S90 includes：
Step S120 determines that cumulative energy value is greater than or equal to the minimal eigenvalue cumulative number of the preset energy threshold value
Amount, and the minimal eigenvalue is added up into quantity as reserve channel quantity；
Step S100 includes：
Step S130 arranges in channel according to convolution weight value parameter size；
Step S140 obtains reserve channel according to sequence from big to small according to the reserve channel quantity.
Based on above-described embodiment, in the present embodiment, after the diagonal matrix for obtaining characteristic value, by the spy in diagonal matrix
Value indicative is arranged according to sequence from big to small, and to characteristic value carry out one by one add up and divided by characteristic value summation, obtain
The cumulative cumlative energy of Different quantitative specificity value.Specifically, the characteristic value in hypothesis diagonal matrix S is according to sequence from big to small
It is arranged as S1, S2……Sd, characteristic value summation is Ssum, then can (S1+S2)/SsumAs first cumulative energy value, by (S1+
S2+S3)/SsumAs second cumulative energy value, and so on.Then by cumulative energy value and preset energy threshold value (such as
95%) compare, determine the minimum cumulative energy value more than preset energy threshold value, and according to the minimum cumulative energy value character pair
The data of value determine reserve channel data, and the absolute value of the weight in each channel and as the channel convolution weighted value is joined
Number, then by channel according to the big minispread of convolution weight value parameter, according to reservation since convolution weight value parameter maximum value
Number of channels obtains reserve channel.After determining reserve channel in the manner described above, data acquisition system is inputted into reserve channel again
The network of composition, re-starts training, determines the new parameter of model, obtains new model.
In the present embodiment, by the characteristic value in the diagonal matrix according to being ranked sequentially from big to small；Determine accumulation
The minimal eigenvalue that energy value is greater than or equal to the preset energy threshold value adds up quantity, and by the minimal eigenvalue cumulative number
Amount is used as reserve channel quantity；Channel is arranged according to convolution weight value parameter size；According to sequence root from big to small
Reserve channel is obtained according to the reserve channel quantity.By the above-mentioned means, by characteristic value according to numerical value according to from big to small
Sequence arranged, and add up one by one and determine reserve channel with the mode accurate quick of preset energy threshold value, reduce operation
Amount.
Further, with reference to Fig. 6, the 5th embodiment of inventive network method of cutting out is proposed.In the present embodiment, step
Further include after S50：
Step S150, judges whether the energy value of maximum eigenvalue in the diagonal matrix is greater than or equal to the default energy
Measure threshold value；
Step S160, when the energy value of the maximum eigenvalue is greater than or equal to the preset energy threshold value, with convolution
The maximum channel of weight value parameter is as reserve channel.
In the present embodiment based on above-described embodiment, after the diagonal matrix for obtaining characteristic value, maximum spy can be first determined
Then the summation of maximum eigenvalue divided by characteristic value is calculated the energy value of maximum eigenvalue, specifically, when maximum feature by value indicative
Value is S1When, the energy value of maximum eigenvalue is S1/Ssum, judge whether the energy value of maximum eigenvalue is more than preset energy threshold
Value determines that reserve channel quantity is 1 when the energy value of maximum eigenvalue is more than preset energy threshold value, maximum with convolution weight
Channel as reserve channel.To quickly determine reserve channel, operand is reduced, improves arithmetic speed.
Further, the present invention also provides a kind of network pruning devices.
With reference to figure 2, the first embodiment of inventive network Scissoring device is proposed.In the present embodiment, the network pruning
Following steps may be implemented when being executed by the processor in device：
Step S10 obtains the output data of each layer of network；
Deep learning refers to solving image, the various problems such as text with various machine learning algorithms on multilayer neural network
Algorithm set.The present embodiment is illustrated by taking LeNet deep learning neural networks as an example.LeNet by convolutional layer, pond layer,
Full articulamentum composition.Wherein convolutional layer coordinates with pond layer, forms multiple convolution groups, feature is successively extracted, eventually by several
A full articulamentum completes classification.Convolutional layer is characterized extract layer, the input of each neuron and the local receptor field phase of preceding layer
Even, and the feature of the part is extracted, after the local feature is extracted, its position relationship between other features is also true therewith
It decides.In the present embodiment, by taking convolutional layer as an example, for specific data set, all samples of the data set are first sent into net
Network carries out convolution algorithm to sample data in convolutional layer and exports the data of convolution algorithm.Convolution process is equivalent to using one
Filter, i.e. convolution kernel, to filter each zonule of image, to obtain the convolution numerical value of these zonules.Specifically answering
In, often there are multiple convolution kernels, it is believed that each convolution kernel represents a kind of image model, a corresponding channel.If
Some image block and the value that this convolution nuclear convolution goes out are big, then it is assumed that this image block is in close proximity to this convolution kernel.Each sample number
An output data can be all generated according in this channel.If port number is d, sample number n, then it is d*n to export number.
Step S20 is cut according to the output data and Principal Component Analysis Algorithm into row of channels, obtains reserve channel；
Based on above-mentioned steps, for the output data of each layer of each network, according to Principal Component Analysis Algorithm to output data
It is handled, carries out cutting channel, obtain reserve channel.The cutting channel of the present embodiment is referred to one through statistical analysis
Subchannel is given up, and retains remaining channel (reserve channel i.e. in the present embodiment), according to remaining channel re -training
Model, to realize the cutting to network.Principal Component Analysis is a kind of statistical method of dimensionality reduction, it is by means of a positive alternation
Change, the relevant former random vector of its component be converted to the incoherent new random vector of its component, this shown as on algebraically by
The covariance matrix of former random vector is transformed into diagonal form battle array, is geometrically showing as the orthogonal coordinates of former coordinate system transformation Cheng Xin
System is allowed to be directed toward several orthogonal directions that sample point distribution is most opened, then carries out dimension-reduction treatment to multidimensional variable system, make it
It is converted into low-dimensional variable system with a higher precision, then by constructing cost function appropriate, further low-dimensional system
It is converted to unidimensional system.Its principle is to try primal variable being reassembled into one group of new several comprehensive change being independent of each other
Amount.Principal component analysis be try by it is original it is numerous there is certain correlation (such as P index), be reassembled into one group of newly mutual
Mutually unrelated overall target replaces original index.Processing usually mathematically is exactly that original P index is made linear combination,
As new overall target.The general process step of Principal Component Analysis Algorithm includes：1) average value of each column is sought respectively, it is then right
In all samples, corresponding average value is all subtracted；2) Eigen Covariance matrix is sought；3) ask covariance characteristic value and feature to
Amount；4) characteristic value is sorted according to sequence from big to small, selects maximum k, then by its corresponding k feature to
Amount is respectively as Column vector groups at eigenvectors matrix.Wherein, average value refers to the output number according to each data value of picture
According to the average value acquired.
In the present embodiment, when being cut into row of channels according to Principal Component Analysis Algorithm, first by the output data of each layer of network
It subtracts and corresponding is averagely worth to the two-dimensional matrix that mean value zero setting matrix Y, Y are d*n, wherein d is port number, and n is sample number.
Mean value zero setting matrix Y is multiplied again to obtain covariance matrix G (wherein G=YY with its transposed matrixT), then G is subjected to feature point
Solution can obtain the diagonal matrix S of the characteristic value of covariance matrix.Then by the characteristic value in diagonal matrix according to size order
Arranged, according to sequence from big to small to characteristic value carry out one by one add up and divided by characteristic value summation, obtain different numbers
The cumulative cumlative energy of measure feature value.Specifically, assume characteristic value in diagonal matrix S according to from big to small be ranked sequentially for
S1, S2……Sd, characteristic value summation is Ssum, then can (S1+S2)/SsumAs first cumulative energy value, by (S1+S2+S3)/
SsumAs second cumulative energy value, and so on.Then by cumulative energy value and preset energy threshold value (such as 95%) ratio
Compared with, the determining minimum cumulative energy value more than preset energy threshold value, and according to the number of the minimum cumulative energy value character pair value
According to reserve channel data are determined, by the absolute value of the weight in each channel and as the channel convolution weight value parameter, so
Afterwards by channel according to the big minispread of convolution weight value parameter, according to reserve channel number since convolution weight value parameter maximum value
Amount obtains reserve channel.Certainly, other than carrying out adding up one by one to characteristic value according to sequence from big to small and obtaining cumlative energy,
The present embodiment can also determine cumlative energy in the way of from small to large, then with preset energy threshold value (such as 5%), determine
Less than the cumulative maximum energy of preset energy threshold value, and according to the data and characteristic value of the minimum cumulative energy value character pair value
Sum determines reserve channel quantity.In the present embodiment, it other than calculating cumlative energy threshold value and determining reserve channel, can also incite somebody to action
Maximum eigenvalue divided by characteristic value summation, determine the energy of this feature value, and it is default to judge whether the energy of maximum eigenvalue is more than
Reserve channel quantity is determined as 1 by energy threshold when the energy of maximum eigenvalue is more than preset energy threshold value, and by convolution
The maximum channel of convolution weight value parameter is determined as reserve channel.
Step S30 obtains new model according to the reserve channel re -training model.
Based on above-mentioned steps, in the present embodiment, after determining reserve channel, data acquisition system is inputted into reserve channel group again
At network, re-start training, determine the new parameter of model, obtain new model.
In the present embodiment, the output data of each layer of network is obtained；According to the output data and Principal Component Analysis Algorithm
It is cut into row of channels, obtains reserve channel；According to the reserve channel re -training model, new model is obtained.By above-mentioned
Mode is cut based on mathematical statistics into row of channels using Principal Component Analysis Algorithm, and the principal component that can obtain each layer of network is logical
Road, and according to principal component channel re -training model, it is disposable to obtain the high prototype network of network performance.
Further, with reference to Fig. 3, the second embodiment of present invention point cloud network Scissoring device is proposed.In the present embodiment,
Following steps may be implemented when being executed by the processor in the network pruning program：
Step S40 determines the covariance matrix of data set according to the output data；
Step S50 carries out feature decomposition to the covariance matrix, obtains the diagonal matrix of characteristic value；
Step S60 determines reserve channel according to the diagonal matrix and preset cumulative energy arithmetic.
Based on above-described embodiment, in the present embodiment, the output data of each layer of network is first subtracted into corresponding average value and is obtained
It is the two-dimensional matrix of d*n to mean value zero setting matrix Y, Y, wherein d is port number, and n is sample number.Again by mean value zero setting matrix Y with
Its transposed matrix is multiplied to obtain covariance matrix G (wherein G=YYT), then G progress feature decomposition can be obtained into covariance matrix
Characteristic value diagonal matrix S.When carrying out feature decomposition to G, following form, G=USU are resolved intoT, wherein U is the feature of G
Vector matrix is orthogonal matrix, and S is the eigenvalue matrix of U, is diagonal matrix.Then by the characteristic value in diagonal matrix according to
Size order is arranged, according to sequence from big to small to characteristic value carry out one by one add up and divided by characteristic value summation, obtain
The cumlative energy to add up to Different quantitative specificity value.Specifically, assuming that characteristic value in diagonal matrix S is suitable according to from big to small
Sequence is arranged as S1, S2……Sd, characteristic value summation is Ssum, then can (S1+S2)/SsumAs first cumulative energy value, by (S1
+S2+S3)/SsumAs second cumulative energy value, and so on.Then by cumulative energy value and preset energy threshold value (such as
95%) compare, determine the minimum cumulative energy value more than preset energy threshold value, and according to the minimum cumulative energy value character pair
The data of value determine reserve channel data, and the absolute value of the weight in each channel and as the channel convolution weighted value is joined
Number, then by channel according to the big minispread of convolution weight value parameter, according to reservation since convolution weight value parameter maximum value
Number of channels obtains reserve channel.Certainly, in addition to being accumulated one by one to characteristic value progress according to sequence from big to small
Outside energy, the present embodiment can also determine cumlative energy in the way of from small to large, then with preset energy threshold value (such as
5%) the cumulative maximum energy less than preset energy threshold value, is determined, and according to the number of the cumulative maximum energy value character pair value
Determine reserve channel quantity according to characteristic value sum, specifically, according to cumulative maximum energy determine it is corresponding give up number of channels,
Further according to total number of channels amount and gives up number of channels and determine reserve channel quantity.It is of course also possible to do not put in order to characteristic value,
It directly calculates the cumlative energy of all Different quantitative specificity value combinations and carries out comparing determining reserve channel with preset energy threshold value.
After determining reserve channel in the manner described above, again by the network of data acquisition system input reserve channel composition, instruction is re-started
Practice, determines the new parameter of model, obtain new model.
In the present embodiment, the covariance matrix of data set is determined according to the output data；To the covariance matrix
Feature decomposition is carried out, the diagonal matrix of characteristic value is obtained；It is determined and is retained according to the diagonal matrix and preset cumulative energy arithmetic
Channel.Reserve channel is determined according to preset cumulative energy arithmetic and characteristic value by the above-mentioned means, realizing, it is ensured that re -training
Network meets specific performance index requirements.
Further, with reference to Fig. 4, the 3rd embodiment of present invention point cloud network Scissoring device is proposed.In the present embodiment,
Following steps may be implemented when being executed by the processor in the network pruning program：
Step S70 arranges the characteristic value in the diagonal matrix according to numerical values recited；
Step S80 adds up one by one to characteristic value according to putting in order, and divided by characteristic value summation, obtain different spies
The corresponding cumulative energy value of the cumulative quantity of value indicative；
Step S90 determines reserve channel quantity according to the cumulative energy value and preset energy threshold value；
Step S100 determines reserve channel according to the convolution weight value parameter in the reserve channel quantity and channel.
Based on above-described embodiment, in the present embodiment, after the diagonal matrix for obtaining characteristic value, by the spy in diagonal matrix
Value indicative is arranged according to size order, to characteristic value add up one by one according to sequence from big to small and divided by characteristic value
Summation obtains the cumulative cumlative energy of Different quantitative specificity value.Specifically, assuming characteristic value in diagonal matrix S according to from big
It is ranked sequentially to small as S1, S2……Sd, characteristic value summation is Ssum, then can (S1+S2)/SsumAs first cumlative energy
Value, by (S1+S2+S3)/SsumAs second cumulative energy value, and so on.Then by cumulative energy value and preset energy threshold
Value (such as 95%) compares, and determines the minimum cumulative energy value more than preset energy threshold value, and according to the minimum cumulative energy value
The data of character pair value determine reserve channel data, by the absolute value of the weight in each channel and as the channel convolution
Weight value parameter, then by channel according to the big minispread of convolution weight value parameter, since convolution weight value parameter maximum value
Reserve channel is obtained according to reserve channel quantity.In the present embodiment, can calculate every time after a cumulative energy value with it is default
Energy threshold is compared, and is carried out again with preset energy threshold value after the cumlative energy of whole quantative attribute values can also have been calculated into
Compare.Certainly, other than carrying out adding up one by one to characteristic value according to sequence from big to small and obtaining cumlative energy, the present embodiment is also
Cumlative energy can be determined in the way of from small to large, then with preset energy threshold value (such as 5%), determined and be less than default energy
The cumulative maximum energy of threshold value is measured, and is determined and is protected according to the data of the cumulative maximum energy value character pair value and characteristic value sum
Stay number of channels, specifically, according to cumulative maximum energy determine it is corresponding give up number of channels, further according to total number of channels amount and house
It abandons number of channels and determines reserve channel quantity.After determining reserve channel in the manner described above, data acquisition system is inputted again and is protected
The network for staying channel to form, re-starts training, determines the new parameter of model, obtain new model.
In the present embodiment, the characteristic value in the diagonal matrix is arranged according to numerical values recited；It is suitable according to arranging
Ordered pair characteristic value is added up one by one, and divided by characteristic value summation, obtain different characteristic value and add up quantity corresponding cumlative energy
Value；Reserve channel quantity is determined according to the cumulative energy value and preset energy threshold value；According to the reserve channel quantity and lead to
The convolution weight value parameter in road determines reserve channel.By the above-mentioned means, by being arranged according to numerical values recited characteristic value,
And add up one by one and determine reserve channel with the mode accurate quick of preset energy threshold value, reduce operand.
Further, with reference to Fig. 5, the fourth embodiment of present invention point cloud network Scissoring device is proposed.In the present embodiment, institute
It states when network pruning program is executed by the processor and following steps may be implemented：
Step S110, by the characteristic value in the diagonal matrix according to being ranked sequentially from big to small；
Step S80 adds up one by one to characteristic value according to putting in order, and divided by characteristic value summation, obtain different spies
The corresponding cumulative energy value of the cumulative quantity of value indicative；
Step S120 determines that cumulative energy value is greater than or equal to the minimal eigenvalue cumulative number of the preset energy threshold value
Amount, and the minimal eigenvalue is added up into quantity as reserve channel quantity；
Step S130 arranges in channel according to convolution weight value parameter size；
Step S140 obtains reserve channel according to sequence from big to small according to the reserve channel quantity.
Based on above-described embodiment, in the present embodiment, after the diagonal matrix for obtaining characteristic value, by the spy in diagonal matrix
Value indicative is arranged according to sequence from big to small, and to characteristic value carry out one by one add up and divided by characteristic value summation, obtain
The cumulative cumlative energy of Different quantitative specificity value.Specifically, the characteristic value in hypothesis diagonal matrix S is according to sequence from big to small
It is arranged as S1, S2……Sd, characteristic value summation is Ssum, then can (S1+S2)/SsumAs first cumulative energy value, by (S1+
S2+S3)/SsumAs second cumulative energy value, and so on.Then by cumulative energy value and preset energy threshold value (such as
95%) compare, determine the minimum cumulative energy value more than preset energy threshold value, and according to the minimum cumulative energy value character pair
The data of value determine reserve channel data, and the absolute value of the weight in each channel and as the channel convolution weighted value is joined
Number, then by channel according to the big minispread of convolution weight value parameter, according to reservation since convolution weight value parameter maximum value
Number of channels obtains reserve channel.After determining reserve channel in the manner described above, data acquisition system is inputted into reserve channel again
The network of composition, re-starts training, determines the new parameter of model, obtains new model.
In the present embodiment, by the characteristic value in the diagonal matrix according to being ranked sequentially from big to small；Determine accumulation
The minimal eigenvalue that energy value is greater than or equal to the preset energy threshold value adds up quantity, and by the minimal eigenvalue cumulative number
Amount is used as reserve channel quantity；Channel is arranged according to convolution weight value parameter size；According to sequence root from big to small
Reserve channel is obtained according to the reserve channel quantity.By the above-mentioned means, by characteristic value according to numerical value according to from big to small
Sequence arranged, and add up one by one and determine reserve channel with the mode accurate quick of preset energy threshold value, reduce operation
Amount.
In addition, the embodiment of the present invention also proposes a kind of computer readable storage medium.
Network pruning program is stored on computer readable storage medium of the present invention, the network pruning program is by processor
The step of network pruning method as described above is realized when execution.
Wherein, the network pruning program run on the processor is performed realized method and can refer to the present invention
The each embodiment of network pruning method method, details are not described herein again.
It should be noted that herein, the terms "include", "comprise" or its any other variant are intended to non-row
His property includes, so that process, method, article or system including a series of elements include not only those elements, and
And further include other elements that are not explicitly listed, or further include for this process, method, article or system institute it is intrinsic
Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including this
There is also other identical elements in the process of element, method, article or system.
The embodiments of the present invention are for illustration only, can not represent the quality of embodiment.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side
Method can add the mode of required general hardware platform to realize by software, naturally it is also possible to by hardware, but in many cases
The former is more preferably embodiment.Based on this understanding, technical scheme of the present invention substantially in other words does the prior art
Going out the part of contribution can be expressed in the form of software products, which is stored in one as described above
In storage medium (such as ROM/RAM, magnetic disc, CD), including some instructions use so that a station terminal equipment (can be mobile phone,
Computer, server, air conditioner or network equipment etc.) execute method described in each embodiment of the present invention.
It these are only the preferred embodiment of the present invention, be not intended to limit the scope of the invention, it is every to utilize this hair
Equivalent structure or equivalent flow shift made by bright specification and accompanying drawing content is applied directly or indirectly in other relevant skills
Art field, is included within the scope of the present invention.
Claims (10)
1. a kind of network pruning method, which is characterized in that the network pruning method includes：
Obtain the output data of each layer of network；
It is cut into row of channels according to the output data and Principal Component Analysis Algorithm, obtains reserve channel；
According to the reserve channel re -training model, new model is obtained.
2. network pruning method as described in claim 1, which is characterized in that described according to the output data and principal component point
Analysing the step of algorithm cuts into row of channels, obtains reserve channel includes：
The covariance matrix of data set is determined according to the output data；
Feature decomposition is carried out to the covariance matrix, obtains the diagonal matrix of characteristic value；
Reserve channel is determined according to the diagonal matrix and preset cumulative energy arithmetic.
3. network pruning method as claimed in claim 2, which is characterized in that described according to the diagonal matrix and preset cumulative
Energy arithmetic determines that the step of reserve channel includes：
Characteristic value in the diagonal matrix is arranged according to numerical values recited；
Added up one by one to characteristic value according to putting in order, and divided by characteristic value summation, obtain different characteristic value add up quantity
Corresponding cumulative energy value；
Reserve channel quantity is determined according to the cumulative energy value and preset energy threshold value；
Reserve channel is determined according to the convolution weight value parameter in the reserve channel quantity and channel.
4. network pruning method as claimed in claim 3, the characteristic value by the diagonal matrix is according to numerical values recited
The step of being arranged include：
By the characteristic value in the diagonal matrix according to being ranked sequentially from big to small；
Described the step of determining reserve channel quantity according to the cumulative energy value and preset energy threshold value includes：
Determine that cumulative energy value is greater than or equal to the minimal eigenvalue of the preset energy threshold value and adds up quantity, and by the minimum
Characteristic value adds up quantity as reserve channel quantity；
The step of convolution weight value parameter according to the reserve channel quantity and channel determines reserve channel include：
Channel is arranged according to convolution weight value parameter size；
Reserve channel is obtained according to the reserve channel quantity according to sequence from big to small.
5. network pruning method as claimed in claim 2, which is characterized in that described to carry out feature point to the covariance matrix
Further include after the step of solving, obtaining the diagonal matrix of characteristic value：
Judge whether the energy value of maximum eigenvalue in the diagonal matrix is greater than or equal to the preset energy threshold value；
It is maximum with convolution weight value parameter when the energy value of the maximum eigenvalue is greater than or equal to the preset energy threshold value
Channel as reserve channel.
6. a kind of network pruning device, which is characterized in that the network pruning device includes：It memory, processor and is stored in
On the memory and the network pruning program that can run on the processor, the network pruning program is by the processor
Following steps are realized when execution：
Obtain the output data of each layer of network；
It is cut into row of channels according to the output data and Principal Component Analysis Algorithm, obtains reserve channel；
According to the reserve channel re -training model, new model is obtained.
7. network pruning device as claimed in claim 6, which is characterized in that the network pruning program is held by the processor
Following steps are also realized when row：
The covariance matrix of data set is determined according to the output data；
Feature decomposition is carried out to the covariance matrix, obtains the diagonal matrix of characteristic value；
Reserve channel is determined according to the diagonal matrix and preset cumulative energy arithmetic.
8. network pruning device as claimed in claim 7, is characterized in that, the network pruning program is executed by the processor
When also realize following steps：
Characteristic value in the diagonal matrix is arranged according to numerical values recited；
Added up one by one to characteristic value according to putting in order, and divided by characteristic value summation, obtain different characteristic value add up quantity
Corresponding cumulative energy value；
Reserve channel quantity is determined according to the cumulative energy value and preset energy threshold value；
Reserve channel is determined according to the convolution weight value parameter in the reserve channel quantity and channel.
9. network pruning device as claimed in claim 8, is characterized in that, the network pruning program is executed by the processor
When also realize following steps：
By the characteristic value in the diagonal matrix according to being ranked sequentially from big to small；
Added up one by one to characteristic value according to putting in order, and divided by characteristic value summation, obtain different characteristic value add up quantity
Corresponding cumulative energy value；
Determine that cumulative energy value is greater than or equal to the minimal eigenvalue of the preset energy threshold value and adds up quantity, and by the minimum
Characteristic value adds up quantity as reserve channel quantity；
Channel is arranged according to convolution weight value parameter size；
Reserve channel is obtained according to the reserve channel quantity according to sequence from big to small.
10. a kind of computer readable storage medium, which is characterized in that be stored with network sanction on the computer readable storage medium
Program is cut, the network as described in any one of claim 1 to 5 is realized when the network pruning program is executed by the processor
The step of method of cutting out.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810116366.3A CN108304930A (en) | 2018-02-05 | 2018-02-05 | Network pruning method, apparatus and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810116366.3A CN108304930A (en) | 2018-02-05 | 2018-02-05 | Network pruning method, apparatus and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108304930A true CN108304930A (en) | 2018-07-20 |
Family
ID=62864156
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810116366.3A Pending CN108304930A (en) | 2018-02-05 | 2018-02-05 | Network pruning method, apparatus and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108304930A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109934300A (en) * | 2019-03-21 | 2019-06-25 | 腾讯科技（深圳）有限公司 | Model compression method, apparatus, computer equipment and storage medium |
| CN111695375A (en) * | 2019-03-13 | 2020-09-22 | 上海云从企业发展有限公司 | Face recognition model compression algorithm based on model distillation, medium and terminal |
-
2018
- 2018-02-05 CN CN201810116366.3A patent/CN108304930A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111695375A (en) * | 2019-03-13 | 2020-09-22 | 上海云从企业发展有限公司 | Face recognition model compression algorithm based on model distillation, medium and terminal |
| CN111695375B (en) * | 2019-03-13 | 2021-04-20 | 上海云从企业发展有限公司 | Face recognition model compression method based on model distillation, medium and terminal |
| CN109934300A (en) * | 2019-03-21 | 2019-06-25 | 腾讯科技（深圳）有限公司 | Model compression method, apparatus, computer equipment and storage medium |
| CN109934300B (en) * | 2019-03-21 | 2023-08-25 | 腾讯科技（深圳）有限公司 | Model compression method, device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11938403B2 (en) | Game character behavior control method and apparatus, storage medium, and electronic device | |
| CN110647921B (en) | User behavior prediction method, device, equipment and storage medium | |
| CN109360097A (en) | Prediction of Stock Index method, apparatus, equipment and storage medium based on deep learning | |
| CN109671020A (en) | Image processing method, device, electronic equipment and computer storage medium | |
| CN107730000A (en) | Coolant leakage detection method and device for air-conditioning | |
| CN107330750A (en) | A kind of recommended products figure method and device, electronic equipment | |
| CN111008640A (en) | Image recognition model training and image recognition method, device, terminal and medium | |
| JP2023523029A (en) | Image recognition model generation method, apparatus, computer equipment and storage medium | |
| CN112991502B (en) | Model training method, device, equipment and storage medium | |
| CN113469340A (en) | Model processing method, federal learning method and related equipment | |
| CN107280693A (en) | Psychoanalysis System and method based on VR interactive electronic sand tables | |
| CN107590799A (en) | The recognition methods of banana maturity period and device based on depth convolutional neural networks | |
| CN108492301A (en) | A kind of Scene Segmentation, terminal and storage medium | |
| CN107146160A (en) | Customer insured's analysis on the health status method and server | |
| CN113408570A (en) | Image category identification method and device based on model distillation, storage medium and terminal | |
| CN110210319A (en) | Computer equipment, tongue body photo constitution identification device and storage medium | |
| CN110111426A (en) | A kind of determination method and apparatus in sound separate pattern house | |
| CN108304930A (en) | Network pruning method, apparatus and computer readable storage medium | |
| CN111282281B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| Wang et al. | Distortion recognition for image quality assessment with convolutional neural network | |
| CN110879993A (en) | Neural network training method, and execution method and device of face recognition task | |
| CN112370790B (en) | Game map drawing method and device, electronic equipment and storage medium | |
| CN113230650A (en) | Data processing method and device and computer readable storage medium | |
| CN112905013A (en) | Intelligent agent control method and device, computer equipment and storage medium | |
| CN106845512A (en) | Beasts shape recognition method and system based on fractal parameter |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180720 |