CN110516806A - The rarefaction method and apparatus of neural network parameter matrix - Google Patents
The rarefaction method and apparatus of neural network parameter matrix Download PDFInfo
- Publication number
- CN110516806A CN110516806A CN201910814977.XA CN201910814977A CN110516806A CN 110516806 A CN110516806 A CN 110516806A CN 201910814977 A CN201910814977 A CN 201910814977A CN 110516806 A CN110516806 A CN 110516806A
- Authority
- CN
- China
- Prior art keywords
- parameter
- neural network
- network parameter
- redundancy
- multiple parameters
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Physics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computational Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Optimization (AREA)
- Mathematical Analysis (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Algebra (AREA)
- Databases & Information Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The present invention discloses the rarefaction method and apparatus of neural network parameter matrix, wherein, a kind of rarefaction method of neural network parameter matrix, comprising: before carrying out each round repetitive exercise to neural network parameter, select the multiple parameters of most redundancy in the neural network parameter matrix；The multiple parameters of the most redundancy are set 0；Parameter after opposed 0 is marked and parameter during repetitive exercise later no longer after update mark.The scheme that the present processes and device provide passes through increment type Sparse Least, the final rarefaction of network can reach 80% or more, memory space needed for parameter greatly reduces, promote calculating speed, and because being the Sparse Least of increment type, although each round sets 0 part weight, but non-zero part still can participate in training, so the performance of identification will not reduce.
Description
Technical field
The invention belongs to nerual network technique field more particularly to the rarefaction methods and dress of neural network parameter matrix
It sets.
Background technique
In the related technology, svd (Singular Value Decomposition, singular value decomposition) decomposition algorithm, node
Prune point trimming algorithm, traditional sparse matrix algorithm are several technologies relatively common at present.Wherein, svd decomposition algorithm:
The matrix of one m*n is svd and resolves into m*r+r*n (r < < n)；Node prune: according to certain a line of matrix or a certain column
Total weight as cost to crop certain row column；Traditional sparse matrix algorithm: L1 norm is done in training process about
Beam；The small element of certain weights is disposed by force according to weight size to network after training.
Inventor has found that existing scheme at least has the following deficiencies: during realizing the application
Svd is decomposed or Node prune algorithm is all a kind of trimming algorithm of rule, and what is cropped is all the whole of matrix
Capable or permutation, however every a line or each column have important element, being forced recognition performance after cutting can drop significantly
It is low.
It does not do and interferes in traditional sparse matrix training process, and the certain elements of Compulsory Removal after entire training,
Meeting is so that recognition performance is remarkably decreased.
Summary of the invention
The embodiment of the present invention provides a kind of rarefaction method and apparatus of neural network parameter matrix, at least solving
State one of technical problem.
In a first aspect, the embodiment of the present invention provides a kind of rarefaction method of neural network parameter matrix, comprising: to mind
Before carrying out each round repetitive exercise through network parameter, the multiple parameters of most redundancy in the neural network parameter matrix are selected；
The multiple parameters of the most redundancy are set 0；Parameter after opposed 0 be marked and during repetitive exercise later no longer more
Parameter after new label.
Second aspect, the embodiment of the present invention provide a kind of rarefaction device of neural network parameter matrix, comprising: redundancy ginseng
Number selecting module, is configured to before carrying out each round repetitive exercise to neural network parameter, selects the neural network parameter
The multiple parameters of most redundancy in matrix；0 module is set, is configured to the multiple parameters of the most redundancy setting 0；And mark module,
Parameter after being configured to opposed 0 is marked and parameter during repetitive exercise later no longer after update mark.
The third aspect provides a kind of electronic equipment comprising: at least one processor, and with described at least one
Manage the memory of device communication connection, wherein the memory is stored with the instruction that can be executed by least one described processor, institute
It states instruction to be executed by least one described processor, so that at least one described processor is able to carry out any embodiment of the present invention
Neural network parameter matrix rarefaction method the step of.
Fourth aspect, the embodiment of the present invention also provide a kind of computer program product, and the computer program product includes
The computer program being stored on non-volatile computer readable storage medium storing program for executing, the computer program include program instruction, when
When described program instruction is computer-executed, the computer is made to execute the neural network parameter matrix of any embodiment of the present invention
Rarefaction method the step of.
The scheme that the present processes and device provide is by using increment type Sparse Least, the final rarefaction of network
80% or more can be reached, greatly reduced parameter (the parameter amount that sparse matrix itself stores can be small), calculating speed is promoted
Although (sparse matrix calculating can accelerate), and because being the Sparse Least of increment type, each round set 0 part weight, but
It is non-zero part or can participates in training, so the performance of identification will not reduces.Further, the significantly reduction meaning of parameter
Identification speed significantly become faster, the consumption of memory is greatly lowered.From the perspective of Product Experience, recognition result is more
No-delay rapidly, customer experience effect is good.From the perspective of cost, the cheaper processor of more low side and interior can be used
It deposits, thus greatly reduces the cost of product.
Detailed description of the invention
In order to illustrate the technical solution of the embodiments of the present invention more clearly, required use in being described below to embodiment
Attached drawing be briefly described, it should be apparent that, drawings in the following description are some embodiments of the invention, for ability
For the those of ordinary skill of domain, without creative efforts, it can also be obtained according to these attached drawings other attached
Figure.
Fig. 1 is a kind of flow chart of the rarefaction method for neural network parameter matrix that one embodiment of the invention provides；
Fig. 2 is the flow chart of the rarefaction method for another neural network parameter matrix that one embodiment of the invention provides；
Fig. 3 is the flow chart of the rarefaction method for another neural network parameter matrix that one embodiment of the invention provides；
Fig. 4 is that one of the rarefaction method for a kind of neural network parameter matrix that one embodiment of the invention provides is specific real
Apply the flow diagram of example；
Fig. 5 is a kind of block diagram of the rarefaction device for neural network parameter matrix that one embodiment of the invention provides；
Fig. 6 is the structural schematic diagram for the electronic equipment that one embodiment of the invention provides.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art
Every other embodiment obtained without creative efforts, shall fall within the protection scope of the present invention.
Referring to FIG. 1, it illustrates the processes of one embodiment of rarefaction method of the neural network parameter matrix of the application
Figure, the rarefaction method of the neural network parameter matrix of the present embodiment can be adapted for various neural networks and various have mind
Terminal through network, as intelligent sound TV, intelligent sound box, Intelligent dialogue toy and other it is existing have neural network
Intelligent terminal etc..
As shown in Figure 1, in a step 101, before carrying out each round repetitive exercise to neural network parameter, selecting nerve
The multiple parameters of most redundancy in network paramter matrix；
In a step 102, the multiple parameters of most redundancy are set 0；
In step 103, the parameter after opposed 0 is marked and no longer update mark during repetitive exercise later
Parameter afterwards.
In the present embodiment, it for step 101, during neural metwork training, is carried out to neural network parameter
Before each round repetitive exercise, the rarefaction device of neural network parameter matrix selects most redundancy in neural network parameter matrix
Multiple parameters can be and judge which parameter most redundancy in neural network parameter matrix by some convenient and fast modes, for example, which
A little parameters are least worth, most not essential, can be defined as the multiple parameters of most redundancy.Later, for step 102, mind
The multiple parameters of the most redundancy are all reset to 0 by the rarefaction device through network paramter matrix.It is right finally, for step 103
The multiple parameters of the most redundancy are marked, and update it no longer during repetitive exercise later, thus iteration later
Trained calculation amount also can be smaller and smaller.
The method of the present embodiment is by before each round repetitive exercise, setting 0 part not too important parameter, Ke Yi great
Big reduces parameter amount (the parameter amount that sparse matrix itself stores can be small), and promoting calculating speed, (sparse matrix calculating can add
Speed).And because being the Sparse Least of increment type, although each round sets 0 part weight, but non-zero part still can join
With training, so identification performance will not reduce.
Further, significantly reducing for parameter means that the speed of identification significantly becomes faster, and the consumption of memory is significantly
It reduces.From the perspective of Product Experience, recognition result is more no-delay rapidly, and customer experience effect is good.From the angle of cost
From the point of view of, more low side cheaper processor and memory can be used, thus greatly reduce the cost of product.
With further reference to Fig. 2, it illustrates another embodiments of rarefaction method of the neural network parameter matrix of the application
Flow chart.The flow chart for the step of flow chart of the present embodiment is further limited primarily directed to step 101 in flow chart 1.
As shown in Fig. 2, in step 201, before carrying out each round repetitive exercise to neural network parameter, by nerve net
Non-zero parameter is arranged according to the sequence of order of magnitude in network parameter matrix；
In step 202, the multiple parameters that the smallest multiple parameters of absolute value are most redundancy are selected based on preset ratio.
In the present embodiment, it for step 201, during neural metwork training, is carried out to neural network parameter
Before each round repetitive exercise, the rarefaction device of neural network parameter matrix is by parameter non-zero in neural network parameter matrix
It is arranged according to the sequence of order of magnitude.Later, for step 202, the rarefaction device of neural network parameter matrix is based on should
The order of magnitude being arranged in order therefrom selects multiple ginsengs of the smallest multiple parameters of absolute value as most redundancy in proportion
Number.It is big according to absolute value for non-zero parameter when each repetitive exercise for example, share 400 parameters in the absolute value matrix
Small to be ranked up, then select wherein 8 parameters of 8 parameters as most redundancy, the application according to 2% ratio does not have herein
Limitation.
Then the method for the present embodiment is selected wherein in proportion by being ranked up to non-zero parameter according to order of magnitude
Partial parameters as the multiple parameters of most redundancy carry out it is subsequent set 0 operation, and the parameter after setting 0 will not carry out again it is subsequent
The update of parameter, can carry out parameter rarefaction by simple mode, and the parameter due to being set to 0 still can participate in it is subsequent
Training, therefore the recognition performance of neural network is unaffected after rarefaction.
With further reference to Fig. 3, it illustrates the another embodiments of rarefaction method of the neural network parameter matrix of the application
Flow chart.The process that the flow chart of the embodiment is further limited primarily directed to the additional step of the flow chart of Fig. 1 or Fig. 2
Figure.
As shown in figure 3, in step 301, before carrying out next round repetitive exercise to neural network parameter, never being marked
The multiple parameters of most redundancy in neural network parameter matrix are selected in the parameter of note；
In step 302, the multiple parameters of most redundancy are set 0；
In step 303, the parameter after opposed 0 is marked and no longer update mark during repetitive exercise later
Parameter afterwards.
In the present embodiment, for step 301, before carrying out next round repetitive exercise to neural network parameter, nerve
The rarefaction device of network paramter matrix select from parameter not labeled before for Current Situation of Neural Network parameter matrix and
The multiple parameters for saying most redundancy can be and be selected in the way of the absolute value sequence in previous embodiment, and the application exists
There is no limit for this.Later, for step 302, the rarefaction device of neural network parameter matrix is by the multiple parameters of the most redundancy
0 is set, for step 303, opposed 0 parameter is marked and ginseng during repetitive exercise later no longer after update mark
Number.
To the embodiment of the present application scheme by each repetitive exercise preposition 0 and mark some nuisance parameters, can be with
Amount of storage needed for greatly reducing neural network parameter matrix before training is completed or before neural network convergence, and energy
Calculation amount is greatly reduced, and since all parameters all can be always involved in training, institute will not reduce instruction in the above described manner
The discrimination for the neural network come is practised, user experience can also obtain great promotion.
In some alternative embodiments, before carrying out each round repetitive exercise to neural network parameter, nerve is selected
The multiple parameters of most redundancy include: and incite somebody to action before carrying out each round repetitive exercise to neural network parameter in network paramter matrix
The absolute value and preset threshold of parameter in neural network parameter matrix are compared；If the absolute value of multiple parameters is less than pre-
If threshold value, multiple parameters are determined as to the multiple parameters of most redundancy.Some parameter definitions also can be simply most superfluous by which
Then remaining multiple parameters carry out LS-SVM sparseness.
In some alternative embodiments, the above method further include: preset if the absolute value of certain parameters is more than or equal to
Certain parameters are determined as nonredundancy parameter by threshold value.Subsequent training and parameter still can be participated in for nonredundancy parameter
In update, the parameter of nonredundancy also can constantly generate new nuisance parameter during subsequent repetitive exercise, no longer superfluous herein
It states.
It is further alternative, to it is selected set 0 parameter be marked include: record set 0 after parameter in neural network
Index position in parameter matrix.To in neural network parameter matrix set 0 matrix can only storage index position,
Greatly save required memory space.
Below to some problems encountered in the implementation of the present invention by description inventor and to finally determination
One specific embodiment of scheme is illustrated, so that those skilled in the art more fully understand the scheme of the application.
Inventor has found that the defect of the prior art is mainly since the following contents causes during realizing the application
:
(1) regularization is cut, and the cutting of full line or permutation often falls many useful weight cuts, is caused final
Recognition performance substantially reduce.
(2) training process does not intervene, and causes e-learning less than sparse information.
If those skilled in the art will solve drawbacks described above, it will usually the method decomposed with svd.General rarefaction side
Method is dealt with after network training, few to intervene in the training process.
The scheme of the application proposes a kind of rarefaction device of neural network parameter matrix.What the embodiment of the present application proposed
Method is not only to intervene in training process, and is that each round goes to intervene different elements, is gradually incremented by, so that network is gradually dilute
Thinization, these have some new meanings.
A kind of method that the embodiment of the present application puts forward new sparse matrix, is called increment type Sparse Least, the algorithm
It will intervention training when training pattern.It is well known that the training of neural network will have many wheels that could restrain, each
Before wheel training, the part of the part most redundancy of matrix is first set 0 by we, and this part is not involved in parameter update, only to other
Nonredundancy parameter (i.e. parameter important in this wheel) updates.And so on, we find out most redundancy at each round
Partial parameters, until training terminates.This method is simple and effective, can reduce 80% or more parameter amount, and recognition performance is not
Have loss.
Below in conjunction with Fig. 4, the embodiment of the present application is described in detail.Specific algorithm steps:
For each round iterative process:
1. non-zero parameter arranges from small to large according to absolute value in matrix.
2. selecting element (the i.e. the smallest element of absolute value, because minimum is to network of front according to a certain percentage
It is minimum to calculate contribution, it is believed that be most redundancy), this Partial Elements is set 0.
3. this Partial Elements elected in pairs 2 carry out label (having recorded the index position of element in a matrix), quilt
Carry out after the element of label all will not undated parameter, i.e., for the training of subsequent each round, this part is 0 always.
4. 0 element in matrix is more and more after excessively taking turns, thus it is more and more sparse, until network convergence terminates.
Wherein, maxiter indicates the number of iterations.
Inventor also attempted following scheme: directly doing rarefaction to training result during realizing the application,
The absolute value of final mask weight and some threshold value comparison, if it is less than this threshold value with regard to clear 0.This method advantage is not change instruction
Practice method, it is simple and quick.Disadvantage is it is also obvious that be exactly when the ratio of rarefaction is relatively high, performance loss is very big.
The scheme of the embodiment of the present application may be implemented it is following the utility model has the advantages that pass through it is proposed that increment type rarefaction calculate
Method, the final rarefaction of network can reach 80% or more, and parameter (the parameter amount that sparse matrix itself stores greatly reduces
Can be small), calculating speed (sparse matrix calculating can accelerate) is promoted, and because being the Sparse Least of increment type, though each round
0 part weight is so set, but non-zero part still can participate in training, so the performance of identification will not reduce.Parameter is significantly
Reducing means that the speed of identification significantly becomes faster, and the consumption of memory is greatly lowered.From the perspective of Product Experience, identification
As a result more no-delay rapidly, customer experience effect is good.From the perspective of cost, the cheaper processing of more low side can be used
Device and memory thus greatly reduce the cost of product.
Referring to FIG. 5, a kind of sparse makeup of the neural network parameter matrix provided it illustrates one embodiment of the invention
The block diagram set.
As shown in figure 5, the rarefaction device 500 of neural network parameter matrix, including nuisance parameter selecting module 510, set 0
Module 520 and mark module 530.
Wherein, nuisance parameter selecting module 510, be configured to neural network parameter carry out each round repetitive exercise it
Before, select the multiple parameters of most redundancy in the neural network parameter matrix；0 module 520 is set, is configured to the most redundancy
Multiple parameters set 0；And mark module 530, the parameter after being configured to opposed 0 are marked and in repetitive exercise processes later
In parameter no longer after update mark.
In some alternative embodiments, above-mentioned nuisance parameter selecting module is further configured to: being joined to neural network
Before number carries out each round repetitive exercise, by parameter non-zero in the neural network parameter matrix according to the suitable of order of magnitude
Sequence arrangement；And the multiple parameters that the smallest multiple parameters of absolute value are most redundancy are selected based on preset ratio.
It should be appreciated that each step in all modules recorded in Fig. 5 and the method with reference to described in Fig. 1, Fig. 2 and Fig. 3
It is corresponding.The operation above with respect to method description and feature and corresponding technical effect are equally applicable to all in Fig. 5 as a result,
Module, details are not described herein.
It is worth noting that, the scheme that the module in embodiments herein is not intended to limit this application, such as segment
Module can be described as the module that received statement text is divided into saying He at least one entry.Furthermore it is also possible to by hard
Part processor realizes that related function module, such as word segmentation module can also realize that details are not described herein with processor.
In further embodiments, the embodiment of the invention also provides a kind of nonvolatile computer storage medias, calculate
Machine storage medium is stored with computer executable instructions, which can be performed in above-mentioned any means embodiment
Neural network parameter matrix rarefaction method；
As an implementation, nonvolatile computer storage media of the invention is stored with the executable finger of computer
It enables, computer executable instructions setting are as follows:
Before carrying out each round repetitive exercise to neural network parameter, select most superfluous in the neural network parameter matrix
Remaining multiple parameters；
The multiple parameters of the most redundancy are set 0；
Parameter after opposed 0 is marked and parameter during repetitive exercise later no longer after update mark.
Non-volatile computer readable storage medium storing program for executing may include storing program area and storage data area, wherein storage journey
It sequence area can application program required for storage program area, at least one function；Storage data area can be stored according to speech recognition
Device uses created data etc..In addition, non-volatile computer readable storage medium storing program for executing may include high random access
Memory, can also include nonvolatile memory, a for example, at least disk memory, flush memory device or other it is non-easily
The property lost solid-state memory.In some embodiments, it includes relative to processing that non-volatile computer readable storage medium storing program for executing is optional
The remotely located memory of device, these remote memories can pass through network connection to speech recognition equipment.The reality of above-mentioned network
Example includes but is not limited to internet, intranet, local area network, mobile radio communication and combinations thereof.
The embodiment of the present invention also provides a kind of computer program product, and computer program product is non-volatile including being stored in
Computer program on computer readable storage medium, computer program include program instruction, when program instruction is held by computer
When row, computer is made to execute any of the above-described audio recognition method.
Fig. 6 is the structural schematic diagram of electronic equipment provided in an embodiment of the present invention, as shown in fig. 6, the equipment includes: one
Or multiple processors 610 and memory 620, in Fig. 6 by taking a processor 610 as an example.The equipment of audio recognition method may be used also
To include: input unit 630 and output device 640.Processor 610, memory 620, input unit 630 and output device 640
It can be connected by bus or other modes, in Fig. 6 for being connected by bus.Memory 620 is above-mentioned non-volatile
Property computer readable storage medium.Processor 610 is by running the non-volatile software program being stored in memory 620, referring to
Order and module, thereby executing the various function application and data processing of server, i.e. realization above method embodiment voice
Recognition methods.Input unit 630 can receive the number or character information of input, and generates and set with the user of speech recognition equipment
It sets and the related key signals of function control inputs.Output device 640 may include that display screen etc. shows equipment.
Method provided by the embodiment of the present invention can be performed in the said goods, has the corresponding functional module of execution method and has
Beneficial effect.The not technical detail of detailed description in the present embodiment, reference can be made to method provided by the embodiment of the present invention.
As an implementation, above-mentioned electronic apparatus application is in the rarefaction device of neural network parameter matrix, packet
It includes:
At least one processor；And the memory being connect at least one processor communication；Wherein, memory stores
There is the instruction that can be executed by least one processor, instruction is executed by least one processor, so that at least one processor energy
It is enough:
Before carrying out each round repetitive exercise to neural network parameter, select most superfluous in the neural network parameter matrix
Remaining multiple parameters；
The multiple parameters of the most redundancy are set 0；
Parameter after opposed 0 is marked and parameter during repetitive exercise later no longer after update mark.
The electronic equipment of the embodiment of the present application exists in a variety of forms, including but not limited to:
(1) mobile communication equipment: the characteristics of this kind of equipment is that have mobile communication function, and to provide speech, data
Communication is main target.This Terminal Type includes: smart phone (such as iPhone), multimedia handset, functional mobile phone and low
Hold mobile phone etc..
(2) super mobile personal computer equipment: this kind of equipment belongs to the scope of personal computer, there is calculating and processing function
Can, generally also have mobile Internet access characteristic.This Terminal Type includes: PDA, MID and UMPC equipment etc., such as iPad.
(3) portable entertainment device: this kind of equipment can show and play multimedia content.Such equipment include: audio,
Video player (such as iPod), handheld device, e-book and intelligent toy and portable car-mounted navigation equipment.
(4) server: providing the equipment of the service of calculating, and the composition of server includes that processor, hard disk, memory, system are total
Line etc., server is similar with general computer architecture, but due to needing to provide highly reliable service, in processing energy
Power, stability, reliability, safety, scalability, manageability etc. are more demanding.
(5) other electronic devices with data interaction function.
The apparatus embodiments described above are merely exemplary, wherein unit can be as illustrated by the separation member
Or may not be and be physically separated, component shown as a unit may or may not be physical unit, i.e.,
It can be located in one place, or may be distributed over multiple network units.It can select according to the actual needs therein
Some or all of the modules achieves the purpose of the solution of this embodiment.Those of ordinary skill in the art are not paying creative labor
In the case where dynamic, it can understand and implement.
Finally, it should be noted that the above embodiments are merely illustrative of the technical solutions of the present invention, rather than its limitations；Although
Present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that: it still may be used
To modify the technical solutions described in the foregoing embodiments or equivalent replacement of some of the technical features；
And these are modified or replaceed, technical solution of various embodiments of the present invention that it does not separate the essence of the corresponding technical solution spirit and
Range.
Claims (10)
1. a kind of rarefaction method of neural network parameter matrix, comprising:
Before carrying out each round repetitive exercise to neural network parameter, most redundancy in the neural network parameter matrix is selected
Multiple parameters；
The multiple parameters of the most redundancy are set 0；
Parameter after opposed 0 is marked and parameter during repetitive exercise later no longer after update mark.
2. according to the method described in claim 1, wherein, it is described to neural network parameter carry out each round repetitive exercise it
Before, the multiple parameters for selecting most redundancy in the neural network parameter matrix include:
Before carrying out each round repetitive exercise to neural network parameter, by parameter non-zero in the neural network parameter matrix
It is arranged according to the sequence of order of magnitude；
The multiple parameters that the smallest multiple parameters of absolute value are most redundancy are selected based on preset ratio.
3. according to the method described in claim 2, wherein, the method also includes:
Before carrying out next round repetitive exercise to neural network parameter, the neural network never is selected in labeled parameter
The multiple parameters of most redundancy in parameter matrix；
The multiple parameters of the most redundancy are set 0；
Parameter after opposed 0 is marked and parameter during repetitive exercise later no longer after update mark.
4. according to the method described in claim 1, wherein, it is described to neural network parameter carry out each round repetitive exercise it
Before, the multiple parameters for selecting most redundancy in the neural network parameter matrix include:
Before carrying out each round repetitive exercise to neural network parameter, by the exhausted of the parameter in the neural network parameter matrix
Value and preset threshold are compared；
If the absolute value of multiple parameters is less than the preset threshold, the multiple parameter is determined as to multiple ginsengs of most redundancy
Number.
5. according to the method described in claim 4, wherein, the method also includes:
If the absolute value of certain parameters is more than or equal to the preset threshold, certain parameters are determined as nonredundancy parameter.
6. method according to any one of claims 1-5, wherein described that packet is marked to selected 0 parameter of setting
It includes:
Index position of the parameter in the neural network parameter matrix after setting 0 described in record.
7. a kind of rarefaction device of neural network parameter matrix, comprising:
Nuisance parameter selecting module is configured to before carrying out each round repetitive exercise to neural network parameter, selects the mind
Multiple parameters through most redundancy in network paramter matrix；
0 module is set, is configured to the multiple parameters of the most redundancy setting 0；
Mark module, the parameter after being configured to opposed 0 is marked and no longer update mark during repetitive exercise later
Parameter afterwards.
8. device according to claim 7, wherein the nuisance parameter selecting module is further configured to:
Before carrying out each round repetitive exercise to neural network parameter, by parameter non-zero in the neural network parameter matrix
It is arranged according to the sequence of order of magnitude；
The multiple parameters that the smallest multiple parameters of absolute value are most redundancy are selected based on preset ratio.
9. a kind of electronic equipment comprising: at least one processor, and deposited with what at least one described processor communication was connect
Reservoir, wherein the memory be stored with can by least one described processor execute instruction, described instruction by it is described at least
One processor executes, so that at least one described processor is able to carry out the step of any one of claim 1 to 6 the method
Suddenly.
10. a kind of storage medium, is stored thereon with computer program, which is characterized in that real when described program is executed by processor
The step of any one of existing claim 1 to 6 the method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910814977.XA CN110516806A (en) | 2019-08-30 | 2019-08-30 | The rarefaction method and apparatus of neural network parameter matrix |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910814977.XA CN110516806A (en) | 2019-08-30 | 2019-08-30 | The rarefaction method and apparatus of neural network parameter matrix |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110516806A true CN110516806A (en) | 2019-11-29 |
Family
ID=68628477
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910814977.XA Pending CN110516806A (en) | 2019-08-30 | 2019-08-30 | The rarefaction method and apparatus of neural network parameter matrix |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110516806A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111210010A (en) * | 2020-01-15 | 2020-05-29 | 上海眼控科技股份有限公司 | Data processing method and device, computer equipment and readable storage medium |
| CN114627342A (en) * | 2022-03-03 | 2022-06-14 | 北京百度网讯科技有限公司 | Training method, device and equipment of image recognition model based on sparsity |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107909147A (en) * | 2017-11-16 | 2018-04-13 | 深圳市华尊科技股份有限公司 | A kind of data processing method and device |
| CN107944555A (en) * | 2017-12-07 | 2018-04-20 | 广州华多网络科技有限公司 | Method, storage device and the terminal that neutral net is compressed and accelerated |
| CN108229644A (en) * | 2016-12-15 | 2018-06-29 | 上海寒武纪信息科技有限公司 | The device of compression/de-compression neural network model, device and method |
| CN109657781A (en) * | 2018-12-11 | 2019-04-19 | 中国航空工业集团公司西安航空计算技术研究所 | Deep neural network compression method, device and the terminal of Embedded application |
| CN109711532A (en) * | 2018-12-06 | 2019-05-03 | 东南大学 | A kind of accelerated method inferred for hardware realization rarefaction convolutional neural networks |
-
2019
- 2019-08-30 CN CN201910814977.XA patent/CN110516806A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108229644A (en) * | 2016-12-15 | 2018-06-29 | 上海寒武纪信息科技有限公司 | The device of compression/de-compression neural network model, device and method |
| CN107909147A (en) * | 2017-11-16 | 2018-04-13 | 深圳市华尊科技股份有限公司 | A kind of data processing method and device |
| CN107944555A (en) * | 2017-12-07 | 2018-04-20 | 广州华多网络科技有限公司 | Method, storage device and the terminal that neutral net is compressed and accelerated |
| CN109711532A (en) * | 2018-12-06 | 2019-05-03 | 东南大学 | A kind of accelerated method inferred for hardware realization rarefaction convolutional neural networks |
| CN109657781A (en) * | 2018-12-11 | 2019-04-19 | 中国航空工业集团公司西安航空计算技术研究所 | Deep neural network compression method, device and the terminal of Embedded application |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111210010A (en) * | 2020-01-15 | 2020-05-29 | 上海眼控科技股份有限公司 | Data processing method and device, computer equipment and readable storage medium |
| CN114627342A (en) * | 2022-03-03 | 2022-06-14 | 北京百度网讯科技有限公司 | Training method, device and equipment of image recognition model based on sparsity |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10832124B2 (en) | Generating natural language descriptions of images | |
| US9904875B2 (en) | Processing images using deep neural networks | |
| CN107342078B (en) | Conversation strategy optimized cold start system and method | |
| JP2020512639A5 (en) | ||
| WO2020253466A1 (en) | Method and device for generating test case of user interface | |
| CN106384591A (en) | Method and device for interacting with voice assistant application | |
| CN105930402A (en) | Convolutional neural network based video retrieval method and system | |
| DE112018006101T5 (en) | Dynamic registration of a user-defined wake-up key phrase for a speech-enabled computer system | |
| CN111523831B (en) | Risk group identification method and device, storage medium and computer equipment | |
| CN109119067B (en) | Speech synthesis method and device | |
| CN109741755A (en) | Voice wakes up word threshold management device and manages the method that voice wakes up word threshold value | |
| Williams et al. | The influence of evolution on population spread through patchy landscapes | |
| CN107657560A (en) | Knowledge point intensive training method, medium and electronic equipment | |
| CN110516806A (en) | The rarefaction method and apparatus of neural network parameter matrix | |
| CN105718943A (en) | Character selection method based on particle swarm optimization algorithm | |
| CN103390003A (en) | Method and device for combining user data information among servers | |
| CN110517692A (en) | Hot word audio recognition method and device | |
| CN115329744B (en) | Natural language processing method, system, equipment and storage medium | |
| CN111292262A (en) | Image processing method, image processing apparatus, electronic device, and storage medium | |
| Tan et al. | End-to-end supermask pruning: Learning to prune image captioning models | |
| CN115827838A (en) | Dialog generation method and system based on story continuous writing and dynamic knowledge base | |
| CN109710303A (en) | The multi version parallel developing method and system of interactive voice product | |
| CN108876009A (en) | The determination of coal mining accident prediction model and monitoring method, storage medium and electronic equipment | |
| CN104866091A (en) | Method and device for outputting audio-effect information in computer equipment | |
| CN109558131A (en) | A kind of intelligence generates the method and system of front end static page |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 215123 building 14, Tengfei Innovation Park, 388 Xinping street, Suzhou Industrial Park, Suzhou City, Jiangsu Province Applicant after: Sipic Technology Co.,Ltd. Address before: 215123 building 14, Tengfei Innovation Park, 388 Xinping street, Suzhou Industrial Park, Suzhou City, Jiangsu Province Applicant before: AI SPEECH Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191129 |
|
| RJ01 | Rejection of invention patent application after publication |