CN110298442A - A kind of multidomain treat-ment method of intelligent weight information - Google Patents
A kind of multidomain treat-ment method of intelligent weight information Download PDFInfo
- Publication number
- CN110298442A CN110298442A CN201910483424.0A CN201910483424A CN110298442A CN 110298442 A CN110298442 A CN 110298442A CN 201910483424 A CN201910483424 A CN 201910483424A CN 110298442 A CN110298442 A CN 110298442A
- Authority
- CN
- China
- Prior art keywords
- weight information
- memory
- stored
- multidomain treat
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 31
- 238000011282 treatment Methods 0.000 title claims abstract description 24
- 230000015654 memory Effects 0.000 claims abstract description 52
- 238000013528 artificial neural network Methods 0.000 claims abstract description 20
- 230000008859 change Effects 0.000 claims abstract description 18
- 238000003062 neural network model Methods 0.000 abstract description 3
- 230000008569 process Effects 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000005303 weighing Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Neurology (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Image Analysis (AREA)
Abstract
The present invention relates to information technology field more particularly to a kind of multidomain treat-ment methods of intelligent weight information, applied to the neural network with weight information；Wherein, during constantly train to neural network, the weight information that change frequency is greater than the first preset threshold is stored in a first kind memory；And change frequency is stored in one second class memory less than the weight information of the first preset threshold；It can be by constantly learning, weight information is stored in different positions, so that training speed is getting faster, and the type of storaging medium increases, the storaging medium that can be adapted for the feature setting of the weight information of storage, evidence of fetching from other memory again is not needed, the price of trained neural network model equipment is reduced yet.
Description
Technical field
The present invention relates to information technology field more particularly to a kind of multidomain treat-ment methods of intelligent weight information.
Background technique
Important branch of the deep learning as artificial intelligence, performer key player, software-hardware synergism innovation are depth
It practises and injects new power.The optimization acceleration of hardware is then set about from two stages of deep learning, respectively training stage and reasoning rank
Section.There is tensor processor (tensor processor unit, the TPU) generation in special useful reasoning stage, also takes into account training
CPU (Central Processing Unit central processing unit, abbreviation CPU), the GPU (Graphics in stage and reasoning stage
Processing Unit graphics processor, abbreviation GPU), NPU (neural-networkprocess units network processing unit,
Abbreviation NPU) etc..The weight information of the neural network in these frameworks requires to access data by memory at present, because refreshing
Quantity through weight information in network is very big, so needing the memory of large capacity, and DRAM (Dynamic Random Access
Memory dynamic random access memory, abbreviation DRAM) be volatile memory, need constantly to refresh, thus its power consumption compared with
Greatly, simultaneously as DRAM power failure data will be lost, it is therefore necessary to by the weight information storage of trained model to large capacity
In nonvolatile memory, such as HDD (Hard Disk Drive hard disk drive, abbreviation HDD), SSD (Solid State
Disk solid state hard disk, abbreviation SSD) in, when powering on next time, need fetch from HDD or SSD evidence, in this way causing property
It can be with the waste of power consumption.In addition, the relevance of most weight information is smaller in neural network, and most of weight information
Variation it is relatively slow or be basically unchanged, then, when storing these weights, can't be distinguished for current neural network
These weight informations, the weight information relatively slow for those constant or variations, DRAM equally will do it refreshing, update weight
Information then causes the waste of unnecessary time and power consumption in this way.
Summary of the invention
In view of the above-mentioned problems, the invention proposes a kind of multidomain treat-ment method of intelligent weight information, applied to having
The neural network of weight information；
Wherein, during constantly train to the neural network, change frequency is greater than the first preset threshold
The weight information be stored in a first kind memory；And
The weight information that change frequency is less than first preset threshold is stored in one second class memory.
Above-mentioned multidomain treat-ment method, wherein judge the change frequency of the weight information in institute by a processing system
State the size between the first preset threshold.
Above-mentioned multidomain treat-ment method, wherein the processing system is the computer with an operating system.
Above-mentioned multidomain treat-ment method, wherein there is relevance between the weight information；
Also the weight information that relevance is greater than the second preset threshold is stored in the first kind memory；And
Relevance is stored in the second class memory less than the weight information of the second preset threshold.
Above-mentioned multidomain treat-ment method, wherein the first kind memory is dynamic RAM.
Above-mentioned multidomain treat-ment method, wherein the second class memory is that phase transition storage or flash memory or resistance are dynamic
State random access memory or ferroelectric dynamic random access memory, or magnetic dynamic RAM.
Above-mentioned multidomain treat-ment method, wherein the Application of Neural Network is in a learning model.
The utility model has the advantages that a kind of multidomain treat-ment method of intelligent weight information proposed by the present invention, it can be by constantly
Weight information, is stored in different positions by study, so that training speed is getting faster, and the type of storaging medium increases,
The storaging medium that can be adapted for the feature setting of the weight information of storage, does not need to take from other memory again yet
Data reduce the price of trained neural network model equipment.
Detailed description of the invention
Fig. 1 is that each unit connects to be formed in the multidomain treat-ment method of weight information intelligent in one embodiment of the invention
Structure principle chart.
Specific embodiment
Invention is further explained with reference to the accompanying drawings and examples.
In a preferred embodiment, a kind of multidomain treat-ment method of intelligent weight information is proposed, can be applied
In the neural network with weight information；
Wherein, during constantly train to neural network, change frequency is greater than to the power of the first preset threshold
Weight information is stored in a first kind memory；And
Change frequency is stored in one second class memory less than the weight information of the first preset threshold.
The multidomain treat-ment method of the weight information of above-mentioned intelligence can be applied to the study stage of neural network；Due to power
The weight information that weight information is being gradually changed with trained change, therefore stored in first kind memory generally can be with training
Process constantly tail off；First kind memory is generally memory, completes the learning process of neural network；At the beginning of learning process,
All weight informations, which can be, to be stored entirely in first kind memory, when weight information is stored into the second class memory
Afterwards, corresponding weight information can be deleted in first kind memory.
In a preferred embodiment, judge the change frequency of weight information in the first default threshold by a processing system
Size between value.
In above-mentioned technical proposal, as shown in Figure 1, the connection of processing system and first kind memory and the second class memory can
With as shown in the figure.
In above-described embodiment, it is preferable that processing system can be the computer with an operating system.
In above-described embodiment, it is preferable that there is relevance between weight information；
Also the weight information that relevance is greater than the second preset threshold is stored in first kind memory；And
Relevance is stored in the second class memory less than the weight information of the second preset threshold.
In a preferred embodiment, first kind memory can be dynamic RAM.
In a preferred embodiment, the second class memory can for phase transition storage or flash memory or resistance dynamic with
Machine memory or ferroelectric dynamic random access memory, or magnetic dynamic RAM etc..
Above-mentioned multidomain treat-ment method, wherein Application of Neural Network is in a learning model.
Specifically, the variation speed of weight information and the relevance of weight are determined by operating system.Carrying out learning tasks
Before, by the way that one second preset threshold m is arranged, when operating system calculates the relevance between the weight information in neural network
When coefficient n is greater than m, i.e. the relevance of judgement weight information is larger, otherwise, when m is greater than n, that is, determines the association of weight information
Property is smaller.Meanwhile one first preset threshold x can be set, when operating system calculates the change of the weight information in neural network
When changing property coefficient y greater than x, i.e. the variation of judgement weight information is very fast, otherwise, when x is greater than y, that is, determines the change of weight information
Change slower.The big and/or biggish weight information of relevance will be changed to be put into first kind memory, such as memory dram, and that
Small and/or relevance is lesser is put into the second class memory for a little weight informations variations, such as nonvolatile memory, is equivalent to
Have compressed the weight information in DRAM, in this way can so that the process of training is accelerated, meanwhile, memory can be allowed to give full play to its length
Place, while avoiding consuming a large amount of power consumption.
Meanwhile in training process over time, it has more and more weight informations to tend towards stability, operating system
The lesser weight information of partially tend towards stability in first kind memory and/or relevance is judged, and is stored in
In two class memories, it is always maintained at the high quality of weight information in first kind memory, i.e. first kind memory so always
Store that relevance is big and/or the weight information that changes greatly is continuously updated and eliminates the first kind by constantly learning and deposit
Weight information in reservoir, and the information that these are eliminated is transferred in the second class memory, gets over trained process
Come it is faster, meanwhile, but also first kind memory can vacate the weight letter that space is gone storage relevance big and/or changed greatly
Breath.
In conclusion a kind of multidomain treat-ment method of intelligent weight information proposed by the present invention, is applied to have weight
The neural network of information；Wherein, during constantly train to neural network, change frequency is greater than the first default threshold
The weight information of value is stored in a first kind memory；And the weight information by change frequency less than the first preset threshold is deposited
In Chu Yi the second class memory；Weight information can be stored in different positions by constantly learning, so that training speed
Degree is getting faster, and the type of storaging medium increases, and can be deposited for what the feature setting of the weight information of storage was adapted
Medium is stored up, evidence of fetching from other memory again is not needed yet, reduces the price of trained neural network model equipment.
By description and accompanying drawings, the exemplary embodiments of the specific structure of specific embodiment are given, based on present invention essence
Mind can also make other conversions.Although foregoing invention proposes existing preferred embodiment, however, these contents are not intended as
Limitation.
For a person skilled in the art, after reading above description, various changes and modifications undoubtedly be will be evident.
Therefore, appended claims should regard the whole variations and modifications for covering true intention and range of the invention as.It is weighing
The range and content of any and all equivalences, are all considered as still belonging to the intent and scope of the invention within the scope of sharp claim.
Claims (7)
1. a kind of multidomain treat-ment method of intelligent weight information, applied to the neural network with weight information；
It is characterized in that, change frequency is greater than the first default threshold during constantly train to the neural network
The weight information of value is stored in a first kind memory；And
The weight information that change frequency is less than first preset threshold is stored in one second class memory.
2. multidomain treat-ment method according to claim 1, which is characterized in that judge that the weight is believed by a processing system
Size of the change frequency of breath between first preset threshold.
3. multidomain treat-ment method according to claim 2, which is characterized in that the processing system is with an operating system
Computer.
4. multidomain treat-ment method according to claim 1, which is characterized in that have relevance between the weight information；
Also the weight information that relevance is greater than the second preset threshold is stored in the first kind memory；And
Relevance is stored in the second class memory less than the weight information of the second preset threshold.
5. multidomain treat-ment method according to claim 1, which is characterized in that the first kind memory is deposited for dynamic random
Reservoir.
6. multidomain treat-ment method according to claim 1, which is characterized in that the second class memory is phase change memory
Device or flash memory or resistance dynamic RAM or ferroelectric dynamic random access memory, or magnetic dynamic RAM.
7. multidomain treat-ment method according to claim 1, which is characterized in that the Application of Neural Network is in a learning model
In.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910483424.0A CN110298442B (en) | 2019-06-04 | 2019-06-04 | Intelligent partition processing method for weight information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910483424.0A CN110298442B (en) | 2019-06-04 | 2019-06-04 | Intelligent partition processing method for weight information |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110298442A true CN110298442A (en) | 2019-10-01 |
| CN110298442B CN110298442B (en) | 2024-03-22 |
Family
ID=68027591
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910483424.0A Active CN110298442B (en) | 2019-06-04 | 2019-06-04 | Intelligent partition processing method for weight information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110298442B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150317556A1 (en) * | 2014-04-30 | 2015-11-05 | Prophetstor Data Services, Inc. | Adaptive quick response controlling system for software defined storage system for improving performance parameter |
| CN107909142A (en) * | 2017-11-14 | 2018-04-13 | 深圳先进技术研究院 | A kind of parameter optimization method of neutral net, system and electronic equipment |
| CN108122031A (en) * | 2017-12-20 | 2018-06-05 | 杭州国芯科技股份有限公司 | A kind of neutral net accelerator architecture of low-power consumption |
-
2019
- 2019-06-04 CN CN201910483424.0A patent/CN110298442B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150317556A1 (en) * | 2014-04-30 | 2015-11-05 | Prophetstor Data Services, Inc. | Adaptive quick response controlling system for software defined storage system for improving performance parameter |
| CN107909142A (en) * | 2017-11-14 | 2018-04-13 | 深圳先进技术研究院 | A kind of parameter optimization method of neutral net, system and electronic equipment |
| CN108122031A (en) * | 2017-12-20 | 2018-06-05 | 杭州国芯科技股份有限公司 | A kind of neutral net accelerator architecture of low-power consumption |
Non-Patent Citations (1)
| Title |
|---|
| 李丽霞: "BP神经网络设计探讨", 《现代预防医学》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110298442B (en) | 2024-03-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110276367B (en) | Learning classification device and learning classification method | |
| US11663125B2 (en) | Cache configuration performance estimation | |
| CN108122031B (en) | Low-power consumption neural network accelerator device | |
| CN112329948B (en) | Multi-agent strategy prediction method and device | |
| EP3973401B1 (en) | Interleaving memory requests to accelerate memory accesses | |
| US7934058B2 (en) | Predictive caching of assets to improve level load time on a game console | |
| CN105242871A (en) | Data writing method and apparatus | |
| JP2005322215A5 (en) | ||
| CN112051969A (en) | Hierarchical storage data self-adaptive migration method based on deep reinforcement learning | |
| CN109240280A (en) | Anchoring auxiliary power positioning system control method based on intensified learning | |
| CN114692829A (en) | DNN model-based checkpoint selection method, equipment and storage medium | |
| CN110298442A (en) | A kind of multidomain treat-ment method of intelligent weight information | |
| KR102192325B1 (en) | Data management device supporting high speed artificial neural network operation with caching data based on data locality of artificial neural networks | |
| KR20200139620A (en) | Data management device supporting high speed artificial neural network operation with caching data based on data locality of artificial neural networks | |
| Ali | Cache replacement algorithm | |
| CN109685208A (en) | A kind of method and device accelerated for the dilute combization of neural network processor data | |
| JP2019164629A (en) | Model generation device, information processing device, model generation method and program | |
| CN115148248B (en) | Deep learning-based DRAM (dynamic random Access memory) refreshing method and device | |
| Gulden et al. | Beyond Zipf: An agent-based understanding of city size distributions | |
| US20210200584A1 (en) | Multi-processor system, multi-core processing device, and method of operating the same | |
| CN109857742A (en) | A kind of big data virtual tables fast display method and system | |
| KR101351648B1 (en) | Multi-media server for managing contents using prediction of user's requests, and management method thereof | |
| JP2019160256A (en) | Learning discrimination device and method for learning discrimination | |
| Walker | Rentier state theory 50 Years on: New developments | |
| Min et al. | Learning to play visual doom using model-free episodic control |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |