CN107562375A - More copy isomery storage methods and framework based on data value - Google Patents
More copy isomery storage methods and framework based on data value Download PDFInfo
- Publication number
- CN107562375A CN107562375A CN201710700649.8A CN201710700649A CN107562375A CN 107562375 A CN107562375 A CN 107562375A CN 201710700649 A CN201710700649 A CN 201710700649A CN 107562375 A CN107562375 A CN 107562375A
- Authority
- CN
- China
- Prior art keywords
- data
- value
- copy
- storage
- stored
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of more copy isomery storage methods based on data value, including：The dynamic value Vd of data is obtained according to data access frequency, the static value Vs of data delimited according to data type, the subjective value Vu of data is defined according to user's request；The total value V=wd*Vd+ws*Vs+wu*Vu of data is calculated, wherein wd is the weight of dynamic value, and ws is the weight of static value, and wu is the weight of subjective value, wd+ws+wu=1；By the total value of data compared with default first threshold and Second Threshold, data are divided into high price Value Data, middle value numerical value and low price Value Data；All copies of low price Value Data are stored on disk, all copies of high price Value Data are stored on solid state hard disc, a copy of middle value data are stored on solid state hard disc, remaining copy is stored on disk.According to the suitable storage strategy of the different choice of data value, by copy storage on a different storage medium, so as to reach the optimization to systematic function and cost.
Description
Technical field
It is different more particularly to a kind of more copies based on data value the present invention relates to a kind of isomery storage method of data
Structure storage method and framework.
Background technology
In distributed memory system, in order to ensure the reliability of data, data redundancy method, i.e. a data are employed
The method for storing multiple copies, the number of copies under default situations is 3, but is selection SSD or magnetic for the storage location of copy
Disk is stored, and system is randomly selected, so as to not fully take into account the characteristic of different storage mediums, it is impossible to given play to
The advantages of isomery stores, therefore strong influence is caused to the storage performance of system.Although SSD performance is largely
Better than disk, but it is not meant to that SSD can substitute disk, because SSD has the characteristic for writing preceding erasing, that is, is written to SSD's
Data can not can be covered directly as disk, it is necessary to which first erasing is repeated and write, so as to not only reducing SSD performance
And it have impact on SSD life-span.
Although being directed to the evaluation of data value in other researchs, dynamic value is only considered, dynamic value refers to
Be data access frequency height, change that visiting frequency can be over time and change, its data value can also change therewith,
Therefore referred to as dynamic value.Visiting frequency is more high then to represent that the use degree of data is higher, i.e., dynamic value is higher；Visiting frequency is got over
Low, then data are low using degree, i.e., dynamic value is low.In general, the data access frequency that newly writes is certainly very high, therefore can be with
Its magnitude of value is provided according to the visiting frequency in one day, a week or one month respectively.But if only consideration data
The data value that this factor of dynamic value obtains is often inaccuracy,
The content of the invention
For above-mentioned technical problem, the present invention seeks to：More copy isomeries based on data value are provided to deposit
Method for storing and framework, from the angle of data value, there is provided the isomery storage architecture of copy more than one, according to the difference of data value
Suitable storage strategy is selected, by copy storage on a different storage medium, so as to reach to the excellent of systematic function and cost
Change, dynamic value is both considered in the evaluation of data value, it is further contemplated that static value and subjective value so that the data valency of calculating
Value is more accurate.
The technical scheme is that：
A kind of more copy isomery storage methods based on data value, comprise the following steps：
S01：The dynamic value Vd of data is obtained according to data access frequency, the static valency of data delimited according to data type
Value Vs, the subjective value Vu of data is defined according to user's request；
S02：The total value V=wd*Vd+ws*Vs+wu*Vu of data is calculated, wherein wd is the weight of dynamic value, and ws is
The weight of static state value, wu is the weight of subjective value, wd+ws+wu=1；
S03：By the total value of data compared with default first threshold and Second Threshold, data are divided into high value number
According to, middle value numerical value and low price Value Data；
S04：All copies of low price Value Data are stored on disk, all copies of high price Value Data are stored in solid
On state hard disk, a copy of middle value data is stored on solid state hard disc, remaining copy is stored on disk.
Preferably, the total value of data is calculated in real time and compared with default first threshold and Second Threshold, judge data
Axiological idea whether be consistent with all copy storage locations of the data, if not being consistent, triggering transferring module change the data
The storage location of copy.
The invention also discloses a kind of more copy isomery storage architectures based on data value, including：
One data memory access logging modle, the memory access information of all data in record system；
One data value computing module, the dynamic value Vd of data is obtained according to data access frequency, according to data type
The static value Vs of data delimited, the subjective value Vu of data is defined according to user's request；Calculate the total value V=wd* of data
Vd+ws*Vs+wu*Vu, wherein wd are the weights of dynamic value, and ws is the weight of static value, and wu is the weight of subjective value,
Wd+ws+wu=1；
One monitoring and mapping block, real-time monitoring system, find all copies of data, judge the axiological ideas of data with
Whether all copy storage locations of the data are consistent, if not being consistent, change data value and the mapping of copy storage strategy are closed
System；
One transferring module, transferring module is triggered after monitoring have changed the storage strategy of data with mapping block, migrates mould
The copy storage strategy migrating data that root tuber is changed according to migration.
Compared with prior art, it is an advantage of the invention that：
From the angle of data value, there is provided the isomery storage architecture of copy more than one, according to the different choice of data value
Suitable storage strategy, by copy storage on a different storage medium, the advantage of different storage mediums is given full play to, using not
The advantages of with storage medium, the shortcomings that avoiding different storage mediums, so as to reach the optimization to systematic function and cost.Data valency
Dynamic value is both considered in the evaluation of value, it is further contemplated that static value and subjective value so that the data value of calculating is more accurate
Really, can completely embody data to system, to the importance of user.
Brief description of the drawings
Below in conjunction with the accompanying drawings and embodiment the invention will be further described：
Fig. 1 is the structural representation of more copy isomery storage architectures based on data value；
Fig. 2 is the flow chart of more copy isomery storage architectures based on data value.
Embodiment
Such scheme is described further below in conjunction with specific embodiment.It should be understood that these embodiments are to be used to illustrate
The present invention and be not limited to limit the scope of the present invention.The implementation condition used in embodiment can be done according to the condition of specific producer
Further adjustment, unreceipted implementation condition is usually the condition in normal experiment.
Embodiment：
As shown in figure 1, framework mainly includes four modules：Data memory access logging modle, data value computing module, monitoring
With mapping block and transferring module, wherein first three module is in the name server of distributed file system, constitutes one
Individual isomery storage control, last module are in data server, are an executors.
Data memory access logging modle is mainly responsible for the memory access information of all data in record system.The module is isomery storage
Inlet module, record the access type, access times and the time accessed every time of all data.Recorded by data memory access
Module can obtain data access frequency, so as to which the module needs the relevant information of collection being sent to data value calculating mould
Block, the information that the module provides are used for the dynamic value Vd for calculating data.
The main task of data value computing module is to calculate the total value of all data.Remembered by data memory access module
The information of record draws dynamic value Vd；The static value Vs of data delimited according to data type, data are defined according to user's request
Subjective value Vu.
Static state value refers to the value that the static feature of data in itself determines, such as different file types, size.One
As provide a magnitude of value respectively to different types of data by system manager.
Subjective value is that user defines according to their needs, can be provided by user.
Pass through formula V=wd*Vd+ws*Vs+wu*Vu (1)
Calculate the total value of data.Different data have different values, and the height of value determines depositing for they
Storage strategy.
Monitoring, it is necessary to monitor whole system in real time, finds all copies of data, judges the value of data with mapping block
Whether type is consistent with all copy storage locations of the data, if not being consistent, change data value and copy storage strategy
Mapping relations.The total value of data can be over time change and change, monitor data value and copy storage strategy
During mismatch, it is necessary to change the copy storage strategy of the data, and match copy storage strategy corresponding to data value therewith.
The function of transferring module is corresponding to migrating data to matching storage strategy in storage medium.When monitoring with
After mapping block have changed the storage strategy of data, transferring module will be triggered, transferring module will immediately be migrated and have changed
The data of copy storage strategy.
The difference of the difference of data storage feature as needed, i.e. data value selects different copies to store plan respectively
Slightly, the low data selection HDD strategies of data value, the high data selection SSD strategies of data value, the medium data of data value
Select SSD+HDD strategies.It is the description tactful to three kinds below：
HDD strategies：All copies for needing storage file are stored on disk.
SSD strategies：All copies for needing storage file are stored on solid state hard disc.
SSD+HDD strategies：One of storage file will be needed to store onto solid state hard disc, disk is arrived in the storage of remaining copy
On.This strategy is a combination of HDD strategies and SSD strategies, and the advantages of can making full use of HDD and SSD, being primarily adapted for use in needs
The file the to be stored data that both repetition in need is read or repetition in need is write.
The overall workflow of adaptive more copy isomery storages based on data value is made up of four major parts, with Fig. 1
Described in module design correspond, they are respectively：The calculating of data memory access record information, data value, system monitoring
With the migration of mapping mechanism and data.In order to which the working mechanism of whole system is explained in more detail, its workflow such as Fig. 2
It is shown
The later step of system operation is as follows：
The first step：Collect the memory access information of data.Including access time, access type and access times, so as to
To obtain access information of the data in different time sections, such as one hour, one day, a week.
Second step：Calculate data value.Data value is calculated according to the memory access information being collected into, different data values
Data are by using in different copy storage classes.
3rd step：Judge whether data value changes.Change over time, data value may change
Become, the mapping relations of different pieces of information value are different.If data value is changed, the 4th step can be carried out, if turning without if
To second step, continue to calculate data value.
4th step：Change the mapping relations of data.Data value is changed, then corresponding mapping relations will be fitted no longer
Close, so as to need to change mapping relations, therefore the copy storage strategy of data is changed.
5th step：Migrating data.After the copy storage strategy of data changes, the current storage class of data has been not suitable for
The storage of data, now just need to migrate data, in the storage class that copy is moved to corresponding storage strategy.
So far, the workflow of more copy isomery storage schemes based on data value terminates.
More copy isomery storage architectures based on data value proposed by the invention, it is most according to the difference of data value
Copy may be stored in most suitable storage medium.Its main target is to give full play to the advantage of different storage mediums,
The advantages of using different storage mediums, the shortcomings that avoiding different storage mediums, there is provided give user one storage compared with high performance-price ratio
Equipment.The cost of storage device so can be both controlled, equally also each storage medium is brought out one's strong points, ensures to deposit
Cost performance is improved as far as possible while storage system capacity.
The foregoing examples are merely illustrative of the technical concept and features of the invention, its object is to allow the person skilled in the art to be
Present disclosure can be understood and implemented according to this, it is not intended to limit the scope of the present invention.It is all smart according to the present invention
The equivalent transformation or modification that refreshing essence is done, should all be included within the scope of the present invention.
Claims (3)
1. a kind of more copy isomery storage methods based on data value, it is characterised in that comprise the following steps：
S01：The dynamic value Vd of data is obtained according to data access frequency, delimiting the static of data according to data type is worth
Vs, the subjective value Vu of data is defined according to user's request；
S02：The total value V=wd*Vd+ws*Vs+wu*Vu of data is calculated, wherein wd is the weight of dynamic value, and ws is static
The weight of value, wu are the weights of subjective value, wd+ws+wu=1；
S03：By the total value of data compared with default first threshold and Second Threshold, by data be divided into high price Value Data,
Middle value numerical value and low price Value Data；
S04：All copies of low price Value Data are stored on disk, it is hard that all copies of high price Value Data are stored in into solid-state
On disk, a copy of middle value data is stored on solid state hard disc, remaining copy is stored on disk.
2. more copy isomery storage methods according to claim 1 based on data value, it is characterised in that calculate in real time
The total value of data and compared with default first threshold and Second Threshold, judge all of the axiological ideas of data and the data
Whether copy storage location is consistent, if not being consistent, triggering transferring module changes the storage location of the data trnascription.
A kind of 3. more copy isomery storage architectures based on data value, it is characterised in that including：
One data memory access logging modle, the memory access information of all data in record system；
One data value computing module, the dynamic value Vd of data is obtained according to data access frequency, delimited according to data type
The static value Vs of data, the subjective value Vu of data is defined according to user's request；Calculate the total value V=wd*Vd+ of data
Ws*Vs+wu*Vu, wherein wd are the weights of dynamic value, and ws is the weight of static value, and wu is the weight of subjective value, wd+
Ws+wu=1；
One monitoring and mapping block, real-time monitoring system, all copies of data are found, judge the axiological idea and the number of data
According to all copy storage locations whether be consistent, if not being consistent, the mapping relations of change data value and copy storage strategy；
One transferring module, transferring module, transferring module root are triggered after monitoring have changed the storage strategy of data with mapping block
According to the copy storage strategy migrating data of migration change.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710700649.8A CN107562375A (en) | 2017-08-16 | 2017-08-16 | More copy isomery storage methods and framework based on data value |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710700649.8A CN107562375A (en) | 2017-08-16 | 2017-08-16 | More copy isomery storage methods and framework based on data value |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107562375A true CN107562375A (en) | 2018-01-09 |
Family
ID=60974092
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710700649.8A Pending CN107562375A (en) | 2017-08-16 | 2017-08-16 | More copy isomery storage methods and framework based on data value |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107562375A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104462389A (en) * | 2014-12-10 | 2015-03-25 | 上海爱数软件有限公司 | Method for implementing distributed file systems on basis of hierarchical storage |
| CN106469029A (en) * | 2011-12-31 | 2017-03-01 | 华为数字技术（成都）有限公司 | Data hierarchy storage processing method, device and storage device |
| WO2017085792A1 (en) * | 2015-11-17 | 2017-05-26 | 株式会社日立製作所 | Storage system and storage system control method |
-
2017
- 2017-08-16 CN CN201710700649.8A patent/CN107562375A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106469029A (en) * | 2011-12-31 | 2017-03-01 | 华为数字技术（成都）有限公司 | Data hierarchy storage processing method, device and storage device |
| CN104462389A (en) * | 2014-12-10 | 2015-03-25 | 上海爱数软件有限公司 | Method for implementing distributed file systems on basis of hierarchical storage |
| WO2017085792A1 (en) * | 2015-11-17 | 2017-05-26 | 株式会社日立製作所 | Storage system and storage system control method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103927251B (en) | Log management method for embedded system | |
| CN102156738B (en) | Method for processing data blocks, and data block storage equipment and system | |
| CN104050097B (en) | Selected between the Nonvolatile memery unit with different smallest addressable data unit sizes | |
| US9141457B1 (en) | System and method for predicting multiple-disk failures | |
| US5764880A (en) | Method and system for rebuilding log-structured arrays | |
| US5794242A (en) | Temporally and spatially organized database | |
| US7653781B2 (en) | Automatic RAID disk performance profiling for creating optimal RAID sets | |
| US11204705B2 (en) | Retention-aware data tiering algorithm for hybrid storage arrays | |
| CN104583930B (en) | Method, controller and the data migration device of Data Migration | |
| CN103135940B (en) | Implementing enhanced fragmented stream handling in a shingled disk drive | |
| CN107037976A (en) | Storage device and its method of work | |
| CN104268099B (en) | A kind of method and device for managing reading and writing data | |
| CN107003937A (en) | For the cache optimization technology of large-scale work data set | |
| CN107273046B (en) | A kind of data processing method and system based on Solid-state disc array | |
| US8234464B2 (en) | Hybrid storage data migration by selective data removal | |
| CN106463145B (en) | Cache systems and method for hard disk drive and hybrid drive | |
| US20170364447A1 (en) | Read cache management | |
| CN101004936A (en) | Information storage device and its control method | |
| CN108897492A (en) | A kind of method for writing data and device | |
| KR20090129791A (en) | Multiple value flash memory | |
| CN102184080B (en) | Curing band based data reading and writing method used after capacity expansion of RAID (Redundant Array of Independent Disk) system | |
| CN107092525A (en) | data cache method and device | |
| Ma et al. | Alleviating hot data write back effect for shingled magnetic recording storage systems | |
| CN106095330A (en) | A kind of storage method and apparatus of metadata | |
| CN105913857A (en) | Magnetic disk drive and rewrite processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180109 |
|
| RJ01 | Rejection of invention patent application after publication |