CN107193500A - A kind of distributed file system Bedding storage method and system - Google Patents
A kind of distributed file system Bedding storage method and system Download PDFInfo
- Publication number
- CN107193500A CN107193500A CN201710385437.5A CN201710385437A CN107193500A CN 107193500 A CN107193500 A CN 107193500A CN 201710385437 A CN201710385437 A CN 201710385437A CN 107193500 A CN107193500 A CN 107193500A
- Authority
- CN
- China
- Prior art keywords
- data storage
- disk
- stored
- storage
- cold
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
- G06F3/0607—Improving or facilitating administration, e.g. storage management by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0647—Migration mechanisms
- G06F3/0649—Lifecycle management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
- G06F3/0685—Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
Abstract
The invention provides a kind of distributed file system Bedding storage method and system, this method includes：When receiving data to be stored, by the dsc data storage disk under data Cun Chudao first layer storage catalogues to be stored；Periodically detection is stored in the access times of the first data storage in dsc data storage disk；When the access times of the first data storage are less than default first heat degree threshold, first data storage is moved in the cold data storage disk under second layer storage catalogue, when the ratio that the internal memory shared by the data stored in cold data storage disk accounts for cold data storage disk memory capacity has exceeded default proportion threshold value, it will meet in backup disk of the second data storage of preset rules under third layer storage catalogue and back up.By the method for the present embodiment, the data hierarchy of different access amount is stored, the use cost of disk is not only controlled, and simplify the flow of Distributed File System Data storage.
Description
Technical field
The present invention relates to the field of distributed file system storage, more particularly to a kind of distributed file system Bedding storage
Method and system.
Background technology
At present, the data exponentially type of various applications generations, explosive growth, these data of generation are needed in distribution
Stored in file system.For the data of the different access frequency, the requirement to storage hard disk is also different, for visitation frequency
Higher dsc data needs the higher disks of IOPS, such as SSD (English full name：Solid State Disk, Chinese full name：Gu
State hard disk), higher SAS (the English full name of rotating speed：Serial Attached SCSI, Chinese full name：Serial Attached SCSI (SAS)) disk
Or flash memory, and can be stored for the relatively low cold data of visitation frequency with common hard disc or tape library.
But, in the prior art, it is impossible to which, according to the temperature of data, intelligentized storage is into corresponding disk.
The content of the invention
In view of this, the embodiments of the invention provide a kind of distributed file system Bedding storage method and system, solve
In the prior art, it is impossible to which, according to the different access amount of data storage, intelligentized storage is controlled into corresponding disk
The use cost of disk.
A kind of distributed file system Bedding storage method provided in an embodiment of the present invention, methods described includes：
When receiving data to be stored, by the hot number under the data Cun Chudao first layer storage catalogues to be stored
According in storage disk；
Periodically detection is stored in the access times of the first data storage in the dsc data storage disk；
When the access times of first data storage are less than default first heat degree threshold, number is stored by described first
According in the cold data storage disk moved under second layer storage catalogue.
Optionally, in addition to：
Set up first layer storage catalogue and second layer storage catalogue；
The first layer storage catalogue is bound into dsc data storage disk；Wherein, the dsc data storage disk includes：
SSD solid state hard discs, SAS disks or flash memory；
The second layer storage catalogue is bound into cold data storage disk；Wherein, stating cold data storage disk includes：Commonly
Hard disk.
Optionally, in addition to：
Hold when the internal memory shared by the data stored in the cold data storage disk accounts for the cold data storage disk storage
When the ratio of amount has exceeded default proportion threshold value, determine to meet the second storage of preset rules in the cold data storage disk
Data；
It will be backed up in backup disk of second data storage under third layer storage catalogue；
Delete the second data storage in the cold data storage disk.
Optionally, second data storage for determining to meet preset rules in the cold data storage disk, including：
Detection is stored in the access times of the data storage in the cold data storage disk；
By within the default period access times be that zero data storage is defined as the second data storage；
Or,
Judge whether the data storage being stored in the cold data storage disk has exceeded the data storage of preset time；
Data storage more than preset time is defined as the second data storage.
Optionally, in addition to：
Periodically detection is stored in the access times of the 3rd data storage in cold data storage disk；
When the access times of the 3rd data storage are more than default second heat degree threshold, number is stored by the described 3rd
According in the dsc data storage disk moved under first layer storage catalogue.
The embodiment of the present invention additionally provides a kind of distributed file system Bedding storage system, and the system includes：First
Memory cell, for when receiving data to be stored, by under the data Cun Chudao first layer storage catalogues to be stored
Dsc data storage disk in；
First detection unit, for periodically detecting the first data storage being stored in the dsc data storage disk
Access times；
First migration units, are less than default first heat degree threshold for the access times when first data storage
When, first data storage is moved in the cold data storage disk under second layer storage catalogue.
Optionally, the system can also include：
Unit is set up, for setting up first layer storage catalogue and second layer storage catalogue；
First binding unit, for the first layer storage catalogue to be bound into dsc data storage disk；Wherein, the hot number
Include according to storage disk：SSD solid state hard discs, SAS disks or flash memory；
Second binding unit, for the second layer storage catalogue to be bound into cold data storage disk；Wherein, cold data is stated
Storage disk includes：Common hard disc.
Optionally, the system can also include：
Determining unit, for being deposited when the internal memory shared by the data stored in the cold data storage disk accounts for the cold data
When the ratio of storage disk storage capacity has exceeded default proportion threshold value, determine to meet default rule in the cold data storage disk
The second data storage then；
Backup units, for will be backed up in backup disk of second data storage under third layer storage catalogue；
Unit is deleted, for deleting the second data storage in the cold data storage disk.
Optionally, the determining unit, including：
Detection sub-unit, the access times for detecting the data storage being stored in the cold data storage disk；
First determination subelement, for by within the default period access times be zero the data storage determine
For the second data storage；
Or,
Whether judgment sub-unit, the data storage for judging to be stored in the cold data storage disk has exceeded default
The data storage of time；
Second determination subelement, for the data storage more than preset time to be defined as into the second data storage.
Optionally, the system can also include：
Second detection unit, the visit for periodically detecting the 3rd data storage being stored in cold data storage disk
Ask number of times；
Second migration units, are more than default second heat degree threshold for the access times when the 3rd data storage
When, the 3rd data storage is moved in the dsc data storage disk under first layer storage catalogue.
In the present embodiment, when receiving data to be stored, the data Cun Chudao first layers to be stored are stored
In dsc data storage disk under catalogue；Periodically detection is stored in the first data storage in the dsc data storage disk
Access times；When the access times of first data storage are less than default first heat degree threshold, described first is deposited
Store up in the cold data storage disk under Data Migration to second layer storage catalogue.By the method for the present embodiment, by different access
The data hierarchy storage of amount, not only controls the use cost of disk, and simplifies Distributed File System Data storage
Flow.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
There is the accompanying drawing used required in technology description to be briefly described, it should be apparent that, drawings in the following description are only this
The embodiment of invention, for those of ordinary skill in the art, on the premise of not paying creative work, can also basis
The accompanying drawing of offer obtains other accompanying drawings.
Fig. 1 shows a kind of flow signal of distributed file system Bedding storage method provided in an embodiment of the present invention
Figure；
Fig. 2 shows a kind of flow signal of distributed file system Bedding storage method provided in an embodiment of the present invention
Figure；
Fig. 3 shows a kind of flow signal of distributed file system Bedding storage method provided in an embodiment of the present invention
Figure；
Fig. 4 shows a kind of schematic diagram of distributed file system Bedding storage logic provided in an embodiment of the present invention；
Fig. 5 shows a kind of structural representation of distributed file system hierarchical system provided in an embodiment of the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Site preparation is described, it is clear that described embodiment is only a part of embodiment of the invention, rather than whole embodiments.It is based on
Embodiment in the present invention, it is every other that those of ordinary skill in the art are obtained under the premise of creative work is not made
Embodiment, belongs to the scope of protection of the invention.
With reference to Fig. 1, a kind of flow of distributed file system Bedding storage method provided in an embodiment of the present invention is shown
Schematic diagram, in the present embodiment, methods described may comprise steps of in S101~S103：
S101：When receiving data to be stored, by under the data Cun Chudao first layer storage catalogues to be stored
Dsc data storage disk in；
In the present embodiment, S201~S203 in can also comprising the following steps before S101：
S201：Set up first layer storage catalogue and second layer storage catalogue；
S202：The first layer storage catalogue is bound into dsc data storage disk；Wherein, the dsc data storage disk bag
Include：SSD solid state hard discs, SAS disks or flash memory；
S203：The second layer storage catalogue is bound into cold data storage disk；Wherein, the cold data storage disk bag
Include：Common hard disc.
In the present embodiment, it is higher that the dsc data storage disk under first layer storage catalogue can be understood as storage hot value
Data, it is understood that for the higher data of storage access times.Cold data storage disk under second layer storage catalogue can be with
Be interpreted as storing the relatively low data of hot value, it is understood that be store access times it is relatively low or be zero data.Wherein,
One layer of storage catalogue is also denoted as hotdir catalogues, and second layer storage catalogue is also denoted as colddir catalogues.
In the present embodiment, it can be the higher disks of IOPS for dsc data storage disk, for example, can include：SSD, turn
Speed higher SAS disks, flash memory etc.；Cold data storage disk can include：Ordinary magnetic disc, for example, can include：Mechanical hard disk.
In the present embodiment, the two data knots of first layer storage catalogue and second layer storage catalogue equivalent to meta-data preservation
Structure, first layer storage catalogue and the binding of dsc data storage disk and second layer storage catalogue and the binding of cold data storage disk
Afterwards, storage is to the data under the first storage catalogue or the second storage catalogue, in fact, equivalent to storing that data in hot number
According in storage disk or in cold data storage disk.
In the present embodiment, the data being deposited into for the first time in distributed file system can be the higher data of hot value,
These data need to store into the dsc data storage disk under first layer storage catalogue.
S102：Periodically detection is stored in the access times of the first data storage in the dsc data storage disk；
In the present embodiment, the first data storage can represent any one data in dsc data storage disk.
In the present embodiment, the access times of the first data storage in a cycle period can be periodically detected,
Exactly detect the access times of any one data storage.
Specifically, S102 can also include：The access times of first data storage in the detection unit interval in real time, then
The access times in a cycle period are periodically counted again.
S103：When the access times of first data storage are less than default first heat degree threshold, by described first
Data storage is moved in the cold data storage disk under second layer storage catalogue.
In the present embodiment, the first heat degree threshold can technical staff rule of thumb set.When the first data storage
Access times are relatively low or when be zero, the first data storage is moved into second layer storage mesh from dsc data storage disk
In cold data storage disk under record.In the present embodiment, when the access times of the first data storage, more than or equal to default first
During heat degree threshold, the first data storage is not moved.
In the present embodiment, when data storage is moved in the cold data storage disk under second layer storage catalogue, due to
The memory capacity of cold data storage disk is limited, after more data storage is stored in cold data storage disk, if too late
Shi Qingli, can influence follow-up data storage, to the Data Migration of cold data storage disk, therefore, may be used also from dsc data storage disk
With including：S301~S303 in following steps：
S301：Deposited when the internal memory shared by the data stored in the cold data storage disk accounts for the cold data storage disk
When the ratio of storage capacity has exceeded default proportion threshold value, determine to meet the second of preset rules in the cold data storage disk
Data storage；
S302：It will be backed up in backup disk of second data storage under third layer storage catalogue；
S303：Delete the second data storage in the cold data storage disk.
In the present embodiment, can include for default rule it is multiple, for example can be according to the access times of data storage
It can also be determined according to the storage time of data storage, specifically, S301 can include two following aspects：
First aspect：Detection is stored in the access times of the data storage in the cold data storage disk；
Using within the default period access times be that zero data storage is used as the second data storage；
Or,
Second aspect：Judge whether include having exceeded in the data storage being stored in the cold data storage disk presetting
The data storage of time；
The second data storage will be used as more than the data storage of preset time.
Illustrate：
It is directed to first aspect：Assuming that the data stored in cold data storage disk account for cold data storage disk storage appearance
The 80% of amount, detection is stored in the access times of data storage in cold data storage disk, if detecting within a period of time (example
Such as can be：Two weeks or one month) interior access times are zero, then these data backuped under third layer storage catalogue
Backup disk in, and delete in cold data storage disk these data.
It is directed to second aspect：Assuming that the data stored in cold data storage disk account for cold data storage disk storage appearance
The 80% of amount, the data storage that will can be preserved before 2 months is backuped in the backup disk under third layer storage disk, and
These data are deleted in cold data storage disk.
, can be other lower-cost for tape library etc. for the backup disk in third layer storage disk in the present embodiment
Storage disk.
, can also be because the increasing of visit capacity for the data storage being stored under second layer storage catalogue in the present embodiment
Plus, moved to from the cold data storage disk in second layer catalogue in the dsc data storage disk under the first catalogue, specifically,
It can also include：
Periodically detection is stored in the access times of the 3rd data storage in cold data storage disk；
When the access times of the 3rd data storage are more than default second heat degree threshold, number is stored by the described 3rd
According in the dsc data storage disk moved under first layer storage catalogue.
In the present embodiment, the 3rd data storage can be stored in any one data in cold data storage disk.
In the present embodiment, when receiving data to be stored, the data Cun Chudao first layers to be stored are stored
In dsc data storage disk under catalogue；Periodically detection is stored in the first data storage in the dsc data storage disk
Access times；When the access times of first data storage are less than default first heat degree threshold, described first is deposited
Store up in the cold data storage disk under Data Migration to second layer storage catalogue.By the method for the present embodiment, by different access
The data hierarchy storage of amount, not only controls the use cost of disk, and simplifies Distributed File System Data storage
Flow.
With reference to Fig. 4, a kind of signal of distributed file system Bedding storage logic provided in an embodiment of the present invention is shown
Figure；
As shown in the figure：Hotdir catalogues and colddir catalogues are set up in distributed file system, wherein, by hotdir
Catalogue is bound with SSD；Colddir catalogues and common hard disc are bound；And tape library is connected with the distributed file system.
When producing data to be stored in client, first by SSD disks of the data storage to be stored under hotdir catalogues.Week
The access times of first data storage of the detection SSD disk storages of phase property, preset when the access times of the first data storage are less than
The first heat degree threshold when, first data storage is moved in the common hard disc under colddir catalogues.If
The internal memory of the data stored in common hard disc under colddir catalogues account for common hard disc memory capacity ratio exceeded it is default
During proportion threshold value, the data that visit capacity in a period of time is zero always can be backed up in tape library, and delete common hard disc
In a period of time in visit capacity be always zero data；Or, the data of longer period of time can also will be stored in magnetic
Backed up in tape pool, and delete the data that these in common hard disc store longer period of time.
Wherein, it can also be stored directly in the common hard disc under colddir catalogues for the data produced once in a while.
In the present embodiment, the data hierarchy of different access amount is stored, the use cost of disk, Er Qiejian is not only controlled
The flow of Distributed File System Data storage is changed.
With reference to Fig. 5, a kind of structural representation of distributed file system hierarchical system provided in an embodiment of the present invention is shown
Figure, in the present embodiment, the system includes：
First memory cell 501, for when receiving data to be stored, by the data Cun Chudao to be stored
In dsc data storage disk under one layer of storage catalogue；
First detection unit 502, for periodically detecting the be stored in the dsc data storage disk first storage
The access times of data；
First migration units 503, are less than default first temperature threshold for the access times when first data storage
During value, first data storage is moved in the cold data storage disk under second layer storage catalogue.
Optionally, the system can also include：
Unit is set up, for setting up first layer storage catalogue and second layer storage catalogue；
First binding unit, for the first layer storage catalogue to be bound into dsc data storage disk；Wherein, the hot number
Include according to storage disk：SSD solid state hard discs, SAS disks or flash memory；
Second binding unit, for the second layer storage catalogue to be bound into cold data storage disk；Wherein, cold data is stated
Storage disk includes：Common hard disc.
Optionally, the system can also include：
Determining unit, for being deposited when the internal memory shared by the data stored in the cold data storage disk accounts for the cold data
When the ratio of storage disk storage capacity has exceeded default proportion threshold value, determine to meet default rule in the cold data storage disk
The second data storage then；
Backup units, for will be backed up in backup disk of second data storage under third layer storage catalogue；
Unit is deleted, for deleting the second data storage in the cold data storage disk.
Optionally, the determining unit, including：
Detection sub-unit, the access times for detecting the data storage being stored in the cold data storage disk；
First determination subelement, for by within the default period access times be zero the data storage determine
For the second data storage；
Or,
Whether judgment sub-unit, the data storage for judging to be stored in the cold data storage disk has exceeded default
The data storage of time；
Second determination subelement, for the data storage more than preset time to be defined as into the second data storage.
Optionally, the system can also include：
Second detection unit, the visit for periodically detecting the 3rd data storage being stored in cold data storage disk
Ask number of times；
Second migration units, are more than default second heat degree threshold for the access times when the 3rd data storage
When, the 3rd data storage is moved in the dsc data storage disk under first layer storage catalogue.
In the present embodiment, when receiving data to be stored, the data Cun Chudao first layers to be stored are stored
In dsc data storage disk under catalogue；Periodically detection is stored in the first data storage in the dsc data storage disk
Access times；When the access times of first data storage are less than default first heat degree threshold, described first is deposited
Store up in the cold data storage disk under Data Migration to second layer storage catalogue.By the system of the present embodiment, by different access
The data hierarchy storage of amount, not only controls the use cost of disk, and simplifies Distributed File System Data storage
Flow.
It should be noted that each embodiment in this specification is described by the way of progressive, each embodiment weight
Point explanation be all between difference with other embodiment, each embodiment identical similar part mutually referring to.
The foregoing description of the disclosed embodiments, enables professional and technical personnel in the field to realize or using the present invention.
A variety of modifications to these embodiments will be apparent for those skilled in the art, as defined herein
General Principle can be realized in other embodiments without departing from the spirit or scope of the present invention.Therefore, it is of the invention
The embodiments shown herein is not intended to be limited to, and is to fit to and principles disclosed herein and features of novelty phase one
The most wide scope caused.
Claims (10)
1. a kind of distributed file system Bedding storage method, it is characterised in that methods described includes：
When receiving data to be stored, the dsc data under the data Cun Chudao first layer storage catalogues to be stored is deposited
Store up in disk；
Periodically detection is stored in the access times of the first data storage in the dsc data storage disk；
When the access times of first data storage are less than default first heat degree threshold, first data storage is moved
Move on in the cold data storage disk under second layer storage catalogue.
2. according to the method described in claim 1, it is characterised in that also include：
Set up first layer storage catalogue and second layer storage catalogue；
The first layer storage catalogue is bound into dsc data storage disk；Wherein, the dsc data storage disk includes：SSD consolidates
State hard disk, SAS disks or flash memory；
The second layer storage catalogue is bound into cold data storage disk；Wherein, stating cold data storage disk includes：It is common hard
Disk.
3. according to the method described in claim 1, it is characterised in that also include：
When the internal memory shared by the data stored in the cold data storage disk accounts for the cold data storage disk memory capacity
When ratio has exceeded default proportion threshold value, determine to meet the second storage number of preset rules in the cold data storage disk
According to；
It will be backed up in backup disk of second data storage under third layer storage catalogue；
Delete the second data storage in the cold data storage disk.
4. method according to claim 3, it is characterised in that met in the determination cold data storage disk default
Second data storage of rule, including：
Detection is stored in the access times of the data storage in the cold data storage disk；
By within the default period access times be that zero data storage is defined as the second data storage；
Or,
Judge whether the data storage being stored in the cold data storage disk has exceeded the data storage of preset time；
Data storage more than preset time is defined as the second data storage.
5. method according to claim 3, it is characterised in that also include：
Periodically detection is stored in the access times of the 3rd data storage in cold data storage disk；
When the access times of the 3rd data storage are more than default second heat degree threshold, the 3rd data storage is moved
Move on in the dsc data storage disk under first layer storage catalogue.
6. a kind of distributed file system Bedding storage system, it is characterised in that the system includes：
First memory cell, for when receiving data to be stored, the data Cun Chudao first layers to be stored to be deposited
Store up in the dsc data storage disk under catalogue；
First detection unit, the visit for periodically detecting the first data storage being stored in the dsc data storage disk
Ask number of times；
First migration units, for when the access times of first data storage are less than default first heat degree threshold, inciting somebody to action
First data storage is moved in the cold data storage disk under second layer storage catalogue.
7. system according to claim 6, it is characterised in that also include：
Unit is set up, for setting up first layer storage catalogue and second layer storage catalogue；
First binding unit, for the first layer storage catalogue to be bound into dsc data storage disk；Wherein, the dsc data is deposited
Storage disk includes：SSD solid state hard discs, SAS disks or flash memory；
Second binding unit, for the second layer storage catalogue to be bound into cold data storage disk；Wherein, cold data storage is stated
Disk includes：Common hard disc.
8. system according to claim 6, it is characterised in that also include：
Determining unit, for accounting for the cold data storage magnetic when the internal memory shared by the data stored in the cold data storage disk
When the ratio of disk capacity has exceeded default proportion threshold value, determine to meet preset rules in the cold data storage disk
Second data storage；
Backup units, for will be backed up in backup disk of second data storage under third layer storage catalogue；
Unit is deleted, for deleting the second data storage in the cold data storage disk.
9. system according to claim 8, it is characterised in that the determining unit, including：
Detection sub-unit, the access times for detecting the data storage being stored in the cold data storage disk；
First determination subelement, for by within the default period access times be that zero data storage is defined as
Two data storages；
Or,
Whether judgment sub-unit, the data storage for judging to be stored in the cold data storage disk has exceeded preset time
Data storage；
Second determination subelement, for the data storage more than preset time to be defined as into the second data storage.
10. system according to claim 8, it is characterised in that also include：
Second detection unit, the access time for periodically detecting the 3rd data storage being stored in cold data storage disk
Number；
Second migration units, for when the access times of the 3rd data storage are more than default second heat degree threshold, inciting somebody to action
3rd data storage is moved in the dsc data storage disk under first layer storage catalogue.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710385437.5A CN107193500A (en) | 2017-05-26 | 2017-05-26 | A kind of distributed file system Bedding storage method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710385437.5A CN107193500A (en) | 2017-05-26 | 2017-05-26 | A kind of distributed file system Bedding storage method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107193500A true CN107193500A (en) | 2017-09-22 |
Family
ID=59875147
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710385437.5A Pending CN107193500A (en) | 2017-05-26 | 2017-05-26 | A kind of distributed file system Bedding storage method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107193500A (en) |
Cited By (30)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107103203A (en) * | 2017-05-22 | 2017-08-29 | 郑州云海信息技术有限公司 | A kind of storage AUTOMATIC ZONING system and method applied to PACS |
| CN107817950A (en) * | 2017-10-31 | 2018-03-20 | 新华三技术有限公司 | A kind of data processing method and device |
| CN108363553A (en) * | 2018-01-31 | 2018-08-03 | 北京兰云科技有限公司 | A kind of data processing method, apparatus and system |
| CN108363813A (en) * | 2018-03-15 | 2018-08-03 | 北京小度信息科技有限公司 | Date storage method, device and system |
| CN108388406A (en) * | 2018-01-10 | 2018-08-10 | 链家网（北京）科技有限公司 | Data processing method and device |
| CN108519862A (en) * | 2018-03-30 | 2018-09-11 | 百度在线网络技术（北京）有限公司 | Storage method, device, system and the storage medium of block catenary system |
| CN108829350A (en) * | 2018-05-31 | 2018-11-16 | 阿里巴巴集团控股有限公司 | Data migration method and device based on block chain |
| CN109144414A (en) * | 2018-07-27 | 2019-01-04 | 阿里巴巴集团控股有限公司 | The multistage storage method and device of block chain data |
| CN109274752A (en) * | 2018-10-10 | 2019-01-25 | 腾讯科技（深圳）有限公司 | The access method and device, electronic equipment, storage medium of block chain data |
| CN109284069A (en) * | 2018-08-23 | 2019-01-29 | 郑州云海信息技术有限公司 | A kind of distributed memory system and method for storing Backup Data |
| CN109324759A (en) * | 2018-09-17 | 2019-02-12 | 山东浪潮云投信息科技有限公司 | The processing terminal of big data platform, the method read data and write data |
| CN109634520A (en) * | 2018-11-29 | 2019-04-16 | 南京航空航天大学 | A kind of storage system based on HDFS CD server |
| CN110045925A (en) * | 2019-03-05 | 2019-07-23 | 北京字节跳动网络技术有限公司 | Data processing method, device, electronic equipment and computer readable storage medium |
| CN110309229A (en) * | 2019-05-09 | 2019-10-08 | 北京极数云舟科技有限公司 | The data processing method and distributed system of distributed system |
| CN110308862A (en) * | 2018-03-20 | 2019-10-08 | 纬颖科技服务股份有限公司 | Management method and its associated reservoir system |
| CN110532234A (en) * | 2019-08-30 | 2019-12-03 | 杭州迪普科技股份有限公司 | A kind of storage method and device of vulnerability scanning data |
| CN110543279A (en) * | 2018-05-29 | 2019-12-06 | 杭州海康威视数字技术股份有限公司 | Data storage and processing method, device and system |
| CN110673785A (en) * | 2019-08-23 | 2020-01-10 | 上海科技发展有限公司 | Data storage method and system |
| CN110727406A (en) * | 2019-10-10 | 2020-01-24 | 深圳力维智联技术有限公司 | Data storage scheduling method and device |
| CN110781129A (en) * | 2019-09-12 | 2020-02-11 | 苏州浪潮智能科技有限公司 | Resource scheduling method, device and medium in FPGA heterogeneous accelerator card cluster |
| US10592491B2 (en) | 2015-05-04 | 2020-03-17 | Alibaba Group Holding Limited | Distributed data processing |
| CN110908608A (en) * | 2019-11-22 | 2020-03-24 | 苏州浪潮智能科技有限公司 | Storage space saving method and system |
| CN111125047A (en) * | 2019-12-06 | 2020-05-08 | 中盈优创资讯科技有限公司 | Cold and hot data directory identification method and device |
| CN111124304A (en) * | 2019-12-19 | 2020-05-08 | 北京浪潮数据技术有限公司 | Data migration method and device, electronic equipment and storage medium |
| CN111625203A (en) * | 2020-07-31 | 2020-09-04 | 苏州浪潮智能科技有限公司 | Method, system, device and medium for hierarchical storage |
| CN112306901A (en) * | 2020-11-16 | 2021-02-02 | 新华三大数据技术有限公司 | Disk refreshing method and device based on layered storage system, electronic equipment and medium |
| CN113835630A (en) * | 2021-09-15 | 2021-12-24 | 联泰集群（北京）科技有限责任公司 | Data storage method, device, data server, storage medium and system |
| CN114726791A (en) * | 2022-03-18 | 2022-07-08 | 中国建设银行股份有限公司 | Method and device for determining flow control threshold, electronic equipment and computer storage medium |
| WO2022156649A1 (en) * | 2021-01-19 | 2022-07-28 | 华为技术有限公司 | Capacity adjustment method and related device |
| CN110727406B (en) * | 2019-10-10 | 2024-05-17 | 深圳力维智联技术有限公司 | Data storage scheduling method and device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104679661A (en) * | 2013-11-27 | 2015-06-03 | 阿里巴巴集团控股有限公司 | Hybrid memory control method and hybrid memory system |
| CN105892952A (en) * | 2016-04-22 | 2016-08-24 | 深圳市深信服电子科技有限公司 | Hyper-converged system and longitudinal extension method thereof |
-
2017
- 2017-05-26 CN CN201710385437.5A patent/CN107193500A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104679661A (en) * | 2013-11-27 | 2015-06-03 | 阿里巴巴集团控股有限公司 | Hybrid memory control method and hybrid memory system |
| CN105892952A (en) * | 2016-04-22 | 2016-08-24 | 深圳市深信服电子科技有限公司 | Hyper-converged system and longitudinal extension method thereof |
Cited By (44)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10592491B2 (en) | 2015-05-04 | 2020-03-17 | Alibaba Group Holding Limited | Distributed data processing |
| US10872070B2 (en) | 2015-05-04 | 2020-12-22 | Advanced New Technologies Co., Ltd. | Distributed data processing |
| CN107103203A (en) * | 2017-05-22 | 2017-08-29 | 郑州云海信息技术有限公司 | A kind of storage AUTOMATIC ZONING system and method applied to PACS |
| CN107817950A (en) * | 2017-10-31 | 2018-03-20 | 新华三技术有限公司 | A kind of data processing method and device |
| CN107817950B (en) * | 2017-10-31 | 2021-07-23 | 新华三技术有限公司 | Data processing method and device |
| CN108388406A (en) * | 2018-01-10 | 2018-08-10 | 链家网（北京）科技有限公司 | Data processing method and device |
| CN108363553A (en) * | 2018-01-31 | 2018-08-03 | 北京兰云科技有限公司 | A kind of data processing method, apparatus and system |
| CN108363813B (en) * | 2018-03-15 | 2020-06-02 | 北京星选科技有限公司 | Data storage method, device and system |
| CN108363813A (en) * | 2018-03-15 | 2018-08-03 | 北京小度信息科技有限公司 | Date storage method, device and system |
| CN110308862A (en) * | 2018-03-20 | 2019-10-08 | 纬颖科技服务股份有限公司 | Management method and its associated reservoir system |
| CN110308862B (en) * | 2018-03-20 | 2023-05-05 | 纬颖科技服务股份有限公司 | Management method and related storage system thereof |
| CN108519862A (en) * | 2018-03-30 | 2018-09-11 | 百度在线网络技术（北京）有限公司 | Storage method, device, system and the storage medium of block catenary system |
| CN110543279A (en) * | 2018-05-29 | 2019-12-06 | 杭州海康威视数字技术股份有限公司 | Data storage and processing method, device and system |
| CN108829350A (en) * | 2018-05-31 | 2018-11-16 | 阿里巴巴集团控股有限公司 | Data migration method and device based on block chain |
| TWI710978B (en) * | 2018-05-31 | 2020-11-21 | 開曼群島商創新先進技術有限公司 | Data migration method and device based on blockchain |
| US10649953B2 (en) | 2018-05-31 | 2020-05-12 | Alibaba Group Holding Limited | Blockchain-based data migration method and apparatus |
| TWI727226B (en) * | 2018-07-27 | 2021-05-11 | 開曼群島商創新先進技術有限公司 | Multi-level storage method and device for blockchain data |
| WO2020023843A1 (en) * | 2018-07-27 | 2020-01-30 | Alibaba Group Holding Limited | Multi-level storage method and apparatus for blockchain data |
| CN111897498A (en) * | 2018-07-27 | 2020-11-06 | 创新先进技术有限公司 | Multi-level storage method and device for block chain data |
| CN109144414A (en) * | 2018-07-27 | 2019-01-04 | 阿里巴巴集团控股有限公司 | The multistage storage method and device of block chain data |
| CN109144414B (en) * | 2018-07-27 | 2020-07-07 | 阿里巴巴集团控股有限公司 | Multi-level storage method and device for block chain data |
| CN109284069A (en) * | 2018-08-23 | 2019-01-29 | 郑州云海信息技术有限公司 | A kind of distributed memory system and method for storing Backup Data |
| CN109324759A (en) * | 2018-09-17 | 2019-02-12 | 山东浪潮云投信息科技有限公司 | The processing terminal of big data platform, the method read data and write data |
| CN109274752A (en) * | 2018-10-10 | 2019-01-25 | 腾讯科技（深圳）有限公司 | The access method and device, electronic equipment, storage medium of block chain data |
| CN109634520A (en) * | 2018-11-29 | 2019-04-16 | 南京航空航天大学 | A kind of storage system based on HDFS CD server |
| CN109634520B (en) * | 2018-11-29 | 2021-12-07 | 南京航空航天大学 | Storage system based on HDFS optical disc library |
| CN110045925A (en) * | 2019-03-05 | 2019-07-23 | 北京字节跳动网络技术有限公司 | Data processing method, device, electronic equipment and computer readable storage medium |
| CN110309229A (en) * | 2019-05-09 | 2019-10-08 | 北京极数云舟科技有限公司 | The data processing method and distributed system of distributed system |
| CN110673785A (en) * | 2019-08-23 | 2020-01-10 | 上海科技发展有限公司 | Data storage method and system |
| CN110532234A (en) * | 2019-08-30 | 2019-12-03 | 杭州迪普科技股份有限公司 | A kind of storage method and device of vulnerability scanning data |
| CN110781129A (en) * | 2019-09-12 | 2020-02-11 | 苏州浪潮智能科技有限公司 | Resource scheduling method, device and medium in FPGA heterogeneous accelerator card cluster |
| CN110781129B (en) * | 2019-09-12 | 2022-02-22 | 苏州浪潮智能科技有限公司 | Resource scheduling method, device and medium in FPGA heterogeneous accelerator card cluster |
| CN110727406B (en) * | 2019-10-10 | 2024-05-17 | 深圳力维智联技术有限公司 | Data storage scheduling method and device |
| CN110727406A (en) * | 2019-10-10 | 2020-01-24 | 深圳力维智联技术有限公司 | Data storage scheduling method and device |
| CN110908608A (en) * | 2019-11-22 | 2020-03-24 | 苏州浪潮智能科技有限公司 | Storage space saving method and system |
| CN111125047A (en) * | 2019-12-06 | 2020-05-08 | 中盈优创资讯科技有限公司 | Cold and hot data directory identification method and device |
| CN111125047B (en) * | 2019-12-06 | 2024-03-12 | 中盈优创资讯科技有限公司 | Cold and hot data catalog identification method and device |
| CN111124304A (en) * | 2019-12-19 | 2020-05-08 | 北京浪潮数据技术有限公司 | Data migration method and device, electronic equipment and storage medium |
| CN111625203A (en) * | 2020-07-31 | 2020-09-04 | 苏州浪潮智能科技有限公司 | Method, system, device and medium for hierarchical storage |
| CN112306901B (en) * | 2020-11-16 | 2022-07-29 | 新华三大数据技术有限公司 | Disk refreshing method and device based on layered storage system, electronic equipment and medium |
| CN112306901A (en) * | 2020-11-16 | 2021-02-02 | 新华三大数据技术有限公司 | Disk refreshing method and device based on layered storage system, electronic equipment and medium |
| WO2022156649A1 (en) * | 2021-01-19 | 2022-07-28 | 华为技术有限公司 | Capacity adjustment method and related device |
| CN113835630A (en) * | 2021-09-15 | 2021-12-24 | 联泰集群（北京）科技有限责任公司 | Data storage method, device, data server, storage medium and system |
| CN114726791A (en) * | 2022-03-18 | 2022-07-08 | 中国建设银行股份有限公司 | Method and device for determining flow control threshold, electronic equipment and computer storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107193500A (en) | A kind of distributed file system Bedding storage method and system | |
| US7831793B2 (en) | Data storage system including unique block pool manager and applications in tiered storage | |
| US8095852B2 (en) | Data recorder | |
| US20140281214A1 (en) | Total quotas for data storage system | |
| US20150355863A1 (en) | Distributing capacity slices across storage system nodes | |
| CN107526689B (en) | Read cache management | |
| US20120166712A1 (en) | Hot sheet upgrade facility | |
| CN106164884A (en) | There is the data storage device of embedded software | |
| US20110145528A1 (en) | Storage apparatus and its control method | |
| KR20130009980A (en) | Multiple cascaded backup process | |
| WO2013157032A1 (en) | Storage subsystem and data management method of storage subsystem | |
| CN107092442A (en) | Storage system resources distribution method and device | |
| US20130254509A1 (en) | Consideration of adjacent track interference and wide area adjacent track erasure during disk defragmentation | |
| US20200341684A1 (en) | Managing a raid group that uses storage devices of different types that provide different data storage characteristics | |
| KR101731606B1 (en) | Recovering method of data in cow-based file system using digital forensic technology | |
| CN108874321A (en) | A kind of raid-array data reconstruction method, device and equipment | |
| US9866634B1 (en) | Managing and accessing data storage systems | |
| CN107092443B (en) | Data migration method and device | |
| US10621059B2 (en) | Site recovery solution in a multi-tier storage environment | |
| US20110119509A1 (en) | Storage system having power saving function | |
| CN107783720A (en) | A kind of data balancing method and storage device | |
| CN101517543B (en) | Error correction for disk storage media | |
| US10452494B1 (en) | Performing storage object recovery | |
| US20220137813A1 (en) | Rebuilding space accounting counters in mapping layer of storage appliances | |
| US11151090B2 (en) | Snapshot management based on data churn delta |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170922 |
|
| RJ01 | Rejection of invention patent application after publication |