CN107391301A - Data managing method, device, computing device and the storage medium of storage system - Google Patents
Data managing method, device, computing device and the storage medium of storage system Download PDFInfo
- Publication number
- CN107391301A CN107391301A CN201710703651.0A CN201710703651A CN107391301A CN 107391301 A CN107391301 A CN 107391301A CN 201710703651 A CN201710703651 A CN 201710703651A CN 107391301 A CN107391301 A CN 107391301A
- Authority
- CN
- China
- Prior art keywords
- data
- hard disk
- storage system
- failure
- access frequency
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3037—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a memory, e.g. virtual memory, cache
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0647—Migration mechanisms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Quality & Reliability (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a kind of data managing method of storage system, device, computing device and computer-readable storage medium, data storage mechanism operation of the storage system based on correcting and eleting codes, and expansion instruction set is integrated with, wherein, the data managing method of storage system includes：Obtain hard disk operational factor corresponding with the hard disk in storage system；Hard disk operational factor is analyzed, obtains hard disk failure prediction result；According to hard disk failure prediction result, to predicting that the hard disk of failure carries out Data Migration；This method also includes：For the data stored in storage system, the access frequency of data is calculated；According to the access frequency of data, the data type of data is determined, and by Data Migration to memory block corresponding with data type.According to technical scheme provided by the invention, data storage cost is reduced, significantly reduces CPU consumption, ensure that the data in the hard disk of prediction failure are not lost, improve data reliability.
Description
Technical field
The present invention relates to Internet technical field, and in particular to a kind of data managing method of storage system, device, calculating
Equipment and computer-readable storage medium.
Background technology
With the continuous development of Internet technology, data storage technology is also developed rapidly, and many is occurred and is based on
The storage system of different modes, for example, HDFS (Hadoop distributed file systems, Hadoop Distributed File
) and GFS (Google file system, Google File System) etc. System.By taking HDFS as an example, in most cases,
HDFS copy coefficient is 3, specifically, a copy is stored on local frame node, a copy is stored in same
On another node of frame, last copy is placed on the node of different frames.Because HDFS needs to store 3 copies,
Its data storage cost is higher.In addition, when the hard disk in storage system breaks down, the data in the hard disk to break down are very
It is easily lost, so as to it is difficult to ensure that data reliability.
Therefore, there is the problem of data storage cost is higher, data reliability is relatively low for existing storage system.
The content of the invention
In view of the above problems, it is proposed that the present invention so as to provide one kind overcome above mentioned problem or at least in part solve on
State data managing method, device, computing device and the computer-readable storage medium of the storage system of problem.
According to an aspect of the invention, there is provided a kind of data managing method of storage system, storage system is based on entangling
The data storage mechanism operation of code is deleted, and is integrated with expansion instruction set, this method includes：
Obtain hard disk operational factor corresponding with the hard disk in storage system；
Hard disk operational factor is analyzed, obtains hard disk failure prediction result；
According to hard disk failure prediction result, to predicting that the hard disk of failure carries out Data Migration；
This method also includes：
For the data stored in storage system, the access frequency of data is calculated；
According to the access frequency of data, the data type of data is determined, and by Data Migration to corresponding with data type
In memory block.
According to another aspect of the present invention, there is provided a kind of data administrator of storage system, storage system are based on entangling
The data storage mechanism operation of code is deleted, and is integrated with expansion instruction set, the device includes：
Acquisition module, suitable for obtaining hard disk operational factor corresponding with the hard disk in storage system；
Analysis module, suitable for analyzing hard disk operational factor, obtain hard disk failure prediction result；
Data Migration module, suitable for according to hard disk failure prediction result, Data Migration is carried out to the hard disk of prediction failure；
The device also includes：
Computing module, suitable for the data for being stored in storage system, calculate the access frequency of data；
Determining module, suitable for the access frequency according to data, determine the data type of data；
Data Migration module is further adapted for：By in Data Migration to memory block corresponding with data type.
According to another aspect of the invention, there is provided a kind of computing device, including：Processor, memory, communication interface and
Communication bus, processor, memory and communication interface complete mutual communication by communication bus；
Memory is used to deposit an at least executable instruction, and executable instruction makes the number of the above-mentioned storage system of computing device
According to operation corresponding to management method.
In accordance with a further aspect of the present invention, there is provided a kind of computer-readable storage medium, be stored with least one in storage medium
Executable instruction, executable instruction make computing device be operated as corresponding to the data managing method of above-mentioned storage system.
According to technical scheme provided by the invention, acquired hard disk operational factor is analyzed, obtains hard disk failure
Prediction result, according to hard disk failure prediction result, to predicting that the hard disk of failure carries out Data Migration, in addition, this method also directed to
The data stored in storage system, the access frequency of data is calculated, according to the access frequency of data, determine the data class of data
Type, and by Data Migration to memory block corresponding with data type.Technical scheme provided by the invention utilizes the number of correcting and eleting codes
According to memory mechanism and the mode of Integration of Extended instruction set, data storage cost is significantly reduced, CPU is significantly reduced and disappears
Consumption；And hard disk failure prediction can be carried out according to hard disk operational factor, automatically be moved to predicting that the hard disk of failure carries out data
Move, so as to ensure that the data in the hard disk of prediction failure are not lost, improve data reliability；In addition it is possible to according to number
According to access frequency, realize dynamic migration and the management of data, reduce further storage system synthesis data storage into
This.
Described above is only the general introduction of technical solution of the present invention, in order to better understand the technological means of the present invention,
And can be practiced according to the content of specification, and in order to allow above and other objects of the present invention, feature and advantage can
Become apparent, below especially exemplified by the embodiment of the present invention.
Brief description of the drawings
By reading the detailed description of hereafter preferred embodiment, it is various other the advantages of and benefit it is common for this area
Technical staff will be clear understanding.Accompanying drawing is only used for showing the purpose of preferred embodiment, and is not considered as to the present invention
Limitation.And in whole accompanying drawing, identical part is denoted by the same reference numerals.In the accompanying drawings：
Fig. 1 shows the schematic flow sheet of the data managing method of storage system according to an embodiment of the invention；
Fig. 2 shows the schematic flow sheet of the data managing method of storage system in accordance with another embodiment of the present invention；
Fig. 3 shows the structured flowchart of the data administrator of storage system according to an embodiment of the invention；
Fig. 4 shows the structured flowchart of the data administrator of storage system in accordance with another embodiment of the present invention；
Fig. 5 shows a kind of structural representation of computing device according to embodiments of the present invention.
Embodiment
The exemplary embodiment of the disclosure is more fully described below with reference to accompanying drawings.Although the disclosure is shown in accompanying drawing
Exemplary embodiment, it being understood, however, that may be realized in various forms the disclosure without should be by embodiments set forth here
Limited.On the contrary, these embodiments are provided to facilitate a more thoroughly understanding of the present invention, and can be by the scope of the present disclosure
Completely it is communicated to those skilled in the art.
In the present invention, storage system is that the data storage mechanism based on EC (correcting and eleting codes, Erasure Coding) is run
, using this memory mechanism, data storage cost can be effectively reduced.Wherein, correcting and eleting codes are a kind of volumes of data protection
Code method, it obtains m block redundant datas into k block number evidences, then based on k block numbers first by original data division according to redundant computation, so
It will be stored afterwards in this k+m blocks data sharing to cluster.When any of which m blocks corrupt data or loss, storage system
Original k block number evidences can be recovered by restructing algorithm, so as to obtain initial data.Specifically, it is of the prior art
HDFS typically 3 copies of storage, then it is believed that its corresponding data storage overhead is 3, and the present invention depositing using correcting and eleting codes
Storage mechanism, when it is 4 that k, which is 10, m, the data storage overhead of the storage system is 1.4.Compared with existing HDFS, the present invention carries
The storage system of confession utilizes the data storage mechanism of correcting and eleting codes, significantly reduces data storage cost.
In addition, in order to reduce CPU consumption, the present invention is also integrated with expansion instruction set within the storage system, wherein, extension refers to
The bit wide for the instruction concentrated is made to be more than the bit wide of the instruction that cpu instruction is concentrated in the prior art.In the prior art, it is conventional
The bit wide of instruction that cpu instruction is concentrated is usually 64, then the present invention integrate extended instruction concentration within the storage system
Instruction bit wide can be 256.Those skilled in the art can be according to the specific expansion instruction set of selection be actually needed, herein not
Limit.For example, expansion instruction set can be AUX expansion instruction sets etc..Storage system provided by the invention utilizes integrated expansion
The bit wide of instruction set is opened up, significantly reduces CPU consumption.
Fig. 1 shows the schematic flow sheet of the data managing method of storage system according to an embodiment of the invention, its
In, data storage mechanism operation of the storage system based on correcting and eleting codes, and it is integrated with expansion instruction set.As shown in figure 1, this method bag
Include following steps：
Step S100, obtain hard disk operational factor corresponding with the hard disk in storage system.
Specifically, storage system includes multiple hard disks for data storage, in the step s 100, based on SMART (certainly
Dynamic detection and analysis and reporting techniques, Self-Monitoring Analysis and Reporting Technology) obtain with
Hard disk operational factor corresponding to hard disk in storage system.SMART can be to the magnetic head unit of hard disk, hard disk temperature, disc surface
Dielectric material, motor and its drive system, hard drive internal circuit etc. are monitored, and obtain that the hard of hard disk running status can be reflected
Disk operational factor.
Step S101, hard disk operational factor is analyzed, obtain hard disk failure prediction result.
After hard disk operational factor is obtained, using preset algorithm, hard disk operational factor is analyzed, obtained hard
Disk failure predication result.Preset algorithm is that the hard disk operation sample data based on magnanimity is trained and analyzed and determines, profit
There is higher predictablity rate with the hard disk failure prediction result obtained by preset algorithm, can be relatively accurately to hard disk
Failure situation is predicted.Specifically, hard disk failure prediction result may include the hard disk identification code, correspondingly for predicting the hard disk of failure
Prediction failure time of origin and the information such as failure cause.Wherein, the hard disk for predicting failure is that prediction can break down
Hard disk, hard disk identification code can be hard disk ID.
Step S102, according to hard disk failure prediction result, to predicting that the hard disk of failure carries out Data Migration.
Understood according to hard disk failure prediction result, which specific hard disk prediction can break down time in which, in order to protect
Card data are not lost, according to hard disk failure prediction result, in time to predicting that the hard disk of failure carries out Data Migration, so as to improve
Data reliability.For example, understood according to hard disk failure prediction result, hard disk A is the hard disk of prediction failure, and corresponding prediction
The time of origin of failure is after 7 days, then illustrate that hard disk A has very big probability to break down after 7 days, then automatically will be hard
Data Migration in disk A is into storage system in other hard disks in normal operating condition or in alternate hard.
Step S103, for the data stored in storage system, calculate the access frequency of data.
Substantial amounts of data are stored in storage system, wherein, some data are that user often accesses in the recent period, and some are counted
According to being access that user had not accessed for a long time or seldom, in order to subsequently be made a distinction to these data, in step s 103, need
Calculate the access frequency of these data.Specifically, data can be calculated in prefixed time interval according to record or daily record etc. is accessed
Interior access frequency.Those skilled in the art can be configured according to being actually needed to prefixed time interval, not limit herein.
Step S104, according to the access frequency of data, determine the data type of data, and by Data Migration to and data class
In memory block corresponding to type.
After the access frequency of data is calculated, according to the access frequency of data, the data type of data is determined, is had
Body, the data type of data may include dsc data type, warm data type and cold data type.Every kind of data type is all set
There is corresponding memory block, and the cost of different types of memory block is different, then according to the data type of data, automatically by number
According to migrating into corresponding memory block, it is achieved thereby that the dynamic migration of data and management, the number of the synthesis of storage system is reduced
According to carrying cost.
In the present embodiment, it is to first carry out step S100 to step S102, it is rear to perform step S103 and step S104.But this
Invention is not defined to the specific execution sequence of step S103 and step S104 in the method.For example, it can also first carry out step
Rapid S103 and step S104, then performs step S100 to step S102 again.
The data managing method of the storage system provided according to the present embodiment, divides acquired hard disk operational factor
Analysis, hard disk failure prediction result is obtained, according to hard disk failure prediction result, the hard disk progress Data Migration to predicting failure, separately
Outside, this method calculates the access frequency of data also directed to the data stored in storage system, according to the access frequency of data,
The data type of data is determined, and by Data Migration to memory block corresponding with data type.Technical side provided by the invention
Case is significantly reduced data storage cost, had using the data storage mechanism of correcting and eleting codes and the mode of Integration of Extended instruction set
Reduce to effect CPU consumption；And hard disk failure prediction can be carried out according to hard disk operational factor, automatically to prediction failure
Hard disk carries out Data Migration, so as to ensure that the data in the hard disk of prediction failure are not lost, improves data reliability；Separately
Outside, additionally it is possible to according to the access frequency of data, realize dynamic migration and the management of data, reduce further the comprehensive of storage system
The data storage cost of conjunction.
Fig. 2 shows the schematic flow sheet of the data managing method of storage system in accordance with another embodiment of the present invention,
Wherein, data storage mechanism operation of the storage system based on correcting and eleting codes, and it is integrated with expansion instruction set.As shown in Fig. 2 this method
Comprise the following steps：
Step S200, obtain hard disk operational factor corresponding with the hard disk in storage system.
Hard disk operational factor corresponding with the hard disk in storage system is obtained based on SMART.Wherein, hard disk operational factor bag
Include but be not limited to the one or more of following parameter：Remapping sector count, current sector number to be mapped, off line can not correct
Sector count, access check errors rates, bottom data read error rate, disk read-write flux performance parameter, start and stop count, are soft
Part read error rate and moment of torsion amplification count.
Specifically, remap sector count be used for represent be remapped the quantity of sector.When certain sector of hard disk is held
Continuous when there are read/write/check errors, the physical address of this sector can be added defect table by hard disk firmware program, by the address
It is re-introduced into the alternate sector advanced for and shifts data therein in the lump, this is referred to as remapping.It is current to be mapped
Sector number is used to represent unstable sector number, that is, waits the quantity for the sector (sector being alternatively referred to as suspended) being mapped.
The sector count that off line can not correct is used to represent the uncorrectable error sum occurred during read-write sector.Access check errors
Rate is used for the number for the data line transfer mistake that expression is found by interface CRC.Bottom data read error rate is used
The wrong number occurred when representing that magnetic head reads data from magnetic disk surface.Disk read-write flux performance parameter is used to represent hard
The read-write flux performance of disk.Start and stop count the number for representing hard disk spindle electric motor starting/stopping.Software read error rate is used
In counting of the expression report to the not calibrated read error of operating system.Moment of torsion amplification, which counts, to be used to represent that spindle motor tries
Figure improves moment of torsion to compensate the number of disc rotation speed change.When main shaft bearing has problem, spindle motor can attempt increase and drive
Power makes disc is stable to rotate.
Step S201, according to preset algorithm and default hard disk operational factor threshold value, hard disk operational factor is analyzed, obtained
To hard disk failure prediction result.
Wherein, default hard disk operational factor threshold value is those skilled in the art or hard disk manufacture manufacturer according to the specific of hard disk
Situation is set in advance.After hard disk operational factor is obtained, in step s 201, transported according to preset algorithm and default hard disk
Row parameter threshold, hard disk operational factor is analyzed, obtain hard disk failure prediction result.Because preset algorithm is to be based on magnanimity
Hard disk operation sample data be trained and analyze and determine, therefore utilize the hard disk failure prediction obtained by preset algorithm
As a result there is higher predictablity rate, relatively accurately the failure situation of hard disk can be predicted.
Step S202, according to hard disk failure prediction result, to predicting that the hard disk of failure carries out Data Migration.
In order to ensure that data are not lost, according to hard disk failure prediction result, in time to predicting that the hard disk of failure enters line number
According to migration, so as to improve data reliability.Specifically, can be by the Data Migration in the hard disk for predicting failure into storage system
In hard disk of the others in normal operating condition or in alternate hard.
Step S203, the hard disk for predicting failure is added in failure blacklist.
Wherein, failure blacklist is used for the hard disk for recording the hard disk of prediction failure and breaking down.Specifically, will can predict
The hard disk identification code of the hard disk of failure is added in failure blacklist, so as to remove the hard disk from storage system, makes this hard
Disk can not provide service again, and operation maintenance personnel just can be reported for repairment easily and removable disk processing according to failure blacklist.
Step S204, for the data stored in storage system, calculate the access frequency of data.
Specifically, for each data stored in storage system, the number can be calculated according to record or daily record etc. is accessed
According to access frequency within a preset time interval.Those skilled in the art can set according to being actually needed to prefixed time interval
Put, do not limit herein.
Step S205, judges whether the access frequency of data is less than the first predeterminated frequency threshold value；If so, then perform step
S206；If it is not, then perform step S207.
Wherein, those skilled in the art can be configured according to being actually needed to the first predeterminated frequency threshold value, not do herein
Limit.If it is determined that the access frequency for obtaining data is less than the first predeterminated frequency threshold value, then step S206 is performed；If it is determined that
Access frequency to data is more than or equal to the first predeterminated frequency threshold value, then performs step S207.
Step S206, the data type for determining data is cold data type, and by Data Migration into tape storage area.
In the case where the access frequency for judging to obtain data through step S205 is less than the first predeterminated frequency threshold value, number is determined
According to data type be cold data type, then by data type for cold data type Data Migration to tape (tape) store
Qu Zhong.
Step S207, judges whether the access frequency of data is more than the second predeterminated frequency threshold value；If so, then perform step
S208；If it is not, then perform step S209.
In the case where the access frequency for judging to obtain data through step S205 is more than or equal to the first predeterminated frequency threshold value,
Judge whether the access frequency of data is more than the second predeterminated frequency threshold value.Wherein, it is default to be less than second for the first predeterminated frequency threshold value
Frequency threshold.Those skilled in the art can be configured according to being actually needed to the second predeterminated frequency threshold value, not limit herein.
If it is determined that the access frequency for obtaining data is more than the second predeterminated frequency threshold value, then step S208 is performed；If it is determined that counted
According to access frequency be less than or equal to the second predeterminated frequency threshold value, then perform step S209.
Step S208, the data type for determining data is dsc data type, and by Data Migration into flash memory storage area.
In the case where the access frequency for judging to obtain data through step S208 is more than the second predeterminated frequency threshold value, number is determined
According to data type be dsc data type, then by Data Migration to flash memory (flash) memory block.
Step S209, the data type for determining data is warm data type, and by Data Migration into disk storage area.
In the case where the access frequency for judging to obtain data through step S208 is less than or equal to the second predeterminated frequency threshold value,
The data type for determining data is warm data type, then by Data Migration to disk (disk) memory block.Wherein, flash memory is deposited
The cost of storage area is higher than the cost of disk storage area, and the response speed in flash memory storage area is faster than the response speed of disk storage area
Degree；The cost of disk storage area is higher than the cost in tape storage area, and the response speed of disk storage area is faster than tape storage area
Response speed.According to technical scheme provided by the invention, using cost is higher, the faster flash memory storage area of response speed stores
The data of dsc data type, and the data for the tape storage area storage cold data type that cost is relatively low, response speed is slower are utilized,
So as to not only ensure that the Whole Response speed of storage system, and also reduce the data storage of the synthesis of storage system into
This.
In the present embodiment, it is to first carry out step S200 to step S203, it is rear to perform step S204 to step S209.But this
Invention is not defined to the specific execution sequences of step S204 to step S209 in the method.For example, it can also first carry out step
Rapid S204 to step S209, then performs step S200 to step S203 again.
The data managing method of the storage system provided according to the present embodiment, utilizes the data storage mechanism and collection of correcting and eleting codes
Into the mode of expansion instruction set, data storage cost is significantly reduced, significantly reduces CPU consumption；And according to hard disk
Operational factor carries out hard disk failure prediction, has higher predictablity rate, can be automatically to predicting that the hard disk of failure is carried out
Data Migration, so as to ensure that the data in the hard disk of prediction failure are not lost, improve data reliability；In addition it is possible to
Dynamic migration and the management of cold and hot data are realized, the data storage cost of the synthesis of storage system is reduce further, optimizes
The data management mode of storage system.
Fig. 3 shows the structured flowchart of the data administrator of storage system according to an embodiment of the invention, wherein,
Data storage mechanism operation of the storage system based on correcting and eleting codes, and it is integrated with expansion instruction set.As shown in figure 3, the device includes：
Acquisition module 310, analysis module 320, Data Migration module 330, computing module 340 and determining module 350.
Acquisition module 310 is suitable to：Obtain hard disk operational factor corresponding with the hard disk in storage system.
Specifically, acquisition module 310 obtains hard disk operational factor corresponding with the hard disk in storage system based on SMART.
Analysis module 320 is suitable to：Hard disk operational factor is analyzed, obtains hard disk failure prediction result.
Wherein, hard disk failure prediction result may include hard disk identification code, the corresponding prediction failure for predicting the hard disk of failure
Time of origin and the information such as failure cause.
Data Migration module 330 is suitable to：According to hard disk failure prediction result, the hard disk to predicting failure carries out data and moved
Move.
Computing module 340 is suitable to：For the data stored in storage system, the access frequency of data is calculated.
Determining module 350 is suitable to：According to the access frequency of data, the data type of data is determined.
Data Migration module 330 is further adapted for：By in Data Migration to memory block corresponding with data type.
Specifically, the data type of data may include dsc data type, warm data type and cold data type.Every kind of data
Type is both provided with corresponding memory block, the data class of the data according to determined by determining module 350 of Data Migration module 330
Type, by Data Migration to memory block corresponding with data type.
The data administrator of the storage system provided according to the present embodiment, analysis module is to hard acquired in acquisition module
Disk operational factor is analyzed, and obtains hard disk failure prediction result, Data Migration module is according to hard disk failure prediction result, to pre-
The hard disk for surveying failure carries out Data Migration, and computing module is directed to the data stored in storage system, calculates the access frequency of data
Rate, Data Migration module determine the data type of data always according to the access frequency of data, and by Data Migration to and data class
In memory block corresponding to type.Technical scheme provided by the invention utilizes the data storage mechanism and Integration of Extended instruction set of correcting and eleting codes
Mode, significantly reduce data storage cost, significantly reduce CPU consumption；And can be according to hard disk operational factor
Hard disk failure prediction is carried out, automatically to predicting that the hard disk of failure carries out Data Migration, so as to ensure that the hard disk of prediction failure
In data do not lose, improve data reliability；In addition it is possible to according to the access frequency of data, the dynamic of data is realized
Migration and management, reduce further the data storage cost of the synthesis of storage system.
Fig. 4 shows the structured flowchart of the data administrator of storage system in accordance with another embodiment of the present invention, its
In, data storage mechanism operation of the storage system based on correcting and eleting codes, and it is integrated with expansion instruction set.As shown in figure 4, the device bag
Include：Acquisition module 410, analysis module 420, Data Migration module 430, blacklist add module 440, computing module 450 and really
Cover half block 460.
Acquisition module 410 is suitable to：Obtain hard disk operational factor corresponding with the hard disk in storage system.
Wherein, hard disk operational factor includes but is not limited to the one or more of following parameter：Remap sector count, current
Sector count that sector number to be mapped, off line can not correct, access check errors rate, bottom data read error rate, disk
Flux performance parameter, start and stop counting, software read error rate and moment of torsion amplification is read and write to count.
Analysis module 420 is suitable to：According to preset algorithm and default hard disk operational factor threshold value, hard disk operational factor is carried out
Analysis, obtains hard disk failure prediction result.
Because preset algorithm is that the hard disk operation sample data based on magnanimity is trained and analyzed and determines, therefore point
Analysis module 420 has higher predictablity rate using the hard disk failure prediction result obtained by preset algorithm, can relatively be defined
Really the failure situation of hard disk is predicted.
Data Migration module 430 is suitable to：According to hard disk failure prediction result, the hard disk to predicting failure carries out data and moved
Move.
Blacklist add module 440 is suitable to：The hard disk for predicting failure is added in failure blacklist.
Specifically, the hard disk identification code for the hard disk for predicting failure is added to failure blacklist by blacklist add module 440
In, so as to remove the hard disk from storage system, the hard disk can not be provided service again.
Computing module 450 is suitable to：For the data stored in storage system, the access frequency of data is calculated.
Determining module 460 is suitable to：According to the access frequency of data, the data type of data is determined.
In a specific embodiment, determining module 460 includes：First judging unit 461, the and of the second judging unit 462
Determining unit 463.
First judging unit 461 is suitable to：Judge whether the access frequency of data is less than the first predeterminated frequency threshold value.
Determining unit 463 is suitable to：If the first judging unit 461 judges that the access frequency for obtaining data is less than the first default frequency
Rate threshold value, it is determined that the data type of data is cold data type.
Second judging unit 462 is suitable to：If the access frequency that the first judging unit 461 judges to obtain data is more than or equal to
First predeterminated frequency threshold value, then judge whether the access frequency of data is more than the second predeterminated frequency threshold value.Wherein, the first default frequency
Rate threshold value is less than the second predeterminated frequency threshold value.Wherein, those skilled in the art can be according to being actually needed to the first predeterminated frequency threshold
Value and the second predeterminated frequency threshold value are configured, and are not limited herein.
Determining unit 463 is further adapted for：If the second judging unit 462 judges that the access frequency for obtaining data is more than second
Predeterminated frequency threshold value, it is determined that the data type of data is dsc data type；If the second judging unit 462 judges to obtain data
Access frequency is less than or equal to the second predeterminated frequency threshold value, it is determined that the data type of data is warm data type.
Data Migration module 430 is further adapted for：By data type for cold data type Data Migration to tape storage
Qu Zhong；By data type for dsc data type Data Migration into flash memory storage area；It is warm data type by data type
Data Migration is into disk storage area.
The data administrator of the storage system provided according to the present embodiment, utilizes the data storage mechanism and collection of correcting and eleting codes
Into the mode of expansion instruction set, data storage cost is significantly reduced, significantly reduces CPU consumption；And according to hard disk
Operational factor carries out hard disk failure prediction, has higher predictablity rate, can be automatically to predicting that the hard disk of failure is carried out
Data Migration, so as to ensure that the data in the hard disk of prediction failure are not lost, improve data reliability；In addition it is possible to
Dynamic migration and the management of cold and hot data are realized, the data storage cost of the synthesis of storage system is reduce further, optimizes
The data management mode of storage system.
Present invention also offers a kind of nonvolatile computer storage media, computer-readable storage medium is stored with least one can
Execute instruction, executable instruction can perform the data managing method of the storage system in above-mentioned any means embodiment.
Fig. 5 shows a kind of structural representation of computing device according to embodiments of the present invention, the specific embodiment of the invention
The specific implementation to computing device does not limit.
As shown in figure 5, the computing device can include：Processor (processor) 502, communication interface
(Communications Interface) 504, memory (memory) 506 and communication bus 508.
Wherein：
Processor 502, communication interface 504 and memory 506 complete mutual communication by communication bus 508.
Communication interface 504, for being communicated with the network element of miscellaneous equipment such as client or other servers etc..
Processor 502, for configuration processor 510, the data managing method that can specifically perform above-mentioned storage system is implemented
Correlation step in example.
Specifically, program 510 can include program code, and the program code includes computer-managed instruction.
Processor 502 is probably central processor CPU, or specific integrated circuit ASIC (Application
Specific Integrated Circuit), or it is arranged to implement the integrated electricity of one or more of the embodiment of the present invention
Road.The one or more processors that computing device includes, can be same type of processor, such as one or more CPU；Also may be used
To be different types of processor, such as one or more CPU and one or more ASIC.
Memory 506, for depositing program 510.Memory 506 may include high-speed RAM memory, it is also possible to also include
Nonvolatile memory (non-volatile memory), for example, at least a magnetic disk storage.
Program 510 specifically can be used for so that processor 502 performs the storage system in above-mentioned any means embodiment
Data managing method.The specific implementation of each step may refer in the data management embodiment of above-mentioned storage system in program 510
Corresponding steps and unit in corresponding description, will not be described here.It is apparent to those skilled in the art that it is
The convenience of description the equipment of foregoing description and the specific work process of module, may be referred in preceding method embodiment with succinctly
Corresponding process description, will not be repeated here.
Algorithm and display be not inherently related to any certain computer, virtual system or miscellaneous equipment provided herein.
Various general-purpose systems can also be used together with teaching based on this.As described above, required by constructing this kind of system
Structure be obvious.In addition, the present invention is not also directed to any certain programmed language.It should be understood that it can utilize various
Programming language realizes the content of invention described herein, and the description done above to language-specific is to disclose this hair
Bright preferred forms.
In the specification that this place provides, numerous specific details are set forth.It is to be appreciated, however, that the implementation of the present invention
Example can be put into practice in the case of these no details.In some instances, known method, structure is not been shown in detail
And technology, so as not to obscure the understanding of this description.
Similarly, it will be appreciated that in order to simplify the disclosure and help to understand one or more of each inventive aspect,
Above in the description to the exemplary embodiment of the present invention, each feature of the invention is grouped together into single implementation sometimes
In example, figure or descriptions thereof.However, the method for the disclosure should be construed to reflect following intention：I.e. required guarantor
The application claims of shield features more more than the feature being expressly recited in each claim.It is more precisely, such as following
Claims reflect as, inventive aspect is all features less than single embodiment disclosed above.Therefore,
Thus the claims for following embodiment are expressly incorporated in the embodiment, wherein each claim is in itself
Separate embodiments all as the present invention.
Those skilled in the art, which are appreciated that, to be carried out adaptively to the module in the equipment in embodiment
Change and they are arranged in one or more equipment different from the embodiment.Can be the module or list in embodiment
Member or component be combined into a module or unit or component, and can be divided into addition multiple submodule or subelement or
Sub-component.In addition at least some in such feature and/or process or unit exclude each other, it can use any
Combination is disclosed to all features disclosed in this specification (including adjoint claim, summary and accompanying drawing) and so to appoint
Where all processes or unit of method or equipment are combined.Unless expressly stated otherwise, this specification (including adjoint power
Profit requires, summary and accompanying drawing) disclosed in each feature can be by providing the alternative features of identical, equivalent or similar purpose come generation
Replace.
In addition, it will be appreciated by those of skill in the art that although some embodiments described herein include other embodiments
In included some features rather than further feature, but the combination of the feature of different embodiments means in of the invention
Within the scope of and form different embodiments.For example, in the following claims, embodiment claimed is appointed
One of meaning mode can use in any combination.
The all parts embodiment of the present invention can be realized with hardware, or to be run on one or more processor
Software module realize, or realized with combinations thereof.It will be understood by those of skill in the art that it can use in practice
Microprocessor or digital signal processor (DSP) are come one of some or all parts in realizing according to embodiments of the present invention
A little or repertoire.The present invention is also implemented as setting for performing some or all of method as described herein
Standby or program of device (for example, computer program and computer program product).Such program for realizing the present invention can deposit
Storage on a computer-readable medium, or can have the form of one or more signal.Such signal can be from because of spy
Download and obtain on net website, either provide on carrier signal or provided in the form of any other.
It should be noted that the present invention will be described rather than limits the invention for above-described embodiment, and ability
Field technique personnel can design alternative embodiment without departing from the scope of the appended claims.In the claims,
Any reference symbol between bracket should not be configured to limitations on claims.Word "comprising" does not exclude the presence of not
Element or step listed in the claims.Word "a" or "an" before element does not exclude the presence of multiple such
Element.The present invention can be by means of including the hardware of some different elements and being come by means of properly programmed computer real
It is existing.In if the unit claim of equipment for drying is listed, several in these devices can be by same hardware branch
To embody.The use of word first, second, and third does not indicate that any order.These words can be explained and run after fame
Claim.
The invention discloses：A1. a kind of data managing method of storage system, number of the storage system based on correcting and eleting codes
Run according to memory mechanism, and be integrated with expansion instruction set, methods described includes：
Obtain hard disk operational factor corresponding with the hard disk in the storage system；
The hard disk operational factor is analyzed, obtains hard disk failure prediction result；
According to the hard disk failure prediction result, to predicting that the hard disk of failure carries out Data Migration；
Methods described also includes：
For the data stored in the storage system, the access frequency of the data is calculated；
According to the access frequency of the data, determine the data type of the data, and by the Data Migration to institute
State in memory block corresponding to data type.
A2. the method according to A1, wherein, it is described that the hard disk operational factor is analyzed, obtain hard disk failure
Prediction result further comprises：
According to preset algorithm and default hard disk operational factor threshold value, the hard disk operational factor is analyzed, obtained hard
Disk failure predication result.
A3. the method according to A1 or A2, wherein, the hard disk operational factor includes one kind or more of following parameter
Kind：Remap sector count that sector count, current sector number to be mapped, off line can not correct, access check errors rate, bottom
Layer data read error rate, disk read-write flux performance parameter, start and stop counting, software read error rate and moment of torsion amplification meter
Number.
A4. the method according to any one of A1-A3, wherein, described according to the hard disk failure prediction result, to pre-
Survey after the hard disk progress Data Migration of failure, methods described also includes：
The hard disk of the prediction failure is added in failure blacklist.
A5. the method according to any one of A1-A4, wherein, the access frequency according to the data, it is determined that described
The data type of data, and the Data Migration to memory block corresponding with the data type is further comprised：
Judge whether the access frequency of the data is less than the first predeterminated frequency threshold value；
If the access frequency for judging to obtain the data is less than the first predeterminated frequency threshold value, it is determined that the data of the data
Type is cold data type, and by the Data Migration into tape storage area；
A6. the method according to A5, wherein, methods described also includes：
If the access frequency for judging to obtain the data is more than or equal to the first predeterminated frequency threshold value, the data are judged
Access frequency whether be more than the second predeterminated frequency threshold value；
If the access frequency for judging to obtain the data is more than the second predeterminated frequency threshold value, it is determined that the data of the data
Type is dsc data type, and by the Data Migration into flash memory storage area；
If the access frequency for judging to obtain the data is less than or equal to the second predeterminated frequency threshold value, it is determined that the data
Data type be warm data type, and by the Data Migration into disk storage area；
Wherein, the first predeterminated frequency threshold value is less than the second predeterminated frequency threshold value.
A7. the method according to any one of A1-A6, wherein, the bit wide for the instruction that the extended instruction is concentrated is 256
Position.
The invention also discloses：B8. a kind of data administrator of storage system, the storage system is based on correcting and eleting codes
Data storage mechanism is run, and is integrated with expansion instruction set, and described device includes：
Acquisition module, suitable for obtaining hard disk operational factor corresponding with the hard disk in the storage system；
Analysis module, suitable for analyzing the hard disk operational factor, obtain hard disk failure prediction result；
Data Migration module, suitable for according to the hard disk failure prediction result, the hard disk to predicting failure carries out data and moved
Move；
Described device also includes：
Computing module, suitable for the data for being stored in the storage system, calculate the access frequency of the data；
Determining module, suitable for the access frequency according to the data, determine the data type of the data；
The Data Migration module is further adapted for：By the Data Migration to memory block corresponding with the data type
In.
B9. the device according to B8, wherein, the analysis module is further adapted for：
According to preset algorithm and default hard disk operational factor threshold value, the hard disk operational factor is analyzed, obtained hard
Disk failure predication result.
B10. the device according to B8 or B9, wherein, the hard disk operational factor includes one kind or more of following parameter
Kind：Remap sector count that sector count, current sector number to be mapped, off line can not correct, access check errors rate, bottom
Layer data read error rate, disk read-write flux performance parameter, start and stop counting, software read error rate and moment of torsion amplification meter
Number.
B11. the device according to any one of B8-B10, wherein, described device also includes：
Blacklist add module, suitable for the hard disk of the prediction failure is added in failure blacklist.
B12. the device according to any one of B8-B11, wherein, the determining module includes：First judging unit and really
Order member；
First judging unit is suitable to：Judge whether the access frequency of the data is less than the first predeterminated frequency threshold value；
The determining unit is suitable to：If the access frequency that first judging unit judges to obtain the data is less than first
Predeterminated frequency threshold value, it is determined that the data type of the data is cold data type；
The Data Migration module is further adapted for：By data type for cold data type Data Migration to tape storage
Qu Zhong；
B13. the device according to B12, wherein, the determining module also includes：Second judging unit；
Second judging unit is suitable to：If the access frequency that first judging unit judges to obtain the data is more than
Or equal to the first predeterminated frequency threshold value, then judge whether the access frequency of the data is more than the second predeterminated frequency threshold value；
The determining unit is further adapted for：If the access frequency that second judging unit judges to obtain the data is big
In the second predeterminated frequency threshold value, it is determined that the data type of the data is dsc data type；If second judging unit is sentenced
The disconnected access frequency for obtaining the data is less than or equal to the second predeterminated frequency threshold value, it is determined that the data type of the data is
Warm data type；
The Data Migration module is further adapted for：By data type for dsc data type Data Migration to flash memory storage
Qu Zhong；By data type for warm data type Data Migration into disk storage area；
Wherein, the first predeterminated frequency threshold value is less than the second predeterminated frequency threshold value.
B14. the device according to any one of B8-B13, wherein, the bit wide for the instruction that the extended instruction is concentrated is 256
Position.
The invention also discloses：C15. a kind of computing device, including：Processor, memory, communication interface and communication are total
Line, the processor, the memory and the communication interface complete mutual communication by the communication bus；
The memory is used to deposit an at least executable instruction, and the executable instruction makes the computing device such as
Operated corresponding to the data managing method of storage system any one of A1-A7.
The invention also discloses：D16. a kind of computer-readable storage medium, being stored with least one in the storage medium can hold
Row instruction, the executable instruction make the data managing method of storage system of the computing device as any one of A1-A7
Corresponding operation.
Claims (10)
1. a kind of data managing method of storage system, data storage mechanism operation of the storage system based on correcting and eleting codes, and
Expansion instruction set is integrated with, methods described includes：
Obtain hard disk operational factor corresponding with the hard disk in the storage system；
The hard disk operational factor is analyzed, obtains hard disk failure prediction result；
According to the hard disk failure prediction result, to predicting that the hard disk of failure carries out Data Migration；
Methods described also includes：
For the data stored in the storage system, the access frequency of the data is calculated；
According to the access frequency of the data, determine the data type of the data, and by the Data Migration to the number
According in memory block corresponding to type.
2. the method according to claim 11, wherein, it is described that the hard disk operational factor is analyzed, obtain hard disk event
Barrier prediction result further comprises：
According to preset algorithm and default hard disk operational factor threshold value, the hard disk operational factor is analyzed, obtains hard disk event
Hinder prediction result.
3. method according to claim 1 or 2, wherein, the hard disk operational factor includes one kind or more of following parameter
Kind：Remap sector count that sector count, current sector number to be mapped, off line can not correct, access check errors rate, bottom
Layer data read error rate, disk read-write flux performance parameter, start and stop counting, software read error rate and moment of torsion amplification meter
Number.
4. according to the method described in claim any one of 1-3, wherein, it is right described according to the hard disk failure prediction result
After predicting that the hard disk of failure carries out Data Migration, methods described also includes：
The hard disk of the prediction failure is added in failure blacklist.
5. according to the method described in claim any one of 1-4, wherein, the access frequency according to the data, determine institute
The data type of data is stated, and the Data Migration to memory block corresponding with the data type is further comprised：
Judge whether the access frequency of the data is less than the first predeterminated frequency threshold value；
If the access frequency for judging to obtain the data is less than the first predeterminated frequency threshold value, it is determined that the data type of the data
For cold data type, and by the Data Migration into tape storage area.
6. according to the method for claim 5, wherein, methods described also includes：
If the access frequency for judging to obtain the data is more than or equal to the first predeterminated frequency threshold value, the visit of the data is judged
Ask whether frequency is more than the second predeterminated frequency threshold value；
If the access frequency for judging to obtain the data is more than the second predeterminated frequency threshold value, it is determined that the data type of the data
For dsc data type, and by the Data Migration into flash memory storage area；
If the access frequency for judging to obtain the data is less than or equal to the second predeterminated frequency threshold value, it is determined that the number of the data
It is warm data type according to type, and by the Data Migration into disk storage area；
Wherein, the first predeterminated frequency threshold value is less than the second predeterminated frequency threshold value.
7. according to the method described in claim any one of 1-6, wherein, the bit wide for the instruction that the extended instruction is concentrated is 256
Position.
8. a kind of data administrator of storage system, data storage mechanism operation of the storage system based on correcting and eleting codes, and
Expansion instruction set is integrated with, described device includes：
Acquisition module, suitable for obtaining hard disk operational factor corresponding with the hard disk in the storage system；
Analysis module, suitable for analyzing the hard disk operational factor, obtain hard disk failure prediction result；
Data Migration module, suitable for according to the hard disk failure prediction result, Data Migration is carried out to the hard disk of prediction failure；
Described device also includes：
Computing module, suitable for the data for being stored in the storage system, calculate the access frequency of the data；
Determining module, suitable for the access frequency according to the data, determine the data type of the data；
The Data Migration module is further adapted for：By in the Data Migration to memory block corresponding with the data type.
9. a kind of computing device, including：Processor, memory, communication interface and communication bus, the processor, the storage
Device and the communication interface complete mutual communication by the communication bus；
The memory is used to deposit an at least executable instruction, and the executable instruction makes the computing device such as right will
Ask and operated corresponding to the data managing method of the storage system any one of 1-7.
10. a kind of computer-readable storage medium, an at least executable instruction, the executable instruction are stored with the storage medium
Make operation corresponding to the data managing method of storage system of the computing device as any one of claim 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710703651.0A CN107391301A (en) | 2017-08-16 | 2017-08-16 | Data managing method, device, computing device and the storage medium of storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710703651.0A CN107391301A (en) | 2017-08-16 | 2017-08-16 | Data managing method, device, computing device and the storage medium of storage system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107391301A true CN107391301A (en) | 2017-11-24 |
Family
ID=60352921
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710703651.0A Pending CN107391301A (en) | 2017-08-16 | 2017-08-16 | Data managing method, device, computing device and the storage medium of storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107391301A (en) |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108073486A (en) * | 2017-12-28 | 2018-05-25 | 新华三大数据技术有限公司 | The Forecasting Methodology and device of a kind of hard disk failure |
| CN108205424A (en) * | 2017-12-29 | 2018-06-26 | 北京奇虎科技有限公司 | Data migration method, device and electronic equipment based on disk |
| CN108780386A (en) * | 2017-12-20 | 2018-11-09 | 华为技术有限公司 | A kind of methods, devices and systems of data storage |
| CN108845760A (en) * | 2018-05-28 | 2018-11-20 | 郑州云海信息技术有限公司 | A kind of HD management method, apparatus, equipment and readable storage medium storing program for executing |
| CN109144202A (en) * | 2018-09-05 | 2019-01-04 | 郑州云海信息技术有限公司 | Data processing method based on the hard disk and relevant apparatus |
| CN109407995A (en) * | 2018-11-05 | 2019-03-01 | 北京三快在线科技有限公司 | Cold and hot data managing method, device, electronic equipment and computer-readable medium |
| CN109582237A (en) * | 2018-11-30 | 2019-04-05 | 上海工程技术大学 | A kind of bad sector mapping method based on mechanical hard disk |
| CN109658975A (en) * | 2018-11-02 | 2019-04-19 | 北京大学 | A kind of active data restorative procedure and system towards correcting and eleting codes |
| CN109710443A (en) * | 2018-12-24 | 2019-05-03 | 平安科技（深圳）有限公司 | A kind of data processing method, device, equipment and storage medium |
| CN110262758A (en) * | 2019-06-28 | 2019-09-20 | 深信服科技股份有限公司 | A kind of data storage and management method, system and relevant device |
| CN110321067A (en) * | 2018-03-30 | 2019-10-11 | 卡巴斯基实验室股份制公司 | The system and method that estimation and management storage equipment are degenerated |
| CN110888763A (en) * | 2018-09-11 | 2020-03-17 | 北京奇虎科技有限公司 | Disk fault diagnosis method and device, terminal equipment and computer storage medium |
| CN111142807A (en) * | 2020-04-08 | 2020-05-12 | 上海飞旗网络技术股份有限公司 | Storage optimization method and device based on storage address traversal |
| CN111309502A (en) * | 2020-02-16 | 2020-06-19 | 西安奥卡云数据科技有限公司 | Solid state disk service life prediction method |
| CN112867984A (en) * | 2018-09-13 | 2021-05-28 | 铠侠股份有限公司 | Pool level storage management |
| CN113687766A (en) * | 2020-05-19 | 2021-11-23 | 中移动信息技术有限公司 | Data storage method, device, equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103186350A (en) * | 2011-12-31 | 2013-07-03 | 北京快网科技有限公司 | Hybrid storage system and hot spot data block migration method |
| CN105138476A (en) * | 2015-08-26 | 2015-12-09 | 广东创我科技发展有限公司 | Data storage method and system based on hadoop heterogeneous storage |
| CN105224888A (en) * | 2015-09-29 | 2016-01-06 | 上海爱数软件有限公司 | A kind of data of magnetic disk array protection system based on safe early warning technology |
| US20160335166A1 (en) * | 2015-05-14 | 2016-11-17 | Cisco Technology, Inc. | Smart storage recovery in a distributed storage system |
| CN106796540A (en) * | 2014-07-29 | 2017-05-31 | 沙特阿拉伯石油公司 | For the active fault recovery model of Distributed Calculation |
-
2017
- 2017-08-16 CN CN201710703651.0A patent/CN107391301A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103186350A (en) * | 2011-12-31 | 2013-07-03 | 北京快网科技有限公司 | Hybrid storage system and hot spot data block migration method |
| CN106796540A (en) * | 2014-07-29 | 2017-05-31 | 沙特阿拉伯石油公司 | For the active fault recovery model of Distributed Calculation |
| US20160335166A1 (en) * | 2015-05-14 | 2016-11-17 | Cisco Technology, Inc. | Smart storage recovery in a distributed storage system |
| CN105138476A (en) * | 2015-08-26 | 2015-12-09 | 广东创我科技发展有限公司 | Data storage method and system based on hadoop heterogeneous storage |
| CN105224888A (en) * | 2015-09-29 | 2016-01-06 | 上海爱数软件有限公司 | A kind of data of magnetic disk array protection system based on safe early warning technology |
Non-Patent Citations (1)
| Title |
|---|
| 陶建: "基于纠删码的高性能云存储系统的设计与实现", 《中国优秀硕士学位论文全文数据库信息科技辑（月刊）》 * |
Cited By (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108780386A (en) * | 2017-12-20 | 2018-11-09 | 华为技术有限公司 | A kind of methods, devices and systems of data storage |
| US10915397B2 (en) | 2017-12-20 | 2021-02-09 | Huawei Technologies Co., Ltd. | Data storage method, apparatus, and system |
| CN108780386B (en) * | 2017-12-20 | 2020-09-04 | 华为技术有限公司 | Data storage method, device and system |
| WO2019119311A1 (en) * | 2017-12-20 | 2019-06-27 | 华为技术有限公司 | Data storage method, device, and system |
| US11487618B2 (en) | 2017-12-20 | 2022-11-01 | Huawei Cloud Computing Technologies Co., Ltd. | Data storage method, apparatus, and system |
| CN108073486B (en) * | 2017-12-28 | 2022-05-10 | 新华三大数据技术有限公司 | Hard disk fault prediction method and device |
| CN108073486A (en) * | 2017-12-28 | 2018-05-25 | 新华三大数据技术有限公司 | The Forecasting Methodology and device of a kind of hard disk failure |
| CN108205424A (en) * | 2017-12-29 | 2018-06-26 | 北京奇虎科技有限公司 | Data migration method, device and electronic equipment based on disk |
| CN108205424B (en) * | 2017-12-29 | 2021-09-28 | 北京奇虎科技有限公司 | Data migration method and device based on disk and electronic equipment |
| CN110321067B (en) * | 2018-03-30 | 2023-04-14 | 卡巴斯基实验室股份制公司 | System and method for estimating and managing storage device degradation |
| CN110321067A (en) * | 2018-03-30 | 2019-10-11 | 卡巴斯基实验室股份制公司 | The system and method that estimation and management storage equipment are degenerated |
| CN108845760A (en) * | 2018-05-28 | 2018-11-20 | 郑州云海信息技术有限公司 | A kind of HD management method, apparatus, equipment and readable storage medium storing program for executing |
| CN109144202A (en) * | 2018-09-05 | 2019-01-04 | 郑州云海信息技术有限公司 | Data processing method based on the hard disk and relevant apparatus |
| CN110888763A (en) * | 2018-09-11 | 2020-03-17 | 北京奇虎科技有限公司 | Disk fault diagnosis method and device, terminal equipment and computer storage medium |
| US11853583B2 (en) | 2018-09-13 | 2023-12-26 | Kioxia Corporation | Pool-level storage management |
| CN112867984B (en) * | 2018-09-13 | 2024-05-17 | 铠侠股份有限公司 | Pool level storage management |
| CN112867984A (en) * | 2018-09-13 | 2021-05-28 | 铠侠股份有限公司 | Pool level storage management |
| CN109658975A (en) * | 2018-11-02 | 2019-04-19 | 北京大学 | A kind of active data restorative procedure and system towards correcting and eleting codes |
| CN109407995A (en) * | 2018-11-05 | 2019-03-01 | 北京三快在线科技有限公司 | Cold and hot data managing method, device, electronic equipment and computer-readable medium |
| CN109582237B (en) * | 2018-11-30 | 2021-12-17 | 上海工程技术大学 | Bad sector mapping method based on mechanical hard disk |
| CN109582237A (en) * | 2018-11-30 | 2019-04-05 | 上海工程技术大学 | A kind of bad sector mapping method based on mechanical hard disk |
| CN109710443A (en) * | 2018-12-24 | 2019-05-03 | 平安科技（深圳）有限公司 | A kind of data processing method, device, equipment and storage medium |
| CN109710443B (en) * | 2018-12-24 | 2023-06-16 | 平安科技（深圳）有限公司 | Data processing method, device, equipment and storage medium |
| CN110262758A (en) * | 2019-06-28 | 2019-09-20 | 深信服科技股份有限公司 | A kind of data storage and management method, system and relevant device |
| CN110262758B (en) * | 2019-06-28 | 2023-07-14 | 深信服科技股份有限公司 | Data storage management method, system and related equipment |
| CN111309502A (en) * | 2020-02-16 | 2020-06-19 | 西安奥卡云数据科技有限公司 | Solid state disk service life prediction method |
| CN111142807A (en) * | 2020-04-08 | 2020-05-12 | 上海飞旗网络技术股份有限公司 | Storage optimization method and device based on storage address traversal |
| CN113687766A (en) * | 2020-05-19 | 2021-11-23 | 中移动信息技术有限公司 | Data storage method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107391301A (en) | Data managing method, device, computing device and the storage medium of storage system | |
| US11119660B2 (en) | Determining when to replace a storage device by training a machine learning module | |
| US9563529B2 (en) | Selective monitoring of archive and backup storage | |
| CN103890724B (en) | Information processing apparatus, method for controlling information processing apparatus, host device, and performance evaluation method used for external storage device | |
| US11698729B2 (en) | System and method for reduced SSD failure via analysis and machine learning | |
| US11204827B2 (en) | Using a machine learning module to determine when to perform error checking of a storage unit | |
| US11620218B2 (en) | Using multi-tiered cache to satisfy input/output requests | |
| US11474941B2 (en) | Using multi-tiered cache to satisfy input/output requests | |
| US10901646B2 (en) | Update of RAID array parity | |
| US9892014B1 (en) | Automated identification of the source of RAID performance degradation | |
| US11768701B2 (en) | Exception analysis for data storage devices | |
| CN109669796B (en) | Disk fault prediction method and device | |
| CN115269289A (en) | Slow disk detection method and device, electronic equipment and storage medium | |
| US10783042B2 (en) | System and method of assessing and managing storage device degradation | |
| CN109144766B (en) | Data storage and reconstruction method and device and electronic equipment | |
| TW200945193A (en) | Adaptation of contentious storage virtualization configurations | |
| US10949359B2 (en) | Optimizing cache performance with probabilistic model | |
| US20210349789A1 (en) | Catalog restoration | |
| CN111788558B (en) | System and method for detecting bit decay in a distributed storage device having a failure domain | |
| US11145332B2 (en) | Proactively refreshing storage zones within a storage device | |
| JP7556656B2 (en) | Deciding which type of destage to perform based on the prioritization between operational performance and maintaining drive life | |
| CN117149473A (en) | Method, system, equipment and medium for predicting hard disk faults | |
| JP2023521724A (en) | Determining the type of destage to perform based on the priority between operational performance and preserving drive life | |
| CN115373903A (en) | Logic backup method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20171124 |