CN110795034A - Data migration method, device and equipment of storage system and readable storage medium - Google Patents
Data migration method, device and equipment of storage system and readable storage medium Download PDFInfo
- Publication number
- CN110795034A CN110795034A CN201910995298.7A CN201910995298A CN110795034A CN 110795034 A CN110795034 A CN 110795034A CN 201910995298 A CN201910995298 A CN 201910995298A CN 110795034 A CN110795034 A CN 110795034A
- Authority
- CN
- China
- Prior art keywords
- data
- solid state
- hard disk
- disk drive
- priority threshold
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G06F3/0616—Improving the reliability of storage systems in relation to life time, e.g. increasing Mean Time Between Failures [MTBF]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0647—Migration mechanisms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Techniques For Improving Reliability Of Storages (AREA)
Abstract
The invention discloses a data migration method of a storage system, which comprises the following steps: when a preset period is reached, counting the access state of each data unit in the current preset period; respectively acquiring access states of each data unit for a preset number of continuous periods; respectively calculating the data migration priority of each data unit according to the access state and the preset state migration diagram of each data unit corresponding to a preset number of continuous preset periods; and migrating the data units reaching the first priority threshold value in the solid state disk to the hard disk drive, and migrating the data units reaching the second priority threshold value in the hard disk drive to the solid state disk. By applying the technical scheme provided by the embodiment of the invention, the labor cost is saved, the frequency of data migration is greatly reduced, the damage probability of the solid state disk is reduced, and the service life of the whole storage system is prolonged. The invention also discloses a data migration device, equipment and a storage medium of the storage system, and the data migration device, the equipment and the storage medium have corresponding technical effects.
Description
Technical Field
The present invention relates to the field of storage technologies, and in particular, to a data migration method, apparatus, device, and computer-readable storage medium for a storage system.
Background
In the big data era, the storage and transmission rate of data plays an important role in the normal and fast operation of the storage server. In a multi-tiered storage system, there must be migration of data. With the continuous expansion of stored services, new files and data are continuously created, and infrequently used cold data must be migrated to a Hard Disk Drive (HDD) with low read-write efficiency and large time delay, while frequently used hot data needs to be migrated to a Solid State Disk (SSD) with high read-write rate and small time delay due to frequent user access. The lifecycle of each data unit undergoes creation, activation, cooling, deletion, etc. processes, and the data migrates among disks of different capabilities depending on the migration algorithm.
One of the data migration methods in the prior art is to select the migrated data unit by defining the cold and hot of each data unit in the storage system by a worker, which requires a huge labor cost. The other is to use a Least Recently Used (LRU) algorithm to calculate the cold and hot of each data unit and then migrate the data unit. However, most of the write operations are concentrated on a small number of data units, and most of the physical page addresses are rarely rewritten in the life cycle, that is, the user reads more frequently than writes, so that considering the problem of limited erase times of the solid state disk, the least recently used algorithm is not very suitable for the solid state disk, which may cause damage to the solid state disk and affect the life of the entire storage system.
In summary, how to effectively solve the problems of high labor cost, high solid state disk damage probability, influence on the service life of the whole storage system and the like is a problem which needs to be solved urgently by technical personnel in the field at present.
Disclosure of Invention
The invention aims to provide a data migration method of a storage system, which saves labor cost, greatly reduces the frequency of data migration, reduces the damage probability of a solid state disk and prolongs the service life of the whole storage system; another object of the present invention is to provide a data migration apparatus, device and computer readable storage medium of a storage system.
In order to solve the technical problems, the invention provides the following technical scheme:
a method of data migration for a storage system, comprising:
when a preset period is reached, counting the access state of each data unit in the current preset period; the data units are obtained by pre-dividing data stored in a solid state disk and data stored in a hard disk drive in a storage system;
respectively acquiring the access states of the data units in a preset number of continuous preset periods; the preset period with the continuous preset number takes the current preset period as an end period;
respectively calculating the data migration priority of each data unit according to the access state and the preset state migration diagram of the data unit corresponding to the preset number of continuous periods; wherein, the state transition diagram is preset with the corresponding relation between each access state and each data transition priority;
and migrating the data units reaching the first priority threshold value in the solid state disk to the hard disk drive, and migrating the data units reaching the second priority threshold value in the hard disk drive to the solid state disk.
In an embodiment of the present invention, migrating the data unit in the hard disk drive that reaches the second priority threshold to the solid state disk includes:
acquiring the erasing times of each physical page address block in the solid state disk; each physical page address block is obtained by dividing the total physical page address of the solid state disk by taking one data storage unit as a unit;
sorting each physical page address block according to the erasing times to obtain a first sorting result;
and migrating the data units reaching the second priority threshold value in the hard disk drive to each physical page address block in the solid state disk according to the priority from small to large in the first sequencing result.
In a specific embodiment of the present invention, before migrating the data unit in the solid state disk that reaches the first priority threshold to the hard disk drive and migrating the data unit in the hard disk drive that reaches the second priority threshold to the solid state disk, the method further includes:
counting a first data volume size formed by each data unit reaching the first priority threshold value in the solid state disk and a second data volume size formed by each data unit reaching the second priority threshold value in the hard disk drive;
comparing the first data size with the second data size to obtain a comparison result;
migrating the data units in the solid state disk, which reach the first priority threshold, to the hard disk drive, and migrating the data units in the hard disk drive, which reach the second priority threshold, to the solid state disk, including:
when the comparison result shows that the first data volume is smaller than the second data volume, migrating the data units reaching the first priority threshold value in the solid state disk to the hard disk drive, and selecting partial data units with the total data volume smaller than or equal to the first data volume from the data units reaching the second priority threshold value in the hard disk drive to migrate to the solid state disk;
and when the comparison result shows that the first data volume is larger than the second data volume, migrating the data units reaching the second priority threshold value in the hard disk drive to the solid state disk, and selecting partial data units with the total data volume less than or equal to the second data volume from the data units reaching the first priority threshold value in the solid state disk to migrate to the hard disk drive.
In an embodiment of the present invention, selecting, from the data units in the hard disk drive that reach the second priority threshold, a part of data units whose total data amount is smaller than or equal to the first data amount to migrate to the solid state disk, includes:
respectively calculating first difference values of the data migration priority of each data unit reaching the second priority threshold value in the hard disk drive and the second priority threshold value, and sorting the first difference values to obtain a second sorting result;
selecting a part of data units with the total data amount less than or equal to the first data amount from one end with a small first difference value in the second sorting result from the data units reaching the second priority threshold value in the hard disk drive to migrate to the solid state disk;
selecting a part of data units with the total data amount less than or equal to the second data amount from the data units reaching the first priority threshold in the solid state disk to migrate to the hard disk drive, wherein the selecting step comprises the following steps:
respectively calculating second difference values of the data migration priority of each data unit reaching a first priority threshold value in the solid state disk and the first priority threshold value, and sorting the second difference values to obtain a third sorting result;
and selecting a part of data units with the total data amount smaller than or equal to the second data amount from one end with a larger second difference value in the third sorting result from the data units reaching the first priority threshold value in the solid state disk, and transferring the part of data units to the hard disk drive.
In a specific embodiment of the present invention, respectively obtaining access states of each data unit for a preset number of consecutive cycles includes:
respectively acquiring the access states of two continuous preset periods of each data unit;
respectively calculating the data migration priority of each data unit according to the access state and the preset state migration diagram of the data unit corresponding to the preset number of continuous preset periods respectively, including:
and respectively calculating the data migration priority of each data unit according to the access state and the preset state migration diagram of two continuous preset periods corresponding to each data unit.
A data migration apparatus of a storage system, comprising:
the access state counting module is used for counting the access state of each data unit in the current preset period when the preset period is reached; the data units are obtained by pre-dividing data stored in a solid state disk and data stored in a hard disk drive in a storage system;
the access state acquisition module is used for respectively acquiring the access states of the data units in the preset period of the continuous preset number; the preset period with the continuous preset number takes the current preset period as an end period;
the priority calculation module is used for calculating the data migration priority of each data unit according to the access state and the preset state migration diagram of each data unit corresponding to a continuous preset number of preset periods respectively; wherein, the state transition diagram is preset with the corresponding relation between each access state and each data transition priority;
and the data migration module is used for migrating the data units reaching the first priority threshold value in the solid state disk to the hard disk drive and migrating the data units reaching the second priority threshold value in the hard disk drive to the solid state disk.
In a specific embodiment of the present invention, the data migration module includes:
the erasing and writing frequency obtaining submodule is used for obtaining the erasing and writing frequency of each physical page address block in the solid state disk; each physical page address block is obtained by dividing the total physical page address of the solid state disk by taking one data storage unit as a unit;
the erasing and writing times sequencing submodule is used for sequencing the physical page address blocks according to the erasing and writing times to obtain a first sequencing result;
and the data migration submodule is used for migrating the data units reaching the second priority threshold value in the hard disk drive to each physical page address block in the solid state disk according to the priority from small to large in the first sequencing result.
In one embodiment of the present invention, the method further comprises:
a data volume counting module, configured to count a first data volume formed by data units reaching the first priority threshold in the solid state disk and a second data volume formed by data units reaching the second priority threshold in the hard disk drive before migrating the data units reaching the first priority threshold in the hard disk drive to the hard disk drive and migrating the data units reaching the second priority threshold in the hard disk drive to the solid state disk;
the data volume comparison module is used for comparing the first data volume with the second data volume to obtain a comparison result;
the data migration module specifically migrates the data units in the solid state disk, which reach the first priority threshold, to the hard disk drive when the comparison result shows that the first data size is smaller than the second data size, and selects a part of data units, which have a data total amount smaller than or equal to the first data size, from the data units in the hard disk drive, which reach the second priority threshold, to migrate to the solid state disk; and when the comparison result shows that the first data volume is larger than the second data volume, migrating the data units reaching the second priority threshold value in the hard disk drive to the solid state disk, and selecting partial data units with the total data volume less than or equal to the second data volume from the data units reaching the first priority threshold value in the solid state disk to migrate to the module of the hard disk drive.
A data migration apparatus of a storage system, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the data migration method of the storage system when executing the computer program.
A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the data migration method of the storage system as set forth above.
By applying the method provided by the embodiment of the invention, when the preset period is reached, the access state of each data unit in the current preset period is counted; each data unit is obtained by pre-dividing data stored in a solid state disk and data stored in a hard disk drive in a storage system; respectively acquiring access states of each data unit for a preset number of continuous periods; the method comprises the following steps that a preset period with a continuous number takes a current preset period as an end period; respectively calculating the data migration priority of each data unit according to the access state and the preset state migration diagram of each data unit corresponding to a preset number of continuous preset periods; wherein, the state transition diagram is preset with the corresponding relation between each access state and each data transition priority; and migrating the data units reaching the first priority threshold value in the solid state disk to the hard disk drive, and migrating the data units reaching the second priority threshold value in the hard disk drive to the solid state disk. By setting the state transition diagram, calculating the data transition priority according to the access state of each data unit and the state transition diagram, and setting the first priority threshold and the second priority threshold to limit the data transition in the solid state disk and the hard disk drive, the data transition can be automatically completed, the labor cost is saved, the frequency of data transition is greatly reduced, the damage probability of the solid state disk is reduced, and the service life of the whole storage system is prolonged.
Accordingly, embodiments of the present invention further provide a data migration apparatus, a device, and a computer-readable storage medium of a storage system corresponding to the data migration method of the storage system, which have the above technical effects and are not described herein again.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flowchart illustrating an implementation of a data migration method of a storage system according to an embodiment of the present invention;
FIG. 2 is a flowchart of another implementation of a data migration method of a storage system according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a state transition diagram according to an embodiment of the present invention;
FIG. 4 is a block diagram of a data migration apparatus of a storage system according to an embodiment of the present invention;
fig. 5 is a block diagram of a data migration apparatus of a storage system according to an embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The first embodiment is as follows:
referring to fig. 1, fig. 1 is a flowchart of an implementation of a data migration method of a storage system according to an embodiment of the present invention, where the method may include the following steps:
s101: and when the preset period is reached, counting the access state of each data unit in the current preset period.
The data units are obtained by pre-dividing data stored in the solid state disk and data stored in the hard disk drive in the storage system.
The method can divide data stored in the solid state disk and data stored in the hard disk drive in the storage system in advance to obtain a plurality of data units, can preset a period for carrying out access state statistics on each data unit, and when the preset period is reached, statistics is carried out on the access state of each data unit in the current preset period.
It should be noted that the preset period may be set and adjusted according to actual situations, which is not limited in the embodiment of the present invention, for example, the preset period may be set to 1 day.
S102: and respectively acquiring the access states of each data unit for a preset number of continuous preset periods.
And taking the current preset period as an end period for the preset period of continuous number.
The number of access state cycles required for calculating the data migration priority of each data unit may be preset, and after the access state of each data unit in the current preset cycle is obtained through statistics, the access state of each data unit in the preset cycle with the current preset cycle as the last cycle may be obtained respectively.
S103: and respectively calculating the data migration priority of each data unit according to the access state and the preset state migration diagram of each data unit corresponding to a preset number of continuous preset periods.
Wherein, the state transition diagram is preset with the corresponding relation between each access state and each data transition priority.
A state transition diagram preset with a correspondence between each access state and each data transition priority may be deployed in advance. The data migration priority of each data unit can be calculated according to the obtained access state and the preset state migration diagram of each data unit corresponding to a preset number of continuous preset periods respectively.
S104: and migrating the data units reaching the first priority threshold value in the solid state disk to the hard disk drive, and migrating the data units reaching the second priority threshold value in the hard disk drive to the solid state disk.
Since the data stored in the high-performance low-latency solid state disk is hot data with more accesses, the data stored in the low-performance high-latency hard disk drive is cold data with less accesses. The first priority threshold to be reached for migrating the data units in the solid state disk to the hard disk drive and the first priority threshold to be reached for migrating the data units in the hard disk drive to the solid state disk may be preset respectively. After the data migration priorities of the data units are respectively calculated, the data units in the solid state disk, which reach the first priority threshold, may be migrated to the hard disk drive, and the data units in the hard disk drive, which reach the second priority threshold, may be migrated to the solid state disk.
It should be noted that the first priority threshold and the second priority threshold may be the same or different, and may be set and adjusted according to actual situations, which is not limited in the embodiment of the present invention.
By applying the method provided by the embodiment of the invention, when the preset period is reached, the access state of each data unit in the current preset period is counted; each data unit is obtained by pre-dividing data stored in a solid state disk and data stored in a hard disk drive in a storage system; respectively acquiring access states of each data unit for a preset number of continuous periods; the method comprises the following steps that a preset period with a continuous number takes a current preset period as an end period; respectively calculating the data migration priority of each data unit according to the access state and the preset state migration diagram of each data unit corresponding to a preset number of continuous preset periods; wherein, the state transition diagram is preset with the corresponding relation between each access state and each data transition priority; and migrating the data units reaching the first priority threshold value in the solid state disk to the hard disk drive, and migrating the data units reaching the second priority threshold value in the hard disk drive to the solid state disk. By setting the state transition diagram, calculating the data transition priority according to the access state of each data unit and the state transition diagram, and setting the first priority threshold and the second priority threshold to limit the data transition in the solid state disk and the hard disk drive, the data transition can be automatically completed, the labor cost is saved, the frequency of data transition is greatly reduced, the damage probability of the solid state disk is reduced, and the service life of the whole storage system is prolonged.
It should be noted that, based on the first embodiment, the embodiment of the present invention further provides a corresponding improvement scheme. In the following embodiments, steps that are the same as or correspond to those in the first embodiment may be referred to each other, and corresponding advantageous effects may also be referred to each other, which are not described in detail in the following modified embodiments.
Example two:
referring to fig. 2, fig. 2 is a flowchart of another implementation of a data migration method of a storage system according to an embodiment of the present invention, where the method may include the following steps:
s201: and when the preset period is reached, counting the access state of each data unit in the current preset period.
The data units are obtained by pre-dividing data stored in the solid state disk and data stored in the hard disk drive in the storage system.
S202: and respectively acquiring the access states of each data unit in two continuous preset periods.
The access state of each data unit in two consecutive preset periods can be specifically obtained, where the two preset periods are specifically a current preset period and a previous preset period. If each data unit is read in one cycle, record R is 1, otherwise record R is 0, if each data unit is accessed (read or written) in one cycle, record a is 1, and a is 0, the access state (R, a) of this data unit in one cycle is represented by a binary group, then each data unit is represented by the state in two cycles, and the state transition diagram can be specifically represented by fig. 3. By setting the number of access state cycles required for calculating the data migration priority of each data unit to two, not only is an effective data migration priority obtained, but also the complexity of data priority calculation is simplified.
S203: and respectively calculating the data migration priority of each data unit according to the access state of each data unit corresponding to two continuous preset periods and a preset state migration diagram.
Wherein, the state transition diagram is preset with the corresponding relation between each access state and each data transition priority.
The corresponding relationship between each access state and each data migration priority may be preset in the state transition diagram, and after the access states of two consecutive preset periods of each data unit are respectively obtained, the data migration priority of each data unit may be respectively calculated according to the access states of two consecutive preset periods respectively corresponding to each data unit and the preset state transition diagram. For example, as shown in fig. 3, when a data unit is read in two consecutive predetermined cycles, the data migration priority is 8; when a certain data unit is only written in two continuous preset periods, the data migration priority is 5; when a certain data unit is not written or read in two continuous preset periods, the data migration priority is 0, 1 and 3; when a certain data unit is written first and then read in two continuous preset periods, the data migration priority is 7 or 8; when a certain data unit is read and then written in two continuous preset periods, the data migration priority is 5, 7 and 8; when a certain data unit is only written first and then is not read and written in two continuous preset periods, the data migration priority is 0, 1, 2 and 4. The specific data migration priority may be calculated based on the access states of two consecutive preset periods and the data migration priority of each data unit at the beginning of counting the access states of the two preset periods.
S204: and counting a first data volume size formed by each data unit reaching the first priority threshold value in the solid state disk and a second data volume size formed by each data unit reaching the second priority threshold value in the hard disk drive.
After calculating the data migration priority of each data unit, a first data size formed by each data unit reaching the first priority threshold in the solid state disk and a second data size formed by each data unit reaching the second priority threshold in the hard disk drive may be counted. In the above example, the first priority threshold may be set to 5, the second priority threshold may be set to 4, and the first data size formed by each data unit with a priority lower than 5 in the solid state disk may be counted, that is, the data size formed by each data unit which is not written, not read, and not read and not written after being written first and then being read and not written in the solid state disk in two consecutive preset periods may be counted. And counting a second data volume size formed by the data units with the priority higher than 4 in the hard disk drive, namely counting the data volume size formed by the data units which are read in two continuous preset periods, only written in the two continuous preset periods and written after being read.
S205: and comparing the first data size with the second data size to obtain a comparison result.
After counting a first data size formed by each data unit reaching the first priority threshold in the solid state disk and a second data size formed by each data unit reaching the second priority threshold in the hard disk drive, the first data size and the second data size may be compared to obtain a comparison result.
S206: and when the comparison result shows that the first data size is smaller than the second data size, migrating the data units reaching the first priority threshold value in the solid state disk to the hard disk drive.
And when the comparison result shows that the first data size is smaller than the second data size, the data amount of the cold data in the solid state disk is smaller than the data amount of the hot data of the hard disk drive. In this case, data units in the solid state disk that reach the first priority threshold may be migrated to the hard disk drive.
S207: and respectively calculating first difference values of the data migration priority of each data unit reaching the second priority threshold value in the hard disk drive and the second priority threshold value, and sorting the first difference values to obtain a second sorting result.
Because the calculated data amount of the hot data of the hard disk drive is greater than the calculated data amount of the cold data in the solid state disk, and because a state that the solid state disk is about to be full currently exists, if all the data units in the hard disk drive reaching the second priority threshold are migrated to the solid state disk, the original hot data in the solid state disk may be deleted. In this case, first difference values between the data migration priority of each data unit in the hard disk drive reaching the second priority threshold and the second priority threshold may be calculated, respectively, and the magnitude of each first difference value may be sorted to obtain a second sorting result.
S208: and selecting a part of data units with the total data amount less than or equal to the first data amount from one end with a small first difference value in the second sorting result from the data units reaching the second priority threshold value in the hard disk drive to migrate to the solid state disk.
Since the smaller the difference from the second priority threshold, the higher the accessed heat of the corresponding data unit, the partial data unit with the total data amount less than or equal to the first data amount can be selected from the data unit with the first difference in the second sorting result to be migrated to the solid state disk in the data unit reaching the second priority threshold in the hard disk drive. Therefore, the hot data in the solid state disk are not deleted.
S209: and acquiring the erasing times of each physical page address block in the solid state disk.
Each physical page address block is obtained by dividing the total physical page address of the solid state disk by taking one data storage unit as a unit.
Considering that the solid state disk has the problem of limited erasing times, if the same block position is frequently written, the solid state disk may be damaged, the total physical page address of the solid state disk may be divided by using one data unit as a unit to obtain each physical page address block, and the erasing times of each physical page address block in the solid state disk may be obtained.
S210: and sequencing the address blocks of the physical pages according to the erasing times to obtain a first sequencing result.
After the erasing times of each physical page address block in the solid state disk are obtained, the physical page address blocks can be sorted according to the erasing times to obtain a first sorting result.
S211: and when the comparison result shows that the first data size is larger than the second data size, migrating the data units reaching the second priority threshold value in the hard disk drive to each physical page address block in the solid state disk according to the priorities from small to large in the first sequencing result.
After the first sequencing result of the erasing times of each physical page address block in the solid state disk is obtained, when the comparison result shows that the first data size is larger than the second data size, the calculated data size of the hot data of the hard disk drive is smaller than the calculated data size of the cold data in the solid state disk. In this case, data units in the hard disk drive that reach the second priority threshold may be migrated to the physical page address blocks in the solid state disk with a priority from small to large in the first ordering result. Therefore, the abrasion to the solid state disk is further reduced, the service life of the solid state disk is prolonged, and the service life of the whole storage system is prolonged.
S212: and respectively calculating second difference values of the data migration priority of each data unit reaching the first priority threshold value in the solid state disk and the first priority threshold value, and sorting the second difference values to obtain a third sorting result.
Because the calculated data amount of the cold data in the solid state disk is greater than the calculated data amount of the hot data in the hard disk drive, the probability of the subsequent access of the data unit with the relatively cold access state currently stored in the solid state disk is generally greater than the probability of the subsequent access of the data unit with the relatively cold access state currently stored in the hard disk drive, and in order to further reduce the data migration amount, second differences between the data migration priority of each data unit reaching the first priority threshold value in the solid state disk and the first priority threshold value may be calculated respectively, and the second differences are sorted in size to obtain a third sorting result.
S213: and selecting a part of data units with the total data quantity less than or equal to the second data quantity from one end with the larger second difference value in the third sorting result from the data units reaching the first priority threshold value in the solid state disk to be migrated to the hard disk drive.
Since the larger the difference from the first priority threshold, the higher the accessed heat of the corresponding data unit, the partial data unit with the total data amount less than or equal to the second data amount can be selected from the end with the larger second difference in the third sorting result to be migrated to the hard disk drive in the data unit reaching the first priority threshold in the solid state disk. Therefore, the data migration amount is reduced, and the abrasion to the solid state disk is further reduced.
Corresponding to the above method embodiments, embodiments of the present invention further provide a data migration apparatus of a storage system, and the data migration apparatus of the storage system described below and the data migration method of the storage system described above may be referred to correspondingly.
Referring to fig. 4, fig. 4 is a block diagram of a data migration apparatus of a storage system according to an embodiment of the present invention, where the apparatus may include:
an access state counting module 41, configured to count access states of the data units in a current preset period when the preset period is reached; each data unit is obtained by pre-dividing data stored in a solid state disk and data stored in a hard disk drive in a storage system;
an access state obtaining module 42, configured to obtain access states of each data unit for a preset number of consecutive cycles; the method comprises the following steps that a preset period with a continuous number takes a current preset period as an end period;
a priority calculating module 43, configured to calculate data migration priorities of the data units according to the access states of the data units in the preset number of continuous preset periods and the preset state migration diagram respectively corresponding to the data units; wherein, the state transition diagram is preset with the corresponding relation between each access state and each data transition priority;
and the data migration module 44 is configured to migrate the data units in the solid state disk that reach the first priority threshold to the hard disk drive, and migrate the data units in the hard disk drive that reach the second priority threshold to the solid state disk.
By applying the device provided by the embodiment of the invention, when the preset period is reached, the access state of each data unit in the current preset period is counted; each data unit is obtained by pre-dividing data stored in a solid state disk and data stored in a hard disk drive in a storage system; respectively acquiring access states of each data unit for a preset number of continuous periods; the method comprises the following steps that a preset period with a continuous number takes a current preset period as an end period; respectively calculating the data migration priority of each data unit according to the access state and the preset state migration diagram of each data unit corresponding to a preset number of continuous preset periods; wherein, the state transition diagram is preset with the corresponding relation between each access state and each data transition priority; and migrating the data units reaching the first priority threshold value in the solid state disk to the hard disk drive, and migrating the data units reaching the second priority threshold value in the hard disk drive to the solid state disk. By setting the state transition diagram, calculating the data transition priority according to the access state of each data unit and the state transition diagram, and setting the first priority threshold and the second priority threshold to limit the data transition in the solid state disk and the hard disk drive, the data transition can be automatically completed, the labor cost is saved, the frequency of data transition is greatly reduced, the damage probability of the solid state disk is reduced, and the service life of the whole storage system is prolonged.
In one embodiment of the present invention, the data migration module 44 includes:
the erasing and writing frequency obtaining submodule is used for obtaining the erasing and writing frequency of each physical page address block in the solid state disk; each physical page address block is obtained by dividing the total physical page address of the solid state disk by taking one data storage unit as a unit;
the erasure frequency sorting submodule is used for sorting the address blocks of the physical pages according to the erasure frequency to obtain a first sorting result;
and the data migration submodule is used for migrating the data units reaching the second priority threshold in the hard disk drive to each physical page address block in the solid state disk according to the priorities from small to large in the first sequencing result.
In one embodiment of the present invention, the apparatus may further include:
the data volume counting module is used for counting a first data volume formed by the data units reaching the first priority threshold value in the solid state disk and a second data volume formed by the data units reaching the second priority threshold value in the hard disk drive before the data units reaching the first priority threshold value in the hard disk drive are migrated to the hard disk drive and the data units reaching the second priority threshold value in the hard disk drive;
the data volume comparison module is used for comparing the first data volume with the second data volume to obtain a comparison result;
the data migration module 44 is specifically configured to, when the comparison result is that the first data size is smaller than the second data size, migrate the data units in the solid state disk that reach the first priority threshold to the hard disk drive, and select, from the data units in the hard disk drive that reach the second priority threshold, a part of the data units whose total data amount is smaller than or equal to the first data size to migrate to the solid state disk; and when the comparison result shows that the first data volume is larger than the second data volume, transferring the data units reaching the second priority threshold value in the hard disk drive to the solid state disk, and selecting partial data units with the total data volume less than or equal to the second data volume from the data units reaching the first priority threshold value in the solid state disk to transfer the partial data units to the hard disk drive.
Corresponding to the above method embodiment, referring to fig. 5, fig. 5 is a schematic diagram of a data migration apparatus of a storage system provided in the present invention, where the apparatus may include:
a memory 51 for storing a computer program;
the processor 52, when executing the computer program stored in the memory 51, may implement the following steps:
when a preset period is reached, counting the access state of each data unit in the current preset period; each data unit is obtained by pre-dividing data stored in a solid state disk and data stored in a hard disk drive in a storage system; respectively acquiring access states of each data unit for a preset number of continuous periods; the method comprises the following steps that a preset period with a continuous number takes a current preset period as an end period; respectively calculating the data migration priority of each data unit according to the access state and the preset state migration diagram of each data unit corresponding to a preset number of continuous preset periods; wherein, the state transition diagram is preset with the corresponding relation between each access state and each data transition priority; and migrating the data units reaching the first priority threshold value in the solid state disk to the hard disk drive, and migrating the data units reaching the second priority threshold value in the hard disk drive to the solid state disk.
For the introduction of the device provided by the present invention, please refer to the above method embodiment, which is not described herein again.
Corresponding to the above method embodiment, the present invention further provides a computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a processor, implementing the steps of:
when a preset period is reached, counting the access state of each data unit in the current preset period; each data unit is obtained by pre-dividing data stored in a solid state disk and data stored in a hard disk drive in a storage system; respectively acquiring access states of each data unit for a preset number of continuous periods; the method comprises the following steps that a preset period with a continuous number takes a current preset period as an end period; respectively calculating the data migration priority of each data unit according to the access state and the preset state migration diagram of each data unit corresponding to a preset number of continuous preset periods; wherein, the state transition diagram is preset with the corresponding relation between each access state and each data transition priority; and migrating the data units reaching the first priority threshold value in the solid state disk to the hard disk drive, and migrating the data units reaching the second priority threshold value in the hard disk drive to the solid state disk.
The computer-readable storage medium may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
For the introduction of the computer-readable storage medium provided by the present invention, please refer to the above method embodiments, which are not described herein again.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device, the apparatus and the computer-readable storage medium disclosed in the embodiments correspond to the method disclosed in the embodiments, so that the description is simple, and the relevant points can be referred to the description of the method.
The principle and the implementation of the present invention are explained in the present application by using specific examples, and the above description of the embodiments is only used to help understanding the technical solution and the core idea of the present invention. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
Claims (10)
1. A data migration method of a storage system is characterized by comprising the following steps:
when a preset period is reached, counting the access state of each data unit in the current preset period; the data units are obtained by pre-dividing data stored in a solid state disk and data stored in a hard disk drive in a storage system;
respectively acquiring the access states of the data units in a preset number of continuous preset periods; the preset period with the continuous preset number takes the current preset period as an end period;
respectively calculating the data migration priority of each data unit according to the access state and the preset state migration diagram of the data unit corresponding to the preset number of continuous periods; wherein, the state transition diagram is preset with the corresponding relation between each access state and each data transition priority;
and migrating the data units reaching the first priority threshold value in the solid state disk to the hard disk drive, and migrating the data units reaching the second priority threshold value in the hard disk drive to the solid state disk.
2. The data migration method of the storage system according to claim 1, wherein migrating the data units in the hard disk drive that reach the second priority threshold to the solid state disk comprises:
acquiring the erasing times of each physical page address block in the solid state disk; each physical page address block is obtained by dividing the total physical page address of the solid state disk by taking one data storage unit as a unit;
sorting each physical page address block according to the erasing times to obtain a first sorting result;
and migrating the data units reaching the second priority threshold value in the hard disk drive to each physical page address block in the solid state disk according to the priority from small to large in the first sequencing result.
3. The data migration method of the storage system according to claim 1 or 2, wherein before migrating the data units in the solid state disk that reach the first priority threshold to the hard disk drive and migrating the data units in the hard disk drive that reach the second priority threshold to the solid state disk, the method further comprises:
counting a first data volume size formed by each data unit reaching the first priority threshold value in the solid state disk and a second data volume size formed by each data unit reaching the second priority threshold value in the hard disk drive;
comparing the first data size with the second data size to obtain a comparison result;
migrating the data units in the solid state disk, which reach the first priority threshold, to the hard disk drive, and migrating the data units in the hard disk drive, which reach the second priority threshold, to the solid state disk, including:
when the comparison result shows that the first data volume is smaller than the second data volume, migrating the data units reaching the first priority threshold value in the solid state disk to the hard disk drive, and selecting partial data units with the total data volume smaller than or equal to the first data volume from the data units reaching the second priority threshold value in the hard disk drive to migrate to the solid state disk;
and when the comparison result shows that the first data volume is larger than the second data volume, migrating the data units reaching the second priority threshold value in the hard disk drive to the solid state disk, and selecting partial data units with the total data volume less than or equal to the second data volume from the data units reaching the first priority threshold value in the solid state disk to migrate to the hard disk drive.
4. The data migration method of the storage system according to claim 3, wherein selecting a portion of the data units with the total data amount less than or equal to the first data amount from the data units in the hard disk drive reaching the second priority threshold to migrate to the solid state disk comprises:
respectively calculating first difference values of the data migration priority of each data unit reaching the second priority threshold value in the hard disk drive and the second priority threshold value, and sorting the first difference values to obtain a second sorting result;
selecting a part of data units with the total data amount less than or equal to the first data amount from one end with a small first difference value in the second sorting result from the data units reaching the second priority threshold value in the hard disk drive to migrate to the solid state disk;
selecting a part of data units with the total data amount less than or equal to the second data amount from the data units reaching the first priority threshold in the solid state disk to migrate to the hard disk drive, wherein the selecting step comprises the following steps:
respectively calculating second difference values of the data migration priority of each data unit reaching a first priority threshold value in the solid state disk and the first priority threshold value, and sorting the second difference values to obtain a third sorting result;
and selecting a part of data units with the total data amount smaller than or equal to the second data amount from one end with a larger second difference value in the third sorting result from the data units reaching the first priority threshold value in the solid state disk, and transferring the part of data units to the hard disk drive.
5. The data migration method of the storage system according to claim 1, wherein respectively obtaining the access states of each data unit for a preset number of consecutive preset periods comprises:
respectively acquiring the access states of two continuous preset periods of each data unit;
respectively calculating the data migration priority of each data unit according to the access state and the preset state migration diagram of the data unit corresponding to the preset number of continuous preset periods respectively, including:
and respectively calculating the data migration priority of each data unit according to the access state and the preset state migration diagram of two continuous preset periods corresponding to each data unit.
6. A data migration apparatus of a storage system, comprising:
the access state counting module is used for counting the access state of each data unit in the current preset period when the preset period is reached; the data units are obtained by pre-dividing data stored in a solid state disk and data stored in a hard disk drive in a storage system;
the access state acquisition module is used for respectively acquiring the access states of the data units in the preset period of the continuous preset number; the preset period with the continuous preset number takes the current preset period as an end period;
the priority calculation module is used for calculating the data migration priority of each data unit according to the access state and the preset state migration diagram of each data unit corresponding to a continuous preset number of preset periods respectively; wherein, the state transition diagram is preset with the corresponding relation between each access state and each data transition priority;
and the data migration module is used for migrating the data units reaching the first priority threshold value in the solid state disk to the hard disk drive and migrating the data units reaching the second priority threshold value in the hard disk drive to the solid state disk.
7. The data migration apparatus of the storage system according to claim 6, wherein the data migration module comprises:
the erasing and writing frequency obtaining submodule is used for obtaining the erasing and writing frequency of each physical page address block in the solid state disk; each physical page address block is obtained by dividing the total physical page address of the solid state disk by taking one data storage unit as a unit;
the erasing and writing times sequencing submodule is used for sequencing the physical page address blocks according to the erasing and writing times to obtain a first sequencing result;
and the data migration submodule is used for migrating the data units reaching the second priority threshold value in the hard disk drive to each physical page address block in the solid state disk according to the priority from small to large in the first sequencing result.
8. The data migration apparatus according to claim 6 or 7, further comprising:
a data volume counting module, configured to count a first data volume formed by data units reaching the first priority threshold in the solid state disk and a second data volume formed by data units reaching the second priority threshold in the hard disk drive before migrating the data units reaching the first priority threshold in the hard disk drive to the hard disk drive and migrating the data units reaching the second priority threshold in the hard disk drive to the solid state disk;
the data volume comparison module is used for comparing the first data volume with the second data volume to obtain a comparison result;
the data migration module specifically migrates the data units in the solid state disk, which reach the first priority threshold, to the hard disk drive when the comparison result shows that the first data size is smaller than the second data size, and selects a part of data units, which have a data total amount smaller than or equal to the first data size, from the data units in the hard disk drive, which reach the second priority threshold, to migrate to the solid state disk; and when the comparison result shows that the first data volume is larger than the second data volume, migrating the data units reaching the second priority threshold value in the hard disk drive to the solid state disk, and selecting partial data units with the total data volume less than or equal to the second data volume from the data units reaching the first priority threshold value in the solid state disk to migrate to the module of the hard disk drive.
9. A data migration apparatus of a storage system, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the data migration method of the storage system according to any one of claims 1 to 5 when executing the computer program.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of a data migration method of a storage system according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910995298.7A CN110795034B (en) | 2019-10-18 | 2019-10-18 | Data migration method, device and equipment of storage system and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910995298.7A CN110795034B (en) | 2019-10-18 | 2019-10-18 | Data migration method, device and equipment of storage system and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110795034A true CN110795034A (en) | 2020-02-14 |
| CN110795034B CN110795034B (en) | 2022-07-22 |
Family
ID=69439421
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910995298.7A Active CN110795034B (en) | 2019-10-18 | 2019-10-18 | Data migration method, device and equipment of storage system and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110795034B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113741810A (en) * | 2021-07-30 | 2021-12-03 | 苏州浪潮智能科技有限公司 | Data migration method and device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090319743A1 (en) * | 2008-06-23 | 2009-12-24 | Kevin John Ash | Apparatus, system, and method for migrating wear spots |
| CN109976680A (en) * | 2019-04-11 | 2019-07-05 | 苏州浪潮智能科技有限公司 | A kind of data migration method, device, equipment and readable storage medium storing program for executing |
-
2019
- 2019-10-18 CN CN201910995298.7A patent/CN110795034B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090319743A1 (en) * | 2008-06-23 | 2009-12-24 | Kevin John Ash | Apparatus, system, and method for migrating wear spots |
| CN109976680A (en) * | 2019-04-11 | 2019-07-05 | 苏州浪潮智能科技有限公司 | A kind of data migration method, device, equipment and readable storage medium storing program for executing |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113741810A (en) * | 2021-07-30 | 2021-12-03 | 苏州浪潮智能科技有限公司 | Data migration method and device |
| CN113741810B (en) * | 2021-07-30 | 2023-08-11 | 苏州浪潮智能科技有限公司 | Data migration method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110795034B (en) | 2022-07-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107368429B (en) | Data storage device, memory controller, data management method thereof and data block management method | |
| CN110674056B (en) | Garbage recovery method and device | |
| Luojie et al. | An improved analytic expression for write amplification in NAND flash | |
| CN112684975B (en) | Data storage method and device | |
| CN110515549B (en) | Data migration method and device | |
| CN110968253A (en) | Data storage method, device and system | |
| CN104503703A (en) | Cache processing method and device | |
| CN109558456A (en) | A kind of file migration method, apparatus, equipment and readable storage medium storing program for executing | |
| WO2019083389A1 (en) | Managing a file system within multiple luns | |
| CN115576505A (en) | Data storage method, device and equipment and readable storage medium | |
| CN116737064A (en) | Data management method and system for solid state disk | |
| CN104750432A (en) | Data storage method and device | |
| CN107229421B (en) | Method and device for creating video data storage system, method and device for writing file into video data storage system and method and device for reading video data storage system | |
| CN106844491B (en) | Temporary data writing and reading method and device | |
| CN114968839A (en) | Hard disk garbage recycling method, device and equipment and computer readable storage medium | |
| CN110795034B (en) | Data migration method, device and equipment of storage system and readable storage medium | |
| CN109343796B (en) | Data processing method and device | |
| CN106020717B (en) | Data processing method and electronic equipment | |
| CN104408126B (en) | A kind of persistence wiring method of database, device and system | |
| CN105808451B (en) | Data caching method and related device | |
| EP2381354A2 (en) | Data recording device | |
| CN110286848B (en) | Data processing method and device | |
| CN112015343A (en) | Cache space management method and device of storage volume and electronic equipment | |
| CN110515562A (en) | A kind of disk access method and device | |
| CN107924350B (en) | Electronic equipment and data compression method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |