CN111143279B - Data migration method, device and equipment and readable storage medium - Google Patents

Data migration method, device and equipment and readable storage medium Download PDF

Info

Publication number
CN111143279B
CN111143279B CN201911386567.6A CN201911386567A CN111143279B CN 111143279 B CN111143279 B CN 111143279B CN 201911386567 A CN201911386567 A CN 201911386567A CN 111143279 B CN111143279 B CN 111143279B
Authority
CN
China
Prior art keywords
data
data migration
storage
access
tamper
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911386567.6A
Other languages
Chinese (zh)
Other versions
CN111143279A (en
Inventor
苏楠
张端
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Inspur Electronic Information Industry Co Ltd
Original Assignee
Inspur Electronic Information Industry Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Inspur Electronic Information Industry Co Ltd filed Critical Inspur Electronic Information Industry Co Ltd
Priority to CN201911386567.6A priority Critical patent/CN111143279B/en
Publication of CN111143279A publication Critical patent/CN111143279A/en
Application granted granted Critical
Publication of CN111143279B publication Critical patent/CN111143279B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/11File system administration, e.g. details of archiving or snapshots
    • G06F16/119Details of migration of file systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/16File or folder operations, e.g. details of user interfaces specifically adapted to file systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/182Distributed file systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a data migration method, a device, equipment and a readable storage medium, wherein the method comprises the following steps: receiving a file tamper-proof request, and setting write-once read-many attributes for a target directory corresponding to the file tamper-proof request; executing a tamper-resistant data migration flow when the target directory is within the protection period so as to migrate data among different storage media in the multi-plane storage system; and executing the common data migration flow when the target directory is in a delayed protection period or a non-protection period. Therefore, the method integrates the multi-plane storage systems of different storage media through data migration, can distribute data in a protection period on different media more reasonably, realizes all heterogeneous mixed storage pooling technologies of cluster nodes, manages and distributes data uniformly, reduces access delay of the whole system, and achieves optimal performance.

Description

Data migration method, device and equipment and readable storage medium
Technical Field
The present invention relates to the field of storage technologies, and in particular, to a data migration method, apparatus, device, and readable storage medium.
Background
When files of audit class or rules made by enterprises are filed, some information is required to be only read and not to be modified. It is therefore desirable to protect such files from being deleted or modified by anyone during the time of protection. The WORM (write once read many) characteristic can realize the function and effectively ensure the security of important data of an enterprise. The WORM function of the distributed file system sets important files for protection, prevents data in the protected files from being modified or deleted, is suitable for scenes preventing data tampering, such as medical images, and can realize filing and archiving of the files.
In a common distributed file storage system, the use of the functional attribute of file tamper-resistance takes a directory as a unit, the operable authority of a directory is divided into 3 stages (including a time-delay protection period, during which deletion, modification and check can be performed, a protection period, during which only check operation is supported and deletion or modification operation cannot be performed on a file, and a non-protection period, during which deletion, modification and check can be performed), and data corresponding to the directory is all stored in the same storage device in the 3 stages.
However, the data explosion-type growth and the increase of the user data access demand have more and more demand on storage capacity, and the common distributed file storage system has difficulty in meeting the user demand due to the use of the functional attribute of file tamper resistance.
In summary, how to effectively solve the problems of reducing the system access delay and the like while realizing the file tamper-proof function is a technical problem that needs to be solved urgently by those skilled in the art at present.
Disclosure of Invention
The invention aims to provide a data migration method, a data migration device, data migration equipment and a readable storage medium, so that the access delay of the whole system is reduced, and the optimal performance is achieved.
In order to solve the technical problems, the invention provides the following technical scheme:
a method of data migration, comprising:
receiving a file tamper-proof request, and setting write-once read-many attributes for a target directory corresponding to the file tamper-proof request;
executing a tamper-resistant data migration process when the target directory is within a protection period so as to migrate data among different storage media in the multi-plane storage system;
and executing a common data migration flow when the target directory is in a delayed protection period or a non-protection period.
Preferably, the storage medium of the multi-plane storage system includes an NVM, an SSD, and an HDD, and the executing the tamper-resistant data migration procedure includes:
counting the access amount corresponding to each data corresponding to the target directory;
monitoring storage capacities of the NVM, the SSD, and the HDD;
and migrating the data by using the access amount and the storage capacity.
Preferably, migrating data by using the access amount and the storage capacity includes:
for first data stored on the HDD, judging whether the access volume of the first data in a first time length is larger than a first access volume threshold value;
if so, judging whether the storage capacity of the NVM exceeds a first storage capacity alarm threshold value; if not, migrating the first data to the NVM; and if the first data exceeds the threshold value, the first data is migrated to the SSD when the storage capacity of the SSD is determined not to exceed a second storage capacity alarm threshold value.
Preferably, migrating data by using the access amount and the storage capacity includes:
for second data stored in the NVM, judging whether the access quantity of the second data in a second time length is larger than a second access quantity threshold value;
if not, migrating the second data to the HDD.
Preferably, migrating data by using the access amount and the storage capacity includes:
judging whether the access quantity of the second data in a third time length is greater than a third access quantity threshold value; wherein the third access amount threshold is greater than the second access amount threshold, and the third duration is greater than the second duration;
if not, migrating the second data to the SDD.
Preferably, migrating data by using the access amount and the storage capacity includes:
for third data stored on the SSD, judging whether the access quantity of the third data in a fourth time length is larger than a fourth access quantity threshold value;
if not, migrating the third data to the HDD.
Preferably, the storage medium of the multi-plane storage system comprises a DRAM, an NVM, an SSD and an HDD, and the NVM is taken as a write cache of the DRAM; a write operation writes directly to the DRAM; the SSD is a high-speed storage pool of persistent storage; the HDD is a slow storage pool of persistent storage
A data migration apparatus, comprising:
the system comprises a tamper-resistant attribute setting module, a file tamper-resistant request sending module and a write-once read-many attribute setting module, wherein the tamper-resistant attribute setting module is used for receiving a file tamper-resistant request and setting a write-once read-many attribute for a target directory corresponding to the file tamper-resistant request;
the first data migration module is used for executing a tamper-resistant data migration process when the target directory is within a protection period so as to migrate data among different storage media in the multi-plane storage system;
and the second data migration module is used for executing a common data migration process when the target directory is in a delayed protection period or a non-protection period.
A data migration apparatus comprising:
a memory for storing a computer program;
and the processor is used for realizing the steps of the data migration method when executing the computer program.
A readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned data migration method.
By applying the method provided by the embodiment of the invention, the file anti-tampering request is received, and the write-once read-many attribute is set for the target directory corresponding to the file anti-tampering request; executing a tamper-resistant data migration flow when the target directory is within the protection period so as to migrate data among different storage media in the multi-plane storage system; and executing the common data migration flow when the target directory is in a delayed protection period or a non-protection period.
In the embodiment, the data corresponding to the target directory is stored in the multi-plane storage system, that is, the data corresponding to the target directory can be stored in common storage media such as DRAM, NVM, SSD and HDD. In order to reduce the access delay of the whole system when the target directory is in the protection period so as to achieve the optimal performance, in the protection period of the data, the data is migrated among different storage media in the multi-plane storage system by executing the tamper-resistant data migration flow, so that the characteristics of various devices in the multi-plane storage system are reasonably utilized to store a large amount of data and guarantee the access speed of a user. Therefore, the method integrates the multi-plane storage systems of different storage media through data migration, can distribute data in a protection period on different media more reasonably, realizes all heterogeneous mixed storage pooling technologies of cluster nodes, manages and distributes data uniformly, reduces access delay of the whole system, and achieves optimal performance.
Accordingly, embodiments of the present invention further provide a data migration apparatus, a device, and a readable storage medium corresponding to the data migration method, which have the above technical effects and are not described herein again.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flow chart of a data migration method according to an embodiment of the present invention;
FIG. 2 is a flow chart illustrating HDD data migration according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a data migration apparatus according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a data migration apparatus according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a data migration device in an embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It is to be noted that relational terms such as first, second, third, fourth, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.
The first embodiment is as follows:
referring to fig. 1, fig. 1 is a flowchart illustrating a data migration method according to an embodiment of the present invention, the method including the following steps:
s101, receiving a file anti-tampering request, and setting write-once read-many attributes for a target directory corresponding to the file anti-tampering request.
The file anti-tampering request can be specifically an anti-tampering request provided when a user uploads a data file; the file tamper-resistant request can also be made for data files stored in the storage system (including ordinary data files and data files with the operable authority in an unprotected period).
After a file anti-tampering request is received, firstly, a data file requiring anti-tampering is determined, and then a target directory corresponding to the data file is found out. Then, a write once read many attribute (WORM attribute) is set to the target directory. Since the WORM property has the anti-tampering function, setting the WORM property can be regarded as setting the anti-tampering property. In particular, the tamper-resistant property may specifically comprise individual durations of the tamper-resistant cycle. For example, the delayed protection period is 1 day, the protection period is one month, and the protection period is the non-protection period.
It should be noted that, after a directory has a WORM attribute set, the subdirectories of the directory inherit the WORM attribute of the directory. If a WORM attribute is set for a directory whose parent directory has already been set with a WORM attribute, then the directory no longer inherits the parent directory, but instead executes in accordance with its own WORM attribute. For the underlying subdirectories, this may be performed in accordance with the WORM property set by the most recent directory.
S102, when the target directory is in the protection period, executing a tamper-proof data migration process so as to migrate data among different storage media in the multi-plane storage system.
Because the target directory is within the protection period, the user can only read the data corresponding to the target directory, but cannot delete and modify the data. In consideration of different characteristics corresponding to different storage media, in order to better utilize the corresponding characteristics of various storage media and reduce the access delay of the whole system, data can be migrated between different storage media in the multi-plane storage system, wherein the storage media of the multi-plane storage system can specifically include at least one memory storage medium and one non-memory storage medium in common storage media such as DRAM NVM, SSD and HDD.
Among them, dram (dynamic Random Access memory), i.e., dynamic Random Access memory, such as the most common system memory; nvm (nonvolatile memory), i.e. fixed memory, nonvolatile memory, memory that is not lost when power is lost, such as nonvolatile memory; ssd (solid State disk), a solid State disk, is a computer storage device that mainly uses Flash memory (NAND Flash) as a permanent memory; HDD (hard Disk drive), a computer hard Disk, may be comprised of one or more aluminum or glass disks.
Preferably, the storage medium of the multi-plane storage system includes DRAM, NVM, SSD and HDD. In order to give full play to the advantages of each storage device, the corresponding defects are avoided, and the data protection function is met. In an embodiment, the data viewing service is provided when the target directory is within the protection period; and providing data deletion access, modification service and viewing service when the target directory is in a delayed protection period or a non-protection period. The storage medium of the multi-plane storage system comprises a DRAM, an NVM, an SSD and an HDD, and the NVM is used as a write cache of the DRAM; write operations are written directly to the DRAM; the SSD is a high-speed storage pool of persistent storage; the HDD is a slow storage pool of persistent storage. Specifically, in the multi-plane storage system, a single-node local DRAM can be used as a primary storage medium, an NVM can be used as a secondary storage medium, the NVM can be used as a write cache of the DRAM, and write operation can be directly written into the DRAM as much as possible, so that the influence on the service life of the NVM is reduced under the condition of ensuring high-speed read-write performance; taking the SSD as a three-level storage medium and a high-speed storage pool for persistent storage, and taking the HDD as a four-level storage medium and a low-speed storage pool for persistent storage.
The storage media of the multi-plane storage system comprise an NVM, an SSD and an HDD, and an anti-tampering data migration process is executed, namely, data migration is carried out on data on different storage media when migration conditions are met. Executing a tamper-resistant data migration flow, comprising:
step one, counting the access amount corresponding to each data corresponding to a target directory;
monitoring the storage capacity of the NVM, the SSD and the HDD;
and step three, migrating the data by using the access amount and the storage capacity.
In this embodiment, data migration specifically refers to data migration between different storage media. When data migration is carried out between different media, the data transmission protocol and the bus corresponding to the two migrated media are followed.
Therefore, in the tamper-resistant data migration process, the access amount of the data and the storage space of the storage device can be mainly referred to. In practical applications, different migration conditions may be set for different storage media (the migration conditions may include specific access amounts and storage capacities), which may include but are not limited to the following examples:
example 1: for the data migration process on the HDD:
step 11, judging whether the access volume of the first data in the first time length is larger than a first access volume threshold value or not for the first data stored on the HDD;
step 12, if yes, judging whether the storage capacity of the NVM exceeds a first storage capacity alarm threshold value; if not, migrating the first data to the NVM; and if the first data exceeds the second storage capacity alarm threshold, the first data is migrated to the SSD when the storage capacity of the SSD is determined not to exceed the second storage capacity alarm threshold.
Specifically, referring to fig. 2, fig. 2 is a schematic diagram illustrating a HDD data migration process. In the present embodiment, for the sake of distinction, the data in the guard period stored on the HDD is referred to as first data. When determining whether the first data needs to be migrated, it may be determined whether the first data is hot data, where the hot data is data with a higher access heat and may be determined based on the access amount. That is, it may be determined whether the access amount of the first data in the first time period is greater than a first access amount threshold, and if so, it indicates that the first data belongs to hot data and needs to be migrated; if not, it indicates that the first data belongs to cold data, and no migration is required. The first time length can be set according to the requirements of users, for example, the first time length can be set to be 1 day; the first access amount threshold value can also be set according to a specific application scene, for example, in a scene where the access amount of data is generally high, the set number value is relatively high; of course, it is also possible to directly set the first access amount threshold to 50% of the average access heat of all data on the current HDD.
After determining that the first data needs to be migrated, it is further determined to which type of storage medium the first data needs to be migrated. Considering that the NVM has high speed read and write performance, it can be prioritized to determine whether the NVM has the capability of receiving the first data, i.e., to determine whether the NVM still has sufficient memory space. The first memory capacity warning threshold may be determined by determining whether the memory capacity of the NVM exceeds a first memory capacity warning threshold, where the first memory capacity warning threshold may be 40-70% of the total amount of NVM memory, and of course, the first memory capacity warning threshold may also be adjusted appropriately based on the total amount of NVM memory. If the storage capacity of the NVM does not exceed the first storage capacity alarm threshold, the NVM is considered to have data receiving capability, and the first data can be directly migrated to the NVM at this time. If the storage capacity of the NVM does not exceed the first storage capacity alarm threshold, the step of determining whether the SSD has the data receiving capability may be omitted, and the determining process is similar to the determining whether the NVM has the storage capability, which is not described herein again. The second storage capacity warning threshold may be set to 80% of the total storage amount of the SSD, and of course, the size of the first storage warning threshold may also be appropriately adjusted based on the total storage amount of the SSD.
When neither the SSD nor the NVM has data reception capability, then the first data may not need to be migrated.
Example 2: for a data migration process on NVM:
step 21, judging whether the access quantity of the second data in the second time length is larger than a second access quantity threshold value or not for the second data stored in the NVM;
and step 22, if not, migrating the second data to the HDD.
For the sake of distinction, in the present embodiment, data stored in the guard period on the NVM is referred to as second data.
This may cause the second data stored on the NVM to be cold data, considering that the heat of the data may change over time. To improve the data access efficiency of the system, cold data may be migrated at this time to make room for storage of other hot data. Wherein, the second time period can be set as 1 day, and can be adjusted according to the data cold and hot change frequency. The second threshold may also be set to the average heat of all data on the NVM, although this value may also be adjustable.
Example 3: considering that the speed is slower when the HDD persists data, in some special cases, the second data can be migrated to the SSD, which can speed up the data persistence, i.e. data migration.
The specific implementation process can include:
step 31, judging whether the access quantity of the second data in the third time length is greater than a third access quantity threshold value; the third visit volume threshold is greater than the second visit volume threshold, and the third duration is greater than the second duration;
and 31, if not, migrating the second data to the SDD.
For example, if the second duration is 3 days, the third duration may be specifically one week; if the third threshold amount of access is 40% of the average heat of data on the third threshold amount of access NVM, the third threshold amount of access may be 60% of the average heat of data on the NVM.
Example 4: for data migration on an SSD, comprising:
step 41, for the third data stored on the SSD, determining whether the access amount of the third data in the fourth duration is greater than a fourth access amount threshold;
and 42, if not, migrating the third data to the HDD.
For the sake of distinction, the data stored on the SSD and in the protection period is referred to as third data in the present embodiment. In this embodiment, a fourth time period may be set, which may be 3 days (adaptive adjustment); the fourth threshold may be set to 40% of the average access on the SSD, and the third data may be migrated to the HDD when the access for the third data for the fourth duration is less than the fourth threshold, indicating that the third data is cooling. Of course, when the third data is migrated to the HDD, it may be determined whether the HDD has the migration condition.
It should be noted that, in addition to the migration of the migration conditions of the above example, other examples may also be derived based on the above example, and are not described in detail here. Specifically, under the condition that the storage capacity control is met (that is, when the storage capacity threshold is reached, data is not migrated inwards), the higher the data heat degree is, the data can be migrated according to the classification of the storage device, and after the heat degree of the storage data at the current stage is changed, if the data is heated, the data is migrated to the upper stage; if the temperature is cold, the system moves to the next stage. When the heat change of the stored data reaches the migration condition at the current stage, but the upper or lower stage has no capability of receiving data (if the available storage space is too low), the data can not be migrated temporarily. Of course, in the data migration process, the migration can be performed step by step, and also can be performed step by step.
S103, executing a common data migration process when the target directory is in a delayed protection period or a non-protection period.
When the target directory is in the delayed protection period or the non-protection period, the user can delete, modify and read the data at the moment. Therefore, in order to guarantee data consistency, a normal data migration flow may be executed at this time. Specifically, not only the current access amount and the storage capacity of various storage media, but also the access situation of the user need to be concerned in the migration process. For example, when data is modified, no data migration is performed. That is, in this case, the data migration process may refer to a normal data migration method. The described common data migration method is a migration method capable of satisfying data reliability in a delayed protection period and a non-protection period, which is different from the above tamper-proof data migration procedure.
By applying the method provided by the embodiment of the invention, the file anti-tampering request is received, and the write-once read-many attribute is set for the target directory corresponding to the file anti-tampering request; executing a tamper-resistant data migration flow when the target directory is within the protection period so as to migrate data among different storage media in the multi-plane storage system; and executing the common data migration flow when the target directory is in a delayed protection period or a non-protection period.
In the embodiment, the data corresponding to the target directory is stored in the multi-plane storage system, that is, the data corresponding to the target directory can be stored in common storage media such as DRAM, NVM, SSD and HDD. In order to reduce the access delay of the whole system when the target directory is in the protection period so as to achieve the optimal performance, in the protection period of the data, the data is migrated among different storage media in the multi-plane storage system by executing the tamper-resistant data migration flow, so that the characteristics of various devices in the multi-plane storage system are reasonably utilized to store a large amount of data and guarantee the access speed of a user. Therefore, the method integrates the multi-plane storage systems of different storage media through data migration, can distribute data in a protection period on different media more reasonably, realizes all heterogeneous mixed storage pooling technologies of cluster nodes, manages and distributes data uniformly, reduces access delay of the whole system, and achieves optimal performance.
Example two:
corresponding to the above method embodiments, the embodiments of the present invention further provide a data migration apparatus, and the data migration apparatus described below and the data migration method described above may be referred to correspondingly.
Referring to fig. 3, the apparatus includes the following modules:
the tamper-resistant attribute setting module 101 is configured to receive a file tamper-resistant request, and set a write-once read-many attribute for a target directory corresponding to the file tamper-resistant request;
the first data migration module 102 is configured to execute a tamper-resistant data migration procedure when the target directory is within the protection period, so as to migrate data between different storage media in the multi-plane storage system;
the second data migration module 103 is configured to execute a normal data migration procedure when the target directory is in a delayed protection period or a non-protection period.
The device provided by the embodiment of the invention is applied to receive the file anti-tampering request and set the write-once read-many attribute for the target directory corresponding to the file anti-tampering request; executing a tamper-resistant data migration flow when the target directory is within the protection period so as to migrate data among the DRAM, the NVM, the SSD and the HDD in the multi-plane storage system; and executing the common data migration flow when the target directory is in a delayed protection period or a non-protection period.
In the embodiment, the data corresponding to the target directory is stored in the multi-plane storage system, that is, the data corresponding to the target directory can be stored in common storage media such as DRAM, NVM, SSD and HDD. In order to reduce the access delay of the whole system when the target directory is in the protection period so as to achieve the optimal performance, in the protection period of the data, the data is migrated among different storage media in the multi-plane storage system by executing the tamper-resistant data migration flow, so that the characteristics of various devices in the multi-plane storage system are reasonably utilized to store a large amount of data and guarantee the access speed of a user. Therefore, the method integrates the multi-plane storage systems of different storage media through data migration, can distribute data in a protection period on different media more reasonably, realizes all heterogeneous mixed storage pooling technologies of cluster nodes, manages and distributes data uniformly, reduces access delay of the whole system, and achieves optimal performance.
In a specific embodiment of the present invention, the storage medium of the multi-plane storage system includes an NVM, an SSD and an HDD, and the first data migration module 102 includes:
the access amount monitoring unit is used for counting the access amount corresponding to each data corresponding to the target directory;
a storage capacity monitoring unit for monitoring storage capacities of the NVM, the SSD, and the HDD;
and the data migration unit is used for migrating the data by using the access amount and the storage capacity.
In a specific embodiment of the present invention, the data migration unit is configured to, for first data stored on the HDD, determine whether an access amount of the first data within a first time period is greater than a first access amount threshold; if so, judging whether the storage capacity of the NVM exceeds a first storage capacity alarm threshold value; if not, migrating the first data to the NVM; and if the first data exceeds the second storage capacity alarm threshold, the first data is migrated to the SSD when the storage capacity of the SSD is determined not to exceed the second storage capacity alarm threshold.
In a specific embodiment of the present invention, the data migration unit is configured to, for second data stored in the NVM, determine whether an access amount of the second data in a second duration is greater than a second access amount threshold;
if not, the second data is migrated to the HDD.
In a specific embodiment of the present invention, the data migration unit is configured to determine whether an access amount of the second data in a third duration is greater than a third access amount threshold; the third visit volume threshold is greater than the second visit volume threshold, and the third duration is greater than the second duration;
if not, migrating the second data to the SDD.
In a specific embodiment of the present invention, the data migration unit is configured to, for third data stored on the SSD, determine whether an access amount of the third data in a fourth duration is greater than a fourth access amount threshold;
if not, the third data is migrated to the HDD.
In a specific embodiment of the present invention, the storage medium of the multi-plane storage system includes a DRAM, an NVM, an SSD, and an HDD, and the NVM is used as a write cache of the DRAM; write operations are written directly to the DRAM; the SSD is a high-speed storage pool of persistent storage; the HDD is a slow storage pool of persistent storage.
Example three:
corresponding to the above method embodiment, an embodiment of the present invention further provides a data migration device, and a data migration device described below and a data migration method described above may be referred to in correspondence.
Referring to fig. 4, the data migration apparatus includes:
a memory D1 for storing computer programs;
a processor D2, configured to implement the steps of the data migration method of the above-described method embodiments when executing the computer program.
Specifically, referring to fig. 5, a specific structural diagram of a data migration device provided in this embodiment is shown, where the data migration device may generate a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 322 (e.g., one or more processors) and a memory 332, and one or more storage media 330 (e.g., one or more mass storage devices) storing an application 342 or data 344. Memory 332 and storage media 330 may be, among other things, transient storage or persistent storage. The program stored on the storage medium 330 may include one or more modules (not shown), each of which may include a series of instructions operating on the data migration apparatus. Still further, the central processor 322 may be configured to communicate with the storage medium 330 to execute a series of instruction operations in the storage medium 330 on the data migration apparatus 301.
The data migration apparatus 301 may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input-output interfaces 358, and/or one or more operating systems 341. Such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
The steps in the data migration method described above may be implemented by the structure of the data migration apparatus.
Example four:
corresponding to the above method embodiment, the embodiment of the present invention further provides a readable storage medium, and a readable storage medium described below and a data migration method described above may be referred to in correspondence with each other.
A readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the data migration method of the above-mentioned method embodiments.
The readable storage medium may be a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and various other readable storage media capable of storing program codes.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

Claims (10)

1. A method of data migration, comprising:
receiving a file tamper-proof request, and setting write-once read-many attributes for a target directory corresponding to the file tamper-proof request;
executing a tamper-resistant data migration process when the target directory is within a protection period so as to migrate data among different storage media in the multi-plane storage system;
and executing a common data migration flow when the target directory is in a delayed protection period or a non-protection period.
2. The data migration method according to claim 1, wherein the storage medium of the multi-plane storage system includes an NVM, an SSD and an HDD, and the performing the tamper-resistant data migration procedure includes:
counting the access amount corresponding to each data corresponding to the target directory;
monitoring storage capacities of the NVM, the SSD, and the HDD;
and migrating the data by using the access amount and the storage capacity.
3. The data migration method according to claim 2, wherein migrating data using the access amount and the storage capacity comprises:
for first data stored on the HDD, judging whether the access volume of the first data in a first time length is larger than a first access volume threshold value;
if so, judging whether the storage capacity of the NVM exceeds a first storage capacity alarm threshold value; if not, migrating the first data to the NVM; and if the first data exceeds the threshold value, the first data is migrated to the SSD when the storage capacity of the SSD is determined not to exceed a second storage capacity alarm threshold value.
4. The data migration method according to claim 2, wherein migrating data using the access amount and the storage capacity comprises:
for second data stored in the NVM, judging whether the access quantity of the second data in a second time length is larger than a second access quantity threshold value;
if not, migrating the second data to the HDD.
5. The data migration method according to claim 4, wherein migrating data using the access amount and the storage capacity comprises:
judging whether the access quantity of the second data in a third time length is greater than a third access quantity threshold value; wherein the third access amount threshold is greater than the second access amount threshold, and the third duration is greater than the second duration;
if not, migrating the second data to the SSD.
6. The data migration method according to claim 2, wherein migrating data using the access amount and the storage capacity comprises:
for third data stored on the SSD, judging whether the access quantity of the third data in a fourth time length is larger than a fourth access quantity threshold value;
if not, migrating the third data to the HDD.
7. The data migration method according to any one of claims 1 to 6, wherein the storage medium of the multi-plane storage system comprises a DRAM, an NVM, an SSD and an HDD, and the NVM is used as a write cache of the DRAM; a write operation writes directly to the DRAM; the SSD is a high-speed storage pool of persistent storage; the HDD is a slow storage pool of persistent storage.
8. A data migration apparatus, comprising:
the system comprises a tamper-resistant attribute setting module, a file tamper-resistant request sending module and a write-once read-many attribute setting module, wherein the tamper-resistant attribute setting module is used for receiving a file tamper-resistant request and setting a write-once read-many attribute for a target directory corresponding to the file tamper-resistant request;
the first data migration module is used for executing a tamper-resistant data migration process when the target directory is within a protection period so as to migrate data among different storage media in the multi-plane storage system;
and the second data migration module is used for executing a common data migration process when the target directory is in a delayed protection period or a non-protection period.
9. A data migration apparatus, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the data migration method according to any one of claims 1 to 7 when executing the computer program.
10. A readable storage medium, characterized in that the readable storage medium has stored thereon a computer program which, when being executed by a processor, carries out the steps of the data migration method according to any one of claims 1 to 7.
CN201911386567.6A 2019-12-29 2019-12-29 Data migration method, device and equipment and readable storage medium Active CN111143279B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911386567.6A CN111143279B (en) 2019-12-29 2019-12-29 Data migration method, device and equipment and readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911386567.6A CN111143279B (en) 2019-12-29 2019-12-29 Data migration method, device and equipment and readable storage medium

Publications (2)

Publication Number Publication Date
CN111143279A CN111143279A (en) 2020-05-12
CN111143279B true CN111143279B (en) 2022-04-22

Family

ID=70521461

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911386567.6A Active CN111143279B (en) 2019-12-29 2019-12-29 Data migration method, device and equipment and readable storage medium

Country Status (1)

Country Link
CN (1) CN111143279B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20220055034A (en) 2020-10-26 2022-05-03 삼성전자주식회사 Method of operating storage device and method of operating storage system using the same
CN114528086B (en) * 2022-02-26 2024-06-07 苏州浪潮智能科技有限公司 Method, device, equipment and medium for thermomigration of pooled heterogeneous cloud computing application
CN117666936A (en) * 2022-08-31 2024-03-08 华为技术有限公司 Migration bandwidth adjustment method, device and system of storage medium and chip

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009001539A1 (en) * 2007-06-25 2008-12-31 Panasonic Corporation Information security device and information security system
WO2016107442A1 (en) * 2014-12-31 2016-07-07 华为技术有限公司 Method for writing data to solid state drive and solid state drive
CN107844269A (en) * 2017-10-17 2018-03-27 华中科技大学 A kind of layering mixing storage system and method based on uniformity Hash
CN109002263A (en) * 2018-07-20 2018-12-14 腾讯科技(深圳)有限公司 The method of adjustment and device of memory capacity
CN110413590A (en) * 2019-07-24 2019-11-05 北京百度网讯科技有限公司 Data migration method, device, equipment and medium

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5878360B2 (en) * 2011-12-19 2016-03-08 インターナショナル・ビジネス・マシーンズ・コーポレーションInternational Business Machines Corporation Method for detecting tampering during data transfer to media and storage device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009001539A1 (en) * 2007-06-25 2008-12-31 Panasonic Corporation Information security device and information security system
WO2016107442A1 (en) * 2014-12-31 2016-07-07 华为技术有限公司 Method for writing data to solid state drive and solid state drive
CN107844269A (en) * 2017-10-17 2018-03-27 华中科技大学 A kind of layering mixing storage system and method based on uniformity Hash
CN109002263A (en) * 2018-07-20 2018-12-14 腾讯科技(深圳)有限公司 The method of adjustment and device of memory capacity
CN110413590A (en) * 2019-07-24 2019-11-05 北京百度网讯科技有限公司 Data migration method, device, equipment and medium

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
"A construction method and data migration strategy for hybrid cloud storage";Zhiying Duan等;《 2015 18th International Conference on Computer and Information Technology (ICCIT)》;20160609;第1-6页 *
"基于TPM的文件保护系统的研究与实现";张静;《中国优秀博硕士学位论文全文数据库(硕士) 信息科技辑》;20080515;参照第四章 *

Also Published As

Publication number Publication date
CN111143279A (en) 2020-05-12

Similar Documents

Publication Publication Date Title
CN111143279B (en) Data migration method, device and equipment and readable storage medium
US11385840B2 (en) Multi-tiered data storage with archival blockchains
US11061620B2 (en) Bandwidth limiting in solid state drives
CN104679661B (en) hybrid storage control method and hybrid storage system
US9355112B1 (en) Optimizing compression based on data activity
US8856484B2 (en) Mass storage system and methods of controlling resources thereof
US20110010514A1 (en) Adjusting Location of Tiered Storage Residence Based on Usage Patterns
US9703500B2 (en) Reducing power consumption by migration of data within a tiered storage system
US20160132580A1 (en) Automatic adjustment of data replication based on data access
US8533417B2 (en) Method and apparatus for controlling data volume creation in data storage system with dynamic chunk allocation capability
US8560801B1 (en) Tiering aware data defragmentation
CN106406759B (en) Data storage method and device
US10382751B1 (en) Transparent, fine-grained and adaptive data compression
EP4020153A1 (en) Cache space management method and device
CN108089825B (en) Storage system based on distributed cluster
CN111857592A (en) Data storage method and device based on object storage system and electronic equipment
JP2007241539A (en) Data management and control system for semiconductor flash memory, and semiconductor flash memory housing device
WO2016148738A1 (en) File management
CN107422989A (en) A kind of more copy read methods of Server SAN systems and storage architecture
CN111124304B (en) Data migration method and device, electronic equipment and storage medium
JP4526337B2 (en) Data management system and method
US10585592B2 (en) Disk area isolation method and device
CN117472285A (en) Intelligent operation acceleration method for solid state disk, computer equipment and storage medium
US11907551B2 (en) Performance efficient and resilient creation of network attached storage objects
US20190369882A1 (en) Techniques for preserving an expected lifespan of a non-volatile memory

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant