CN106569732B - Data migration method and device - Google Patents
Data migration method and device Download PDFInfo
- Publication number
- CN106569732B CN106569732B CN201510655794.XA CN201510655794A CN106569732B CN 106569732 B CN106569732 B CN 106569732B CN 201510655794 A CN201510655794 A CN 201510655794A CN 106569732 B CN106569732 B CN 106569732B
- Authority
- CN
- China
- Prior art keywords
- data
- hierarchical storage
- migrated
- cache
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
The invention provides a data migration method and a data migration device, which are used for solving the problems that the number of times of reading a hard disk is large and the internal performance of a system is reduced in the traditional hierarchical storage data migration process, wherein the data migration method comprises the following steps: the Cache module of the Cache memory obtains the address space of the hot spot data to be migrated from the hierarchical storage module; the Cache module writes hot spot data to be migrated according to the address space; the Cache module informs the address space of the hot spot data to be migrated in the Cache module to the hierarchical storage module, and the scheme improves the performance of the storage system.
Description
Technical Field
The present invention relates to the field of communications, and in particular, to a data migration method and apparatus.
Background
With the advancement of hard disk technology, the types and number of storage media supported by storage systems have increased dramatically. The storage system supports storage media comprising: SSD (Solid state drive), SAS (Serial Attached SCSI) hard disk, and nl (near line) SAS hard disk. The allocation of storage resources (SSD, SAS, NL _ SAS) is a task with high repeatability and long time consumption, it is difficult to balance the performance requirement and the capacity requirement of the storage system through the early planning, even if the balance is achieved at the early stage, the performance and capacity requirements will change, and the allocation of the resources of the storage system needs to be continuously adjusted, so the storage system in the industry generally supports the hierarchical storage function, which aims to improve the overall performance of the storage system and reduce the user cost through counting the storage positions of the active-level intelligent management data of the data, wherein the user cost generally includes the costs of equipment purchase, space, energy consumption and management. The resources are automatically allocated through the hierarchical storage, the difficulty of early planning can be greatly reduced, and the later maintenance and expansion become easier.
At present, the implementation of hierarchical storage in the industry is basically performed through three stages of statistics, analysis and migration, that is, through statistics of access frequency of data in a certain area, whether the data is hot data is periodically analyzed, and then migration from one data space to another data space is implemented through a data migration means. The migration mode is generally realized by reading data to be migrated from a hard disk and then writing the data to a destination hard disk address to be migrated by a background program in the storage system for data relocation without affecting host IO. Specifically, for a storage system, since hierarchical storage is located under the Cache, a conventional flow is shown in fig. 1, and hot spot migration of the hierarchical storage is a read-write operation of a hierarchical storage module on a hot spot area of a hard disk. The hierarchical storage module counts hot areas of the hard disk, the migration process periodically carries out data migration on the hot areas, generally, the migration process firstly reads a section of data from a disk to be migrated, the read data is stored through temporary memory resources, then the hierarchical storage module writes the data to a target hard disk address space to be migrated, the temporary memory resources are released after the data are written, and thus, after a normal migration action is completed, the migration process can involve disk reading once, and disk reading and disk writing once are applied. Meanwhile, considering that the granularity of the hot spot statistical area in the storage system is not matched with the granularity of one relocation, as shown in fig. 2, the hot spot statistical granularity (usually, the granularity of mapping from an upper-layer logical volume address to a hard disk address) is usually greater than the granularity of one relocation, so that the mapping from the logical volume address to the hard disk address can be modified only after all data in the hot spot area are relocated. Therefore, before the data in the hot spot area is not moved, if the write IO accesses the moved part of the data content at the moment, the hierarchical storage processing needs to write two data at the same time, one data is written in the original old hard disk area, and the other data is written in the new hard disk area, so that the purpose of ensuring the data consistency is achieved, the process is a double-write process of the hierarchical storage module, and the overview is the traditional data moving process of the hierarchical storage. Therefore, the number of times of reading the hard disk in the traditional hierarchical storage data migration process is large, and the internal performance of the storage system is reduced.
Disclosure of Invention
The invention provides a data migration method and a data migration device, which are used for solving the problems that the number of times of reading a hard disk is large and the internal performance of a system is reduced in the traditional hierarchical storage data migration process.
According to a first aspect of the present invention, there is provided a data migration method, comprising: the Cache module of the Cache memory obtains the address space of the hot spot data to be migrated from the hierarchical storage module; the Cache module writes hot spot data to be migrated according to the address space; and the Cache module informs the address space of the hot spot data to be migrated in the Cache module to the hierarchical storage module.
The Cache module writes hot spot data to be migrated according to the address space, and the method comprises the following steps: when the address spaces of all the hot spot data to be migrated are not found in the Cache module, the Cache module issues a pre-reading request to the hierarchical storage module; and the Cache module receives the hot spot data to be migrated sent by the hierarchical storage module and writes the received hot spot data into the Cache module.
According to a second aspect of the present invention, there is provided a data migration method comprising: the hierarchical storage module informs the address space of the hot spot data to be migrated to a Cache module of a Cache memory; after acquiring the address space of the data to be migrated in the Cache module, the hierarchical storage module writes the hot spot data into the address space of a new medium; and the hierarchical storage module modifies the corresponding relation between the logical volume and the medium mapping table according to the address space of the new medium.
After learning the address space of the data to be migrated in the Cache module, the hierarchical storage module writes the hot data into the address space of a new medium, and the method comprises the following steps: when a hotspot data access request forwarded by a Cache module is received, judging whether data required to be read by the request is migrated data according to the migration progress of the hotspot data to obtain a judgment result; and reading the data according to the judgment result, and informing the Cache module of finishing the data reading after the data reading is finished.
Further, the method further comprises: before informing the Cache module of the address space of the hot data to be migrated, the hierarchical storage module performs hot data statistics on the hard disk address space of the level where the logical volume LUN is located according to the hierarchical storage statistical granularity; and the hierarchical storage module analyzes the space address of the hot spot data to be migrated according to the statistical data.
According to a third aspect of the present invention, there is provided a data migration apparatus comprising: the learning unit is used for learning the address space of the hot spot data to be migrated from the hierarchical storage module through the Cache module of the Cache memory; the first writing unit is used for writing the hot spot data to be migrated into the Cache module according to the address space; and the notification unit is used for notifying the address space of the hot spot data to be migrated in the Cache module to the hierarchical storage module through the Cache module.
Wherein the first writing unit includes: the read-ahead subunit is used for issuing a read-ahead request to the hierarchical storage module by the Cache module under the condition that address spaces of all the hot spot data to be migrated are not found in the Cache module; and the write-in subunit is used for receiving the hot spot data to be migrated sent by the hierarchical storage module through the Cache module and writing the received hot spot data into the Cache module.
According to a fourth aspect of the present invention, there is provided a data migration apparatus comprising: the informing unit is used for informing the address space of the hot spot data to be migrated to the Cache module of the Cache memory through the hierarchical storage module; the second writing unit is used for writing the hot data into the address space of a new medium after the address space of the data to be migrated in the Cache module is obtained through the hierarchical storage module; and the modifying unit is used for modifying the corresponding relation between the logical volume and the medium mapping table according to the address space of the new medium by the hierarchical storage module.
Wherein the second writing unit includes: the judging subunit is used for judging whether the data required to be read by the request is migrated data or not according to the migration progress of the hot data when receiving the hot data access request forwarded by the Cache module, and obtaining a judgment result; and the notification subunit is used for reading the data according to the judgment result and notifying the Cache module that the data reading is finished after the data reading is finished.
Further, the above apparatus further comprises: the statistical unit is used for carrying out hot data statistics on the hard disk address space of the level where the logical volume LUN is located according to the hierarchical storage statistical granularity by the hierarchical storage module before informing the Cache module of the address space of the hot data to be migrated; and the analysis unit is used for analyzing the space address of the hot spot data to be migrated by the hierarchical storage module according to the statistical data.
The invention has the following beneficial effects:
according to the scheme provided by the embodiment of the invention, one disk reading operation and one memory occupation can be reduced in the data migration process, and only one disk writing operation is involved in the disk refreshing migration process, so that the internal performance of the storage system is improved.
Drawings
FIG. 1 is a flow diagram of hot spot data migration for tiered storage in the prior art;
FIG. 2 is a flow diagram of a double write process in a hot spot data migration process for hierarchical storage in the prior art;
fig. 3 is a flowchart of a data migration method provided in embodiment 1 of the present invention;
FIG. 4 is a basic schematic diagram of the Cache in embodiment 1 of the present invention;
FIG. 5 is a flowchart of a data migration method provided in embodiment 2 of the present invention;
FIG. 6 is a data migration flowchart in the case where all migration data are hit in the Cache in embodiment 3 of the present invention;
FIG. 7 is a flowchart of data migration in the case where all the migration data is missed by Cache in embodiment 3 of the present invention;
fig. 8 is a data migration flowchart in the case where a migration area is accessed by write IO in the data migration process in embodiment 3 of the present invention;
fig. 9 is a data migration flowchart in the case where there is a read IO access migration area during data migration in embodiment 3 of the present invention;
fig. 10 is a block diagram of a data migration apparatus according to embodiment 4 of the present invention;
fig. 11 is a block diagram of a data migration apparatus according to embodiment 5 of the present invention.
Detailed Description
In order to solve the problems that the number of times of reading a hard disk is large and the internal performance of a system is reduced in the traditional hierarchical storage data migration process, the invention provides a data migration method and a data migration device, and the invention is further described in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the invention.
Example 1
This embodiment provides a data migration method, which is applied to a Cache module, that is, implemented by a Cache module, fig. 3 is a flowchart of a data migration method provided in embodiment 1 of the present invention, and as shown in fig. 3, the method includes the following steps:
step 301: the Cache module acquires the address space of the hot spot data to be migrated from the hierarchical storage module;
step 302: the Cache module writes hot spot data to be migrated according to the address space;
step 303: and the Cache module informs the address space of the hot spot data to be migrated in the Cache module to the hierarchical storage module.
The step of writing the hot spot data to be migrated into the Cache module according to the address space may specifically include:
when the address spaces of all the hot spot data to be migrated are not found in the Cache module, the Cache module issues a pre-reading request to the hierarchical storage module; and the Cache module receives the hot spot data to be migrated sent by the hierarchical storage module and writes the received hot spot data into the Cache module.
As shown in fig. 4, first, a data space LUN accessed by a host is represented as a tree in a storage system Cache, and the Cache functions to manage the tree to some extent, and the access of a host IO is actually the access of the tree, such as hit and read-ahead of the tree. Meanwhile, the Cache also performs operations such as elimination, data scrubbing and the like on the tree. Therefore, feedback of access of the Cache to the host IO is realized. Once the host IO reaches the Cache stage of the storage system, the host IO can be resident in the Cache for a period of time, so that the Cache hit effect is achieved, and the Cache is significant.
In this embodiment, the migration action of the original hierarchical storage module is handed over to the cache module to be implemented, the cache module is decomposed into a disk-flushing operation, and the disk-flushing operation of the cache module triggers the single-writing of the hierarchical storage migration module, so that the effect of data migration is achieved. Based on this, the Cache module can utilize the existing Cache resources, if hit, one disk reading operation and one temporary memory occupation can be reduced, the data in the Cache module is directly printed to the hierarchical storage module, the data is written to the destination address through the hierarchical storage module, and at this time, the data is only written to the destination address without being written to the original medium address. Thus, write once operations are reduced. If the data is not hit, the Cache module needs to pre-read the data to the hierarchical storage module, and then writes the data to the destination address in a disk-flushing mode, at this time, as with the conventional process, a disk-reading operation is needed once, but the use of the temporary memory is also advantageous, and meanwhile, the hit rate of the Cache is increased by the pre-read page. In this embodiment, the whole data migration process is completed by cooperation between the hierarchical storage module and the cache module, and the migration operation of the hierarchical storage is replaced by the disk-flushing operation of the cache, so that one-time disk reading and memory occupation in the migration process are reduced.
Example 2
This embodiment provides a data migration method, which is applied to a hierarchical storage module side and implemented by a hierarchical storage module, and fig. 5 is a flowchart of a data migration method provided in embodiment 2 of the present invention, and as shown in fig. 5, the method includes the following steps:
step 501: the hierarchical storage module informs the address space of the hot spot data to be migrated to the Cache module;
step 502: after acquiring the address space of the data to be migrated in the Cache module, the hierarchical storage module writes the hot spot data into the address space of a new medium;
step 503: and the hierarchical storage module modifies the corresponding relation between the logical volume and the medium mapping table according to the address space of the new medium.
After learning the address space of the data to be migrated in the Cache module, the hierarchical storage module writes the hot data into the address space of the new medium, which may specifically include:
when a hotspot data access request forwarded by a Cache module is received, judging whether the data requested to be accessed is migrated according to the migration progress of the hotspot data to obtain a judgment result; and reading the data according to the judgment result, and after the data is completely read, informing the Cache module that the data is completely read.
Further, the method may further include:
before informing the address space of the hot data to be migrated to the Cache module, the hierarchical storage module performs hot data statistics on the hard disk address space of the level where the logical volume LUN is located according to the hierarchical storage statistics granularity; and the hierarchical storage module analyzes the space address of the hot spot data to be migrated according to the statistical data.
In this embodiment, the statistics of hot spots in the hierarchical storage is the access frequency to a certain address space on the disk, and the Cache view in the storage system mainly looks like the access frequency of the Cache dirty data to the disk and the Cache write-through to the disk. At this time, if a certain address space on the disk is counted as hot spot data, the hot spot data is caused by the Cache flushing the address space or the host frequently writes through the address space. And these hot spot data may also reside in the Cache at this time. Once the hot spot data is not eliminated in the Cache, the hot spot data can be written to a hard disk area with new performance through the hierarchical storage function module. However, if it is determined that the hot spot data is not in the Cache, the Cache needs to pre-read the part of the missed data through the hierarchical storage module, and then write the missed data into a new performance hard disk area through the hierarchical storage function module, and meanwhile, if the storage system is in a hot spot caused by a write-through state at this time, the pre-read data just meets the hit of the Cache, so that the performance of the system is improved.
And the hierarchical storage module receives hot spot data flushed by the Cache, and determines to write the Cache data to a new medium address area according to the migration progress mark. Because the hot spot statistics need to be always larger than the migration granularity, before the hot spot area data is not migrated, the hierarchical storage is a mapping table which cannot modify the address of the logical volume to the disk address, and only after the hot spot data is migrated, the address mapping table from the logical volume to the disk is modified. Thus, the next time the IO accesses the logical volume area, the area of the new medium is accessed. Therefore, before the hot spot data is not migrated, if a read request accesses a hot spot area, if the read request data is in the Cache, the Cache is directly accessed to return, if the request data is not in the Cache or is partially in the Cache, the Cache is required to issue a pre-read request to a hierarchical storage module, and the hierarchical storage module judges whether to read old medium data or new medium data through migrating bitmap information, so that the purpose of data access is achieved.
Example 3
In this embodiment, a flow of cooperatively implementing data migration between a Cache and a hierarchical storage module is described according to four conditions, namely migration when a migration data Cache is hit, migration when a migration data Cache is not hit, a migration area with write IO access in a migration process, and a migration area with read IO access in a migration process:
in the data migration process, the data migration process under the condition that all the migration data are hit in the Cache is as follows:
as shown in fig. 6, the method specifically includes the following steps:
the hierarchical storage function module carries out hot spot data statistics on the address space of the hierarchical hard disk where the LUN is located according to the hierarchical storage statistic granularity;
the hierarchical storage function module analyzes which address areas are hot areas according to the granularity statistical data;
the hierarchical storage function module informs the Cache module of which addresses need to be subjected to hotspot migration;
the Cache function module performs data disk writing operation according to an address space provided by the hierarchical storage, and at the moment, a disk writing address area is just hit in the Cache;
the Cache flushes the address space requested by the hierarchical storage module;
the hierarchical storage writes data to the address space of a new medium according to a disk writing request issued by the Cache;
the hierarchical storage module modifies the logical volume to media mapping table to a new media address space.
In the data migration process, under the condition that Cache misses all the migration data, the data migration flow comprises the following steps:
as shown in fig. 7, the method specifically includes the following steps:
the hierarchical storage function module carries out hot spot data statistics on the address space of the hierarchical hard disk where the LUN is located according to the hierarchical storage statistic granularity;
the hierarchical storage function module analyzes which address areas are hot areas according to the granularity statistical data;
the hierarchical storage function module informs the Cache of which addresses need to be subjected to hot spot migration;
the Cache function module performs data disk writing operation according to an address space provided by hierarchical storage, wherein a disk writing address area is not in the Cache or is partially in the Cache;
the Cache issues a pre-reading request to the hierarchical storage module, and the hierarchical storage module pre-reads data and returns the data to the Cache;
the Cache continues to issue an address space required by the hierarchical storage to the hierarchical storage module;
the hierarchical storage writes data to the address space of a new medium according to a disk writing request issued by the Cache;
the hierarchical storage module modifies the logical volume to media mapping table to a new media address space.
In the data migration process, under the condition that write IO accesses the migration area:
as shown in fig. 8, the method specifically includes the following steps:
the hierarchical storage function module carries out hot spot data statistics on the address space of the hierarchical hard disk where the LUN is located according to the hierarchical storage statistic granularity;
the hierarchical storage function module analyzes which address areas are hot areas according to the granularity statistical data;
the hierarchical storage function module informs the Cache of which addresses need to be subjected to hot spot migration;
the Cache function module writes data into a disk according to an address space provided by the hierarchical storage, and the disk writing address area is in the Cache;
the Cache continues to issue an address space required by the hierarchical storage to the hierarchical storage module;
the hierarchical storage writes data to the address space of a new medium according to a disk writing request issued by the Cache;
the hierarchical storage module modifies the logical volume to medium mapping table into a new medium address space;
if the data in the hot spot area is accessed by the write request at the moment, the Cache page is accessed by the request at the moment, and the hit happens, the data is directly returned to the host.
In the data migration process, under the condition that a read IO accesses a migration area:
as shown in fig. 9, the method specifically includes the following steps:
the hierarchical storage function module carries out hot spot data statistics on the address space of the hierarchical hard disk where the LUN is located according to the hierarchical storage statistic granularity;
the hierarchical storage function module analyzes which address areas are hot areas according to the granularity statistical data;
the hierarchical storage function module informs the Cache of which addresses need to be subjected to hot spot migration;
the Cache function module writes data into a disk according to an address space provided by the hierarchical storage, and the disk writing address area is in the Cache;
the Cache continues to issue an address space required by the hierarchical storage to the hierarchical storage module;
and writing the data to the address space of the new medium according to the disk writing request issued by the Cache by the hierarchical storage. But only partial data is written, and the data of the whole hot spot area is not written;
at this time, the read request is right to access the hot spot data, and the hot spot data is the migrated area data;
if the Cache is hit, the Cache returns directly, and if the Cache is not hit, the Cache issues a request to the hierarchical storage module;
the hierarchical storage module judges whether the read request needs to read the migrated new data or the old data according to the migration progress mark, if the migrated data is written, the migrated data is the new data, and if not, the migrated data is the old data;
storing read data to a Cache in a grading manner, and returning the data to a host read request by the Cache;
the hierarchical storage module modifies the logical volume to media mapping table to a new media address space.
Example 4
This embodiment provides a data migration apparatus, which is disposed on one side of a Cache, and fig. 10 is a block diagram of a structure of the data migration apparatus provided in embodiment 4 of the present invention, and as shown in fig. 10, the apparatus 100 includes the following components:
a learning unit 101, configured to learn, from the hierarchical storage module through the Cache module, an address space of the hot spot data to be migrated;
the first writing unit 102 is configured to write hot spot data to be migrated according to an address space through a Cache module;
and the notification unit 103 is configured to notify the hierarchical storage module of an address space of the hot spot data to be migrated in the Cache module through the Cache module.
The first writing unit 102 may specifically include: the read-ahead subunit is used for issuing a read-ahead request to the hierarchical storage module by the Cache module under the condition that address spaces of all the hot spot data to be migrated are not found in the Cache module; and the write-in subunit is used for receiving the hot spot data to be migrated sent by the hierarchical storage module through the Cache module and writing the received hot spot data into the Cache module.
Example 5
This embodiment provides a data migration apparatus, which is disposed on one side of a hierarchical storage module, and fig. 11 is a block diagram of a structure of the data migration apparatus provided in embodiment 5 of the present invention, as shown in fig. 11, the apparatus 110 specifically includes the following components:
the informing unit 111 is used for informing the Cache module of the address space of the hot spot data to be migrated through the hierarchical storage module;
the second writing unit 112 is configured to write the hot data to an address space of a new medium after acquiring, by the hierarchical storage module, an address space of the data to be migrated in the Cache module;
and a modifying unit 113, configured to modify, by the hierarchical storage module, a correspondence between the logical volume and the media mapping table according to the address space of the new media.
The second writing unit 112 may specifically include: the judging subunit is used for judging whether the data required to be read by the request is migrated data or not according to the migration progress of the hot data when receiving the hot data access request forwarded by the Cache module, and obtaining a judgment result; and the notification subunit is used for reading the data according to the judgment result and notifying the Cache module that the data reading is finished after the data reading is finished.
Further, the apparatus 110 may further include: the statistical unit is used for carrying out hot data statistics on the hard disk address space of the level where the logical volume LUN is located according to the hierarchical storage statistical granularity by the hierarchical storage module before informing the Cache module of the address space of the hot data to be migrated; and the analysis unit is used for analyzing the space address of the hot spot data to be migrated by the hierarchical storage module according to the statistical data.
According to the scheme provided by the embodiment of the invention, one-time disk reading operation and one-time memory occupation are reduced in the data migration process, only one-time disk writing operation is involved in the disk refreshing migration process, and the traditional migration method needs to perform double writing when the disk is refreshed so as to ensure the data consistency. Meanwhile, compared with the traditional migration method, the scheme of the embodiment of the invention can improve the hit rate of the system by the pre-read data under the scene that the migration data is not hit in the Cache, thereby improving the performance of the system to a certain extent.
Although the preferred embodiments of the present invention have been disclosed for illustrative purposes, those skilled in the art will appreciate that various modifications, additions and substitutions are possible, and the scope of the invention should not be limited to the embodiments described above.
Claims (10)
1. A method of data migration, comprising:
the Cache module of the Cache memory obtains the address space of the hot spot data to be migrated from the hierarchical storage module;
the Cache module writes the hot spot data to be migrated into the Cache module according to the address space;
and the Cache module informs the hierarchical storage module of the address space of the hot spot data to be migrated in the Cache module.
2. The method according to claim 1, wherein the Cache module writes the hot spot data to be migrated according to the address space, and the method comprises:
when the address spaces of all the hot spot data to be migrated are not found in the Cache module, the Cache module issues a pre-reading request to the hierarchical storage module;
and the Cache module receives the hot spot data to be migrated sent by the hierarchical storage module and writes the received hot spot data into the Cache module.
3. A method of data migration, comprising:
the hierarchical storage module informs the address space of the hot spot data to be migrated to a Cache module of a Cache memory;
after the hierarchical storage module acquires the address space of the data to be migrated in the Cache module, writing the hot spot data into the address space of a new medium;
and the hierarchical storage module modifies the corresponding relation between the logical volume and the medium mapping table according to the address space of the new medium.
4. The method according to claim 3, wherein after learning the address space of the data to be migrated in the Cache module, the hierarchical storage module writes the hot spot data to the address space of a new medium, including:
when a hotspot data access request forwarded by the Cache module is received, judging whether the data required to be read by the request is migrated data according to the migration progress of the hotspot data to obtain a judgment result;
and reading data according to the judgment result, and informing the Cache module that the data reading is finished after the data reading is finished.
5. The method according to claim 3 or 4, characterized in that the method further comprises:
before informing the Cache module of the address space of the hot data to be migrated, the hierarchical storage module performs hot data statistics on the hard disk address space of the level where the logical volume LUN is located according to the hierarchical storage statistical granularity;
and the hierarchical storage module analyzes the space address of the data of the hot spot to be migrated according to the statistical data.
6. A data migration apparatus, comprising:
the learning unit is used for learning the address space of the hot spot data to be migrated from the hierarchical storage module through the Cache module of the Cache memory;
the first writing unit is used for writing the hot spot data to be migrated into the Cache module according to the address space through the Cache module;
and the notification unit is used for notifying the address space of the hot spot data to be migrated in the Cache module to the hierarchical storage module through the Cache module.
7. The apparatus of claim 6, wherein the first writing unit comprises:
the read-ahead subunit is used for issuing a read-ahead request to the hierarchical storage module by the Cache module under the condition that address spaces of all the hot spot data to be migrated are not found in the Cache module;
and the writing subunit is used for receiving the hot spot data to be migrated sent by the hierarchical storage module through the Cache module and writing the received hot spot data into the Cache module.
8. A data migration apparatus, comprising:
the informing unit is used for informing the address space of the hot spot data to be migrated to the Cache module of the Cache memory through the hierarchical storage module;
the second writing unit is used for writing the hot data into the address space of a new medium after the address space of the data to be migrated in the Cache module is obtained through the hierarchical storage module;
and the modifying unit is used for modifying the corresponding relation between the logical volume and the medium mapping table by the hierarchical storage module according to the address space of the new medium.
9. The apparatus of claim 8, wherein the second writing unit comprises:
the judging subunit is used for judging whether the data required to be read by the request is migrated data or not according to the migration progress of the hot data when receiving the hot data access request forwarded by the Cache module, and obtaining a judgment result;
and the notification subunit is used for reading data according to the judgment result and notifying the Cache module that the data reading is finished after the data reading is finished.
10. The apparatus of claim 8 or 9, further comprising:
the hierarchical storage module is used for carrying out hot data statistics on the hard disk address space of the hierarchy where the logical volume LUN is located according to the hierarchical storage statistical granularity before informing the Cache module of the address space of the hot data to be migrated;
and the analysis unit is used for analyzing the space address of the hot spot data to be migrated by the hierarchical storage module according to the statistical data.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510655794.XA CN106569732B (en) | 2015-10-12 | 2015-10-12 | Data migration method and device |
| PCT/CN2016/099934 WO2017063495A1 (en) | 2015-10-12 | 2016-09-23 | Data migration method and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510655794.XA CN106569732B (en) | 2015-10-12 | 2015-10-12 | Data migration method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106569732A CN106569732A (en) | 2017-04-19 |
| CN106569732B true CN106569732B (en) | 2021-04-20 |
Family
ID=58506322
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510655794.XA Active CN106569732B (en) | 2015-10-12 | 2015-10-12 | Data migration method and device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN106569732B (en) |
| WO (1) | WO2017063495A1 (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110162376A (en) * | 2018-02-12 | 2019-08-23 | 杭州宏杉科技股份有限公司 | A kind of data read-write method and device |
| CN108519860B (en) * | 2018-03-31 | 2021-01-19 | 深圳忆联信息系统有限公司 | SSD read hit processing method and device |
| CN109376094A (en) * | 2018-09-27 | 2019-02-22 | 郑州云海信息技术有限公司 | A kind of method of the cold and hot Data Migration of garbage reclamation in storage system |
| CN111124307B (en) * | 2019-12-20 | 2022-06-07 | 北京浪潮数据技术有限公司 | Data downloading and brushing method, device, equipment and readable storage medium |
| CN111273861B (en) * | 2020-01-15 | 2022-07-29 | 苏州浪潮智能科技有限公司 | Method and system for setting data volume Bitmap |
| CN111722797B (en) * | 2020-05-18 | 2021-06-29 | 西安交通大学 | SSD and HA-SMR hybrid storage system oriented data management method, storage medium and device |
| CN112214169B (en) * | 2020-09-28 | 2021-08-10 | 深圳大学 | Data storage device and stored data migration method |
| CN112667525A (en) * | 2020-12-23 | 2021-04-16 | 北京浪潮数据技术有限公司 | Used space measuring method and component of persistent memory |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102799390A (en) * | 2012-06-14 | 2012-11-28 | 国家超级计算深圳中心(深圳云计算中心) | Cooperative adaptive pre-fetching method for sensing energy consumption and middleware |
| CN102831087A (en) * | 2012-07-27 | 2012-12-19 | 国家超级计算深圳中心(深圳云计算中心) | Data reading-writing processing method and device based on mixing memory |
| CN103186350A (en) * | 2011-12-31 | 2013-07-03 | 北京快网科技有限公司 | Hybrid storage system and hot spot data block migration method |
| US8688904B1 (en) * | 2005-05-23 | 2014-04-01 | Hewlett-Packard Development Company, L.P. | Method for managing data storage |
| CN104321754A (en) * | 2012-12-21 | 2015-01-28 | 华为技术有限公司 | Method and device for setting working mode of Cache |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7756943B1 (en) * | 2006-01-26 | 2010-07-13 | Symantec Operating Corporation | Efficient data transfer between computers in a virtual NUMA system using RDMA |
| CN102215268A (en) * | 2011-07-14 | 2011-10-12 | 北京飞杰信息技术有限公司 | Method and device for transferring file data |
| CN102968498B (en) * | 2012-12-05 | 2016-08-10 | 华为技术有限公司 | Data processing method and device |
| CN103106045A (en) * | 2012-12-20 | 2013-05-15 | 华为技术有限公司 | Data migration method, system and device at host machine end |
| CN104917784B (en) * | 2014-03-10 | 2018-06-05 | 华为技术有限公司 | A kind of data migration method, device and computer system |
-
2015
- 2015-10-12 CN CN201510655794.XA patent/CN106569732B/en active Active
-
2016
- 2016-09-23 WO PCT/CN2016/099934 patent/WO2017063495A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8688904B1 (en) * | 2005-05-23 | 2014-04-01 | Hewlett-Packard Development Company, L.P. | Method for managing data storage |
| CN103186350A (en) * | 2011-12-31 | 2013-07-03 | 北京快网科技有限公司 | Hybrid storage system and hot spot data block migration method |
| CN102799390A (en) * | 2012-06-14 | 2012-11-28 | 国家超级计算深圳中心(深圳云计算中心) | Cooperative adaptive pre-fetching method for sensing energy consumption and middleware |
| CN102831087A (en) * | 2012-07-27 | 2012-12-19 | 国家超级计算深圳中心(深圳云计算中心) | Data reading-writing processing method and device based on mixing memory |
| CN104321754A (en) * | 2012-12-21 | 2015-01-28 | 华为技术有限公司 | Method and device for setting working mode of Cache |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2017063495A1 (en) | 2017-04-20 |
| CN106569732A (en) | 2017-04-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106569732B (en) | Data migration method and device | |
| CN105930282B (en) | A kind of data cache method for NAND FLASH | |
| US9665293B2 (en) | Method for a storage device processing data and storage device | |
| CN103136121B (en) | Cache management method for solid-state disc | |
| CN105159622B (en) | A kind of method and system reducing SSD read-write IO time delay | |
| US20190138517A1 (en) | Hot-Spot Adaptive Garbage Collection | |
| CN107908571B (en) | Data writing method, flash memory device and storage equipment | |
| CN105808163B (en) | Method for accessing shingled magnetic recording SMR hard disk and server | |
| CN106548789A (en) | Method and apparatus for operating stacked tile type magnetic recording equipment | |
| US20100070733A1 (en) | System and method of allocating memory locations | |
| CN104794064A (en) | Cache management method based on region heat degree | |
| CN103543955A (en) | Method and system for reading cache with solid state disk as equipment and solid state disk | |
| KR102681181B1 (en) | Memory system and operating method thereof | |
| CN109240939B (en) | Method for rapidly processing solid state disk TRIM | |
| AU2016397188A1 (en) | Storage system and system garbage collection method | |
| CN114327272B (en) | Data processing method, solid state disk controller and solid state disk | |
| US10133517B2 (en) | Storage control device | |
| CN108664217B (en) | Caching method and system for reducing jitter of writing performance of solid-state disk storage system | |
| CN103530236A (en) | Method and device for obtaining hybrid hard disk | |
| CN109783019A (en) | A kind of data intelligence memory management method and device | |
| CN110968527B (en) | FTL provided caching | |
| JP6100927B2 (en) | Information processing device | |
| CN103176753A (en) | Storage device and data management method of storage device | |
| CN112631518B (en) | Data storage method and equipment | |
| US7836247B2 (en) | Method, apparatus, and computer program product for permitting access to a storage drive while the drive is being formatted |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |