WO2018113317A1

WO2018113317A1 - Data migration method, apparatus, and system

Info

Publication number: WO2018113317A1
Application number: PCT/CN2017/096784
Authority: WO
Inventors: 周臣; 董浩
Original assignee: 华为技术有限公司
Priority date: 2016-12-22
Filing date: 2017-08-10
Publication date: 2018-06-28
Also published as: CN107122126B; CN107122126A

Abstract

A data migration method for a storage system is provided. The storage system comprises a first storage device and a second storage device. The method comprises the following steps: obtaining the number of read/write operations per unit time of the second storage device; obtaining, according to the number of read/write operations per unit time of the second storage device, a strength ratio of the second storage device; looking up a strength ratio versus capacity ratio correspondence table, to obtain a capacity ratio corresponding to the strength ratio; obtaining, according to the obtained capacity ratio, the number of data blocks that need to be migrated from the first storage device to the second storage device; and migrating, according to the number, corresponding data blocks from the first storage device to the second storage device. By means of the method of the invention, the number of data blocks to be migrated from the first storage device to the second storage device can be accurately determined, thereby ensuring that performance of the second storage device is not affected while as much data as possible is migrated to the second storage device, and improving performance and efficiency of the storage system.

Description

Data migration method, device and system

Technical field

The present application relates to the field of communications technologies, and in particular, to a data migration method, apparatus, and system.

Background technique

With the development of the network, more and more data is generated. Different data has different usage values, some data will be accessed frequently, some data will not be accessed for a long time, and some data will be accessed frequently at certain times.

The use value of data has its own life cycle and can be followed regularly. In general, various types of metadata (such as part of the operating system data of a virtual machine) are of high value, are frequently accessed, and have high response speed requirements. The frequency of newly generated data is usually accessed more frequently. Over time, the frequency of newly generated data will decrease and will not be accessed frequently.

Automatic tiered storage technology has thus emerged and is widely used in a variety of industries. The automatic tiered storage technology stores data on different performance storage devices according to the characteristics of data access frequency, importance, retention time, etc.; and migrates data with low access frequency to lower performance based on the frequency at which data is accessed. In a storage device, data that is frequently accessed is migrated to a storage device with high performance.

Data migration strategies include migrating low-access data to lower-performance storage devices and migrating frequently accessed data to high-performance storage devices. During data migration, data that is frequently accessed is preferentially migrated to a high-performance storage device. The amount of data migrated is often determined according to the current available capacity of the high-performance storage device, and is generally performed by IO request for accessing data. Monitoring, the number of accessed IO requests reaches a certain amount of data and migrates to high-performance storage devices.

The inventor found in the process of the invention that the data migration strategy of the automatic hierarchical storage technology is relatively simple, and the overall performance of the migrated storage system cannot be predicted, which may lead to overload of the high-performance storage device.

Summary of the invention

The embodiment of the present application provides a method, an apparatus, and a system for migrating data, which can accurately determine the amount of data of data that needs to be migrated from the first storage device to the second storage device, and migrate as much data as possible to the second storage device. At the same time, it can ensure that the performance of the second storage device is not affected, and the performance and efficiency of the storage system are improved.

The embodiments of the present application provide the following technical solutions:

A first aspect, a method for migrating data in a storage system, the storage system first storage device and the second storage device, the method comprising: acquiring a read/write number of the second storage device per unit time; and according to the second storage device Obtaining the number of readings and the number of times per unit time, obtaining the intensity ratio of the second storage device; querying the intensity ratio to the capacity ratio correspondence table, obtaining a capacity ratio corresponding to the intensity ratio; obtaining the required ratio from the capacity ratio according to the a number of data blocks that a storage device migrates to the second storage device; and a corresponding data block is migrated from the first storage device to the second storage device according to the quantity.

After obtaining the intensity ratio of the second storage device, according to the strength ratio and the capacity ratio correspondence table, it is required to migrate to the second storage. The number of data blocks of the storage device. In this way, the specific value of the migrated data can be determined according to the performance of the second storage device, which can ensure that the performance of the second storage device is not affected after the data is migrated, and the performance of the entire storage system is ensured.

In one possible design, the intensity ratio to capacity ratio correspondence table is obtained by pre-monitoring analysis.

After the strength ratio of the second storage device is obtained, the strength ratio to capacity ratio correspondence table can be directly queried to quickly obtain the number of data blocks that need to be migrated from the first storage device to the second storage device.

In one possible design, the query intensity ratio to capacity ratio correspondence table is obtained by a fuzzy matching rule to obtain a capacity ratio corresponding to the intensity ratio.

It is impossible to record all possible data in the intensity ratio and capacity ratio correspondence table. Therefore, when the calculated specific value of the intensity ratio cannot be found in the strength ratio and capacity ratio correspondence table, according to the fuzzy matching rule, The ratio of the strength ratio calculated by the ratio of the intensity ratio to the capacity ratio is higher than the ratio of the intensity ratio of the higher one level to determine the corresponding capacity ratio, thereby improving the search efficiency.

In a possible design, the number of readings and writes per unit time in each storage block in the storage system is obtained, and the number of read and write times per unit time of each data block obtained is in order from high to low. arrangement. Obtaining, according to the capacity ratio, the number of data blocks that need to be migrated from the first storage device to the second storage device, specifically, according to the capacity ratio and the unit time of each data block acquired The number of read and write times determines a data block corresponding to the capacity ratio; and the number of data blocks corresponding to the capacity ratio in the first storage device is confirmed.

In order to determine that the data with more access times is migrated from the first storage device to the second storage device, the number of read and write times per unit time of each data block in the obtained storage system is arranged in order from high to low, so that The data block with higher read/write times per unit time can be migrated from the first storage device to the second storage device with higher performance in order according to the determined number of data blocks that need to be migrated, and the IO request can be responded more quickly. .

In a possible design, obtaining the number of reading and writing times of the first storage device per unit time;

The intensity ratio of the second storage device is the sum of the number of reading and writing times per unit time of the second storage device and the number of reading and writing times per unit time of the first storage device and the number of reading and writing times per unit time of the second storage device. proportion.

In a possible design, the number of reading and writing times of the first storage device per unit time is the product of the number of reading and writing times per unit time in the first storage device and the number of disks in the first storage device. Dividing by the first conversion factor, the first conversion factor is related to the ratio of the read request and the write request of the first storage device and the RAID level of the first storage device; the reading of the second storage device per unit time The number of writes is a product of the number of reads and writes in a single disk unit time and the number of disks in the second storage device in the second storage device divided by a second conversion factor, the second conversion factor and the The ratio of the read request of the second storage device to the write request and the RAID level of the second storage device.

In a possible design, the number of readings and writes in a single disk unit time in the first storage device is related to the load characteristics of the first storage device and the response time; the single disk unit time in the second storage device The number of read and write times is related to the load characteristics of the second storage device and the response duration.

By setting and obtaining the specific values of each parameter, the number of data blocks that need to be migrated from the first storage device to the second storage device can be accurately obtained, and the hot data can be migrated to the high-performance second storage as much as possible. At the same time, the device can guarantee the performance of the storage device and improve the efficiency and performance of the storage system.

In a second aspect, a storage system for implementing data migration is provided. The storage system includes a first storage device, a second storage device, and a processor, and the processor is used in each step of the above method. For details on the implementation of each step and the corresponding benefits, please refer to the relevant description in the first aspect.

In a third aspect, a storage system for implementing data migration is provided. The storage system includes a first storage device, a second storage device, and a processor. The processor includes a data collection and analysis module and a data migration module. The data collection and analysis module is configured to obtain the read/write times of the second storage device in a unit time; and obtain the intensity ratio of the second storage device according to the read/write times of the second storage device per unit time; the query intensity ratio and the capacity ratio Corresponding to the table, obtaining a capacity ratio corresponding to the intensity ratio; obtaining, according to the capacity ratio, the number of data blocks that need to be migrated from the first storage device to the second storage device; and obtaining the data block to be migrated The amount is sent to the data migration module. The data migration module is configured to migrate the obtained number of data blocks from the first storage device to the second storage device.

The data collection and analysis module obtains the intensity ratio of the second storage device, and obtains the number of data blocks that need to be migrated to the second storage device according to the strength ratio and the capacity ratio correspondence table. In this way, the specific value of the migrated data can be determined according to the performance of the second storage device, which can ensure that the performance of the second storage device is not affected after the data is migrated, and the performance of the entire storage system is ensured.

In a possible design, the data acquisition and analysis module is further configured to: pre-collect an IO request of the analysis application to the storage system, and obtain a strength ratio to capacity ratio correspondence table.

After the strength ratio of the second storage device is obtained, the data collection and analysis module can directly query the strength ratio and capacity ratio correspondence table to quickly obtain the number of data blocks that need to be migrated from the first storage device to the second storage device.

In a possible design, the data acquisition and analysis module is configured to obtain a capacity ratio corresponding to the intensity ratio by using a query strength ratio and a capacity ratio correspondence table, and specifically: the data collection and analysis module is configured to query a strength ratio and a capacity ratio correspondence table. A capacity ratio corresponding to the intensity ratio is obtained by a fuzzy matching rule.

In a possible design, the data acquisition and analysis module is further configured to acquire the number of readings and writes per unit time in each data block in the storage system, and read and write the unit time of each data block obtained. The number of data blocks from the first storage device to the second storage device is obtained by the data collection and analysis module according to the capacity ratio: The capacity ratio is obtained in order to obtain the number of data blocks that need to be migrated from the first storage device to the second storage device.

In order to determine that the data with more access times is migrated from the first storage device to the second storage device, the data collection and analysis module presses the number of read and write times per unit time of each data block in the obtained storage system from high to low. Arrange sequentially, so that data blocks with higher read/write times per unit time can be sequentially migrated from the first storage device to the second storage device with higher performance according to the determined number of data blocks to be migrated, faster Respond to IO requests.

In a possible design, the data acquisition and analysis module is further configured to obtain the number of read and write times of the first storage device per unit time. The intensity ratio of the second storage device is the proportion of the number of reading and writing times per unit time of the second storage device to the sum of the number of reading and writing times per unit time of the first storage device and the number of reading and writing times per unit time of the second storage device.

In a possible design, the number of readings and writes per unit time of the first storage device is the product of the number of readings and writes in a single disk unit time and the number of disks in the first storage device in the first storage device, and then divided by a conversion factor, wherein the first conversion factor is related to a ratio of a read request and a write request of the first storage device and a RAID level of the first storage device; and the number of read and write times of the second storage device per unit time is a single storage device The product of the number of reads and writes in the unit time and the number of disks in the second storage device is divided by the second conversion factor, the second conversion factor and the read request of the second storage device It is related to the ratio of the write request and the RAID level of the second storage device.

DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly described below.

1 is a schematic structural diagram of a storage system according to an embodiment of the present invention;

2 is a schematic flowchart of a method for migrating data according to an embodiment of the present invention;

FIG. 3 is a schematic structural diagram of another storage system according to an embodiment of the present invention.

detailed description

An embodiment of the present invention provides a method for migrating data in a storage system, and calculates a size of data to be migrated by collecting and collecting load characteristics of an IO request to access the storage system, and estimating the performance of the storage device in the storage system. Maximize the performance of your storage system for the purpose of migrating data on demand

In the embodiment of the present invention, data whose access frequency reaches a predetermined value is referred to as hot data, and data whose access frequency is lower than a predetermined value is referred to as cold data.

A storage system to which the embodiments of the present invention are applied is shown in FIG. The storage system 1 includes a storage medium 101, a processor 103, and a cache 105. The storage medium 101 in the storage system 1 can be various and divided into different performance layers according to the performance of the storage medium. For example, the storage medium in the storage system consists of an SSD disk and a common hard disk. The SSD disk constitutes a high-performance layer storage device, and the ordinary hard disk constitutes a storage device of a common performance layer. In the embodiment of the present invention, the high performance layer and the common performance layer are included in the storage system as an example. For convenience of description, the common performance layer in the storage system is referred to as a first storage device, and the high performance in the storage system. The layer is called the second storage device. In addition, in the embodiment of the present invention, the hot data is moved to the high performance layer as an example, and the hot data cooling from the high performance layer to the common performance layer can also be processed by the same method. When other performance layers are added to the storage system, the data can be processed in the same way when migrating between different performance layers. The processor 103 is configured to execute the method of migrating data described below, and the cache 105 is configured to store required information.

The flow of the method for migrating data in the storage system provided by the embodiment of the present invention is as shown in FIG. 2, and is completed by a processor in the storage system. The method embodiment of the present invention mainly describes an implementation manner of migrating data from a common performance layer to a high performance layer, which is described in detail below. As described above, the common performance layer in the storage system is referred to as a first storage device, and the high performance layer in the storage system is referred to as a second storage device. In addition, the IO request in the embodiment of the present invention is an object of monitoring and analyzing the IO request for accessing the storage medium.

Step 201: Acquire the number of times of reading and writing of the second storage device per unit time.

The processor obtains the number of times of reading and writing of the second storage device in a unit time. The unit time can be determined by the user according to the service type, which is not limited in the embodiment of the present invention. When the unit time is 1 second, the number of read/write times per second (IOPS) of the second storage device is obtained.

In the embodiment of the present invention, each disk in the same performance layer storage device has the same type. Embodiments of the present invention provide A method for obtaining the number of reading and writing times per unit time of the second storage device. For example, the number of reads and writes per unit time of the second storage device = (number of read/write times per unit time * number of disks) / RAID conversion factor. The number of disks is the number of disks in the second storage device. For random write IO, the storage device uses RAID mode to calculate the check data, so it needs to generate additional IO. The RAID conversion factor is related to the ratio of read and write IO and the write penalty coefficient. The write penalty coefficient refers to the write IO amplification factor of the disk. Refer to Table 1 for the relationship between the value of the write penalty coefficient and the RAID level. Specifically, the RAID conversion factor = read request ratio + write request ratio * write penalty coefficient.

RAID级别RAID level	写惩罚系数Write penalty factor
00	11
11	22
55	44
66	66
1010	22

Table 1: Write penalty factor corresponding to RAID level

The number of reads and writes in a single disk unit is related to the load characteristics and response duration of the second storage device. The data can be collected in advance and obtained statistically, as shown in Table 2. The load characteristic of the second storage device here is the load characteristic of the IO request to access the second storage device. When the number of read/write times of the second storage device is required to be obtained, the IOPS of the single disk in the second storage device can be directly found according to the load characteristic and the response time value of the IO request of the second storage device. The above method calculates the number of times of reading and writing in the second storage device time.

Table 2: Lookup table of the number of read/write times per unit time in the second storage device

Optionally, the number of times of reading and writing in a single disk unit time can also be obtained according to the load characteristics and the response time length of the IO request for accessing the second storage device, and the value of the second table is not required to be obtained in advance.

Step 203: Acquire an intensity ratio of the second storage device according to the number of accesses of the second storage device per unit time.

The strength ratio of the second storage device is a ratio of the number of accesses per unit time of the second storage device to the number of accesses per unit time of the storage system.

In the embodiment of the present invention, the storage system includes two performance layer storage devices of the first storage device and the second storage device, and the intensity ratio of the second storage device=the number of accesses per second time of the second storage device/(second The number of accesses per unit time of the storage device + the number of accesses per unit time of the first storage device), that is, the intensity ratio of the second storage device is the number of read and write times per unit time of the second storage device, and the first storage device unit time The proportion of the number of reads and writes within the unit and the number of reads and writes per unit time of the second storage device. The number of accesses of the second storage device per unit time has been described in step 201 and will not be further described herein.

The method for obtaining the number of accesses per unit time of the first storage device is the same as the method for obtaining the number of accesses per unit time of the second storage device, and is not further described. Similarly, the reading of the single disk unit time in the first storage device The number of writes can also be collected in advance and statistically obtained, as shown in Table 3.

Table 3: Lookup table for the number of read/write times per unit time in the first storage device

It should be noted that the contents in Tables 2 and 3 can be combined into one table storage, that is, the number of readings and writes in a single disk unit time in each performance layer in the storage system can be reflected in a table, as shown in Table 4. . When there are other storage devices of different performance layers in the storage system, the number of readings and writes per unit time in a single disk in other performance layers may be reflected in one table, or may be embodied in different tables, in the implementation of the present invention. There is no limit in the example.

Table 4: Lookup table for the number of read and write times per unit time in the storage system

Step 205: Query a correspondence table between the intensity ratio and the capacity ratio, and obtain a capacity ratio corresponding to the intensity ratio.

The correspondence table between the intensity ratio and the capacity ratio is obtained by monitoring and analyzing the business data in advance. The service data in the storage system (including the first storage device and the second storage device) is split into a plurality of data blocks, and the number of read and write times of the data blocks in a unit time, that is, the access strength of the data block is obtained. Business data can be split into multiple data blocks according to the granularity of migration, or can be split into multiple data blocks according to the specified size. The number of reads and writes of these data blocks is accessed during the statistical monitoring time, and the number of read and write times of the data blocks per unit time is calculated, that is, the access strength. The data blocks in the storage system are sorted according to the access intensity from high to low, and then the data blocks are divided into groups according to a certain capacity ratio. For example, as shown in Table 3, data blocks are grouped by a capacity ratio of 2.5%, that is, data1 to dataA form a group having a capacity ratio of 2.5%, that is, data1 to data2A form a group having a capacity ratio of 5%... and so on. As shown in Table 5.

Table 5: Statistical Table of Hotspot Data Blocks

Calculate the number of reads and writes per unit of data per unit time, and then compare each unit to the unit time of the corresponding data block. The sum of the number of readings and writes in the storage system is compared with the sum of the number of readings and readings of all the data blocks in the storage system per unit time, and the intensity ratio corresponding to the capacity ratio, that is, the correspondence table of the intensity ratio and the capacity ratio is obtained, as shown in Table 6. Shown. The data in Table 6 is an example description, and the specific data will vary depending on the business. The value of the capacity ratio can also be set as needed.

容量比Capacity ratio	强度比Intensity ratio
0.0％0.0%	0.0％0.0%
2.5％2.5%	35.0％35.0%
5.0％5.0%	50.0％50.0%
7.5％7.5%	60.0％60.0%
10.0％10.0%	65.0％65.0%
12.5％12.5%	67.0％67.0%
15.0％15.0%	69.0％69.0%
20.0％20.0%	70.0％70.0%
25.0％25.0%	71.5％71.5%
30.0％30.0%	72.0％72.0%
100.0％100.0%	100.0％100.0%

Table 6: Correspondence table between intensity ratio and capacity ratio of a typical load

As shown in Table 6, when the capacity ratio of the data block in the storage system is 10%, the corresponding intensity ratio is 65%, that is, the P intensity ratio [10%]=65%, indicating that the data block occupying 10% of the total data amount. The number of I/O accesses is 65% of the total number of I/O accesses in the storage system. Conversely, when the strength ratio of the data block is 70%, the corresponding capacity ratio is 20%, and the P hot spot capacity ratio [70%]=20%, indicating that the number of I/O accesses accounts for all data blocks I/ in the storage system. The number of O accesses 70% of the data blocks accounts for 20% of all data blocks in the storage system. From the data law in the table, we can see that the intensity ratio corresponding to 10% data block has been as high as 65%; while the intensity ratio corresponding to more data blocks does not change much, indicating that the hot data of the scene is very concentrated, suitable for using grading. Technology to improve performance.

In order to improve the accuracy of the data, optionally, the data block access situation of the primary business model can be used as a statistical basis to obtain a strength ratio to capacity ratio correspondence table.

After the strength ratio of the second storage device obtained in step 203, the capacity ratio corresponding to the strength ratio of the second storage device can be obtained by querying the capacity ratio correspondence table. The query rule may adopt a fuzzy matching rule, that is, a query strength ratio and a capacity ratio correspondence table, and obtain a capacity ratio corresponding to the intensity ratio by a fuzzy matching rule. If the intensity ratio of the query needs to be inaccurately matched in the intensity ratio to the capacity ratio correspondence table, the corresponding capacity ratio is searched for according to the intensity ratio of the obtained second storage device and the intensity ratio of the higher one. If the data block to be migrated according to the capacity ratio obtained by the query satisfies the condition, the capacity ratio is calculated. If the data block that needs to be migrated according to the capacity ratio obtained by the query does not satisfy the condition, the strength ratio of the obtained second storage device is lower than the strength ratio of the lower one to find the corresponding capacity ratio. The specific conditions are explained below and will not be described in detail herein.

Step 207: Obtain a quantity of data blocks that need to be migrated from the first storage device to the second storage device according to the capacity ratio.

According to the capacity ratio obtained in step 205, the number of data blocks corresponding to the capacity ratio is obtained. Obtaining the number of readings and writes per unit time in each data block in the storage system, and reading the unit time of each data block obtained The number of writes is arranged in order from high to low; according to the obtained number of data blocks corresponding to the capacity ratio, a corresponding number of data blocks are selected according to the access intensity from high to low, and it is determined that the first storage device needs to be migrated to the second The number of data blocks of the storage device.

If some of the selected corresponding number of data blocks are already in the second storage device, determining the number of data blocks in the first storage device among the data blocks, and determining that this needs to be migrated from the first storage device to the second The number of data blocks of the storage device.

The following is an example in which the data block has not been moved in the second storage device. If the data block exists in the second storage device, the corresponding value is subtracted from the value already in the second storage device. , no longer explain one by one. The amount of migrated data = P capacity ratio [second storage device IOPS / (second storage device IOPS + first storage device IOPS)] * total data amount. If the calculated amount of data of the data block to be migrated is smaller than the total capacity of the second storage device, the number of data blocks that need to be migrated is the number of the calculated data blocks. If the calculated data volume of the data block to be migrated is greater than the total capacity of the second storage device, the number of data blocks that need to be migrated is based on the total capacity of the second storage device.

Step 209: Migrate the corresponding data block from the first storage device to the second storage device according to the quantity. The number of data blocks that need to be migrated from the first storage device to the second storage device calculated according to step 207, and the corresponding number of data blocks are migrated from the first storage device to the second storage device according to the access intensity from high to low. in.

The method of the embodiment of the present invention is further illustrated by a specific example below.

Through the monitoring and analysis of typical business data, the values of the following parameters are obtained, as shown in Table 7:

Table 7: Business Parameter Value Table

The second storage device IOPS=3500*10/(0.7+0.3*4), the first storage device IOPS=180*100/(0.7+0.3*4), then the second storage device intensity ratio=the second storage device IOPS/ (Second storage device IOPS+first storage device IOPS)=66%, according to the fuzzy matching rule, look up the intensity ratio and capacity ratio correspondence table shown in Table 6, with the intensity ratio 66% of the first-order strength ratio being 67%. Determine the corresponding capacity ratio is 12.5%, then the amount of migrated data = the amount of data in the storage system * 12.5% = 3.75TB; because the high-performance layer capacity (6TB) > 3.75TB, the final migrated data volume is 3.75TB, 3.75 TB of data is migrated from the first storage device to the second storage device according to the access intensity from high to low. The data that accounts for 66% of the storage system access strength is migrated to the second storage device. Since the second storage device is a high-performance storage device, the user's request can be responded to in time. Moreover, the amount of data migrated does not exceed the capacity of the second storage device, and does not affect the performance of the second storage device, thereby ensuring the overall performance of the storage system.

The embodiment of the present invention further provides a storage system 3, which can implement the data migration method as described above, and the structure of the storage system 3 is as shown in FIG. 3. The storage system 3 includes a first storage device 321 and a second storage device 323, and the performance of the first storage device 321 and the second storage device 323 are different. In the embodiment of the present invention, the first storage device 321 is composed of a storage medium of a general performance, such as a conventional disk, and can store user data. The second storage device 323 is composed of a high-performance storage medium, such as an SSD disk, and can be used to store hot data. . When it is necessary to indicate the same features of the first storage device 321 and the second storage device 323, a description will be made using the storage device. The storage device in the embodiment of the present invention is only an exemplary description. In practical applications, storage media with different performances may also be added. The number of the storage media included in the storage device may also be set as needed, which is not limited in the embodiment of the present invention. In addition, the storage device may be composed of a storage medium of the same performance, or may be composed of a storage medium having similar performance. The embodiment of the present invention is described by taking an example of migrating data in the first storage device 321 to the second storage device 323. In this embodiment, only the functions of the components in the storage system 3 are briefly described. For details of the method steps involved, refer to the description of the method embodiments in the foregoing.

The storage system 3 further includes a processor 31 including a data acquisition and analysis module 311 and a data migration module 313. The data collection and analysis module 311 is configured to perform an analysis and calculation on the IO request of the application access storage device, obtain the number of data blocks that need to be migrated from the first storage device 321 to the second storage device 323, and notify the data migration module 313. The data migration module 313 is configured to migrate the obtained number of data blocks from the first storage device 321 to the second storage device 323.

Specifically, the data collection and analysis module 311 is configured to acquire the read/write times of the second storage device 323 in a unit time; and obtain the intensity ratio of the second storage device 323 according to the read/write times of the second storage device 323 per unit time. a query intensity ratio to capacity ratio correspondence table, obtaining a capacity ratio corresponding to the intensity ratio; obtaining a number of data blocks that need to be migrated from the first storage device 321 to the second storage device 323 according to the capacity ratio The obtained number of data blocks that need to be migrated is sent to the data migration module 313. The detailed processing method has been described in detail in the foregoing method and will not be described separately here.

The data collection and analysis module 311 obtains the intensity ratio of the second storage device 323, and obtains the number of data blocks that need to be migrated to the second storage device 323 according to the intensity ratio and capacity ratio correspondence table. In this way, the amount of migrated data can be determined according to the performance of the second storage device 323, and the performance of the second storage device 323 can be ensured after the data is migrated, thereby ensuring the performance of the entire storage system.

The data migration module 313 is configured to migrate the corresponding data block from the first storage device 321 to the second storage device 323 according to the received number of data blocks that need to be migrated.

The data collection and analysis module 311 is further configured to pre-collect an IO request of the analysis application to the storage system 3 to obtain an intensity ratio and capacity ratio correspondence table. After the intensity ratio of the second storage device 323 is obtained, the strength ratio and capacity ratio correspondence table can be directly queried, and the number of data blocks that need to be migrated from the first storage device 321 to the second storage device 323 can be quickly obtained.

The data collection and analysis module 311 is configured to obtain a capacity ratio corresponding to the intensity ratio by using the query strength ratio and the capacity ratio correspondence table. The data collection and analysis module 311 is configured to query the intensity ratio and the capacity ratio correspondence table, and obtain the The intensity ratio is a corresponding capacity ratio.

The data collection and analysis module 311 is further configured to acquire the number of reading and writing times per unit time in each data block in the storage system 3, and press the number of times of reading and writing in the unit time of each data block obtained from high to low. Arranging sequentially; determining a data block corresponding to the capacity ratio according to the capacity ratio and the number of read and write times per unit time of each acquired data block; confirming in the first storage device 321 in order The capacity ratio is the number of corresponding data blocks.

In order to determine that the hot data with more access times is migrated from the first storage device 321 to the second storage device 323, the data collection and analysis module 311 will read and write the number of read and write times per unit time of each data block in the storage system 3. Arranged in order from high to low, so that data blocks with higher read/write times per unit time can be sequentially migrated from the first storage device 321 to the second higher performance module according to the determined number of data blocks to be migrated. In storage device 323, the IO request is responded more quickly.

The data collection and analysis module 311 is further configured to acquire the number of times of reading and writing of the first storage device 321 per unit time. The intensity ratio of the second storage device 323 is the number of reading and writing times per unit time of the second storage device 323, and the number of reading and writing times per unit time of the first storage device 321 and the number of reading and writing times per unit time of the second storage device 323. And the proportion.

The number of times of reading and writing of the first storage device 321 per unit time is the product of the number of reading and writing times per unit time in the first storage device 321 and the number of disks in the first storage device 321 divided by a first conversion factor, the first conversion factor is related to a ratio of a read request and a write request of the first storage device 321 and a RAID level of the first storage device 321;

The number of times of reading and writing of the second storage device 323 per unit time is the product of the number of reading and writing times per unit time in the second storage device 323 and the number of disks in the second storage device 323 divided by a second conversion factor, the second conversion factor being related to a ratio of a read request to a write request of the second storage device 323 and a RAID level of the second storage device 323.

By setting and obtaining the specific values of the respective parameters, the number of data blocks that need to be migrated from the first storage device 321 to the second storage device 323 can be accurately obtained, and the hot data can be migrated to the high performance as much as possible. At the same time, the storage device 323 can ensure the performance of the storage device 323 and improve the efficiency and performance of the storage system.

The possible values and calculation methods of the specific parameters have been described in detail in the foregoing method embodiments, and are not described in detail herein.

The embodiment of the present application further provides a computer storage medium for storing computer software instructions used by the storage system, which includes a program designed to execute the foregoing method embodiments. By executing a stored program, a method of migrating data between storage devices can be implemented.

The embodiment of the present application further provides a computer program, which includes instructions, when the computer program is executed by a computer, to enable the computer to execute the flow of the foregoing method embodiment.

Although the present application has been described herein in connection with the various embodiments, those skilled in the art can Other variations of the disclosed embodiments are achieved. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill several of the functions recited in the claims. Certain measures are recited in mutually different dependent claims, but this does not mean These measures cannot be combined to produce good results.

Those skilled in the art will appreciate that embodiments of the present application can be provided as a method, apparatus (device), or computer program product. Thus, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment in combination of software and hardware. Moreover, the application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code. The computer program is stored/distributed in a suitable medium, provided with other hardware or as part of the hardware, or in other distributed forms, such as over the Internet or other wired or wireless telecommunication systems.

The present application is described with reference to flowchart illustrations and/or block diagrams of the methods, apparatus, and computer program products of the embodiments of the present application. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine for the execution of instructions for execution by a processor of a computer or other programmable data processing device. Means for implementing the functions specified in one or more of the flow or in a block or blocks of the flow chart.

The computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device. The apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.

These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device. The instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

The above embodiments are merely illustrative of the technical solutions of the present invention, and are not intended to be limiting; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art will understand that The technical solutions described in the examples are modified, or some or all of the technical features are equivalently substituted; and such modifications or substitutions do not depart from the scope of the claims.

Claims

A method for migrating data in a storage system, the storage system comprising a first storage device and a second storage device, including:

Obtaining the number of times of reading and writing of the second storage device per unit time;

Obtaining an intensity ratio of the second storage device according to the number of times of reading and writing of the second storage device per unit time;

Querying an intensity ratio to a capacity ratio correspondence table, and obtaining a capacity ratio corresponding to the intensity ratio;

Obtaining, according to the capacity ratio, a quantity of data blocks that need to be migrated from the first storage device to the second storage device;

And migrating corresponding data blocks from the first storage device to the second storage device according to the quantity.
The method according to claim 1, wherein said intensity ratio ratio to capacity ratio correspondence table is obtained by pre-monitoring analysis.
The method according to claim 1 or 2, wherein the step of querying the intensity ratio to the capacity ratio correspondence table, and obtaining the capacity ratio corresponding to the intensity ratio is specifically:

The query intensity ratio and the capacity ratio correspondence table obtain a capacity ratio corresponding to the intensity ratio by a fuzzy matching rule.
The method according to any one of claims 1-3, wherein the method further comprises:

Obtaining the number of readings and writes per unit time in each data block in the storage system, and arranging the number of readings and readings per unit time of each data block in order from high to low;

The number of data blocks that need to be migrated from the first storage device to the second storage device according to the capacity ratio specifically includes:

Determining a data block corresponding to the capacity ratio according to the capacity ratio and the number of read and write times per unit time of each acquired data block;

A number of data blocks corresponding to the capacity ratio in the first storage device is confirmed.
The method according to any one of claims 1 to 4, wherein the method further comprises:

Obtaining the number of times of reading and writing of the first storage device per unit time;

The intensity ratio of the second storage device is the sum of the number of reading and writing times per unit time of the second storage device and the number of reading and writing times per unit time of the first storage device and the number of reading and writing times per unit time of the second storage device. proportion.
The method according to claim 5, wherein the number of readings and writes in the unit time of the first storage device is the number of readings and writes in a single disk unit time in the first storage device and the first storage device. The product of the number of disks in the second conversion factor is further divided by the ratio of the read request to the write request of the first storage device and the RAID level of the first storage device;

The number of times of reading and writing of the second storage device per unit time is the product of the number of reading and writing times per unit time in the second storage device and the number of disks in the second storage device divided by the second conversion a coefficient, the second conversion factor being related to a ratio of a read request to a write request of the second storage device and a RAID level of the second storage device.
The method according to claim 6, wherein the method further comprises: the number of times of reading and writing in a single disk unit time in the first storage device is related to a load characteristic of the first storage device and a response duration;

The number of read/write times per unit time in the second storage device in the second storage device is related to the load characteristics of the second storage device and the response duration.
A storage system for implementing data migration, characterized in that the storage system comprises a first storage device, a second storage device and a processor, the processor for performing the method of claims 1-7.
A storage system for implementing data migration, wherein the storage system includes a first storage device, a second storage device, and a processor, where the processor includes a data collection and analysis module and a data migration module;

The data collection and analysis module is configured to obtain the read/write times of the second storage device in a unit time; and obtain the intensity ratio of the second storage device according to the read/write times of the second storage device per unit time; the query intensity ratio and a capacity ratio correspondence table, obtaining a capacity ratio corresponding to the intensity ratio; obtaining, according to the capacity ratio, a number of data blocks that need to be migrated from the first storage device to the second storage device; The number of data blocks is sent to the data migration module;

The data migration module is configured to migrate the obtained number of data blocks from the first storage device to the second storage device.
The storage system according to claim 9, wherein the data collection and analysis module is further configured to:

The IO request of the storage system is analyzed and collected in advance, and a table of strength ratio and capacity ratio is obtained.
The storage system according to claim 9 or 10, wherein the data collection and analysis module is configured to obtain a capacity ratio corresponding to the intensity ratio by using a query strength ratio and a capacity ratio correspondence table, specifically: the data collection and analysis module is configured to query The intensity ratio to capacity ratio correspondence table obtains a capacity ratio corresponding to the intensity ratio by a fuzzy matching rule.
The storage system according to any one of claims 9-11, wherein the data collection and analysis module is further configured to acquire the number of readings and writes per unit time in each data block in the storage system, and obtain each of the obtained data. The number of readings and writes per unit time of a data block is arranged in order from high to low;

The data collection and analysis module obtains, according to the capacity ratio, the number of data blocks that need to be migrated from the first storage device to the second storage device, specifically: the data collection and analysis module obtains according to the capacity ratio in order from the The number of data blocks that the first storage device migrates to the second storage device.