WO2018113317A1 - Procédé de migration, appareil et système - Google Patents

Procédé de migration, appareil et système Download PDF

Info

Publication number
WO2018113317A1
WO2018113317A1 PCT/CN2017/096784 CN2017096784W WO2018113317A1 WO 2018113317 A1 WO2018113317 A1 WO 2018113317A1 CN 2017096784 W CN2017096784 W CN 2017096784W WO 2018113317 A1 WO2018113317 A1 WO 2018113317A1
Authority
WO
WIPO (PCT)
Prior art keywords
storage device
data
unit time
ratio
capacity ratio
Prior art date
Application number
PCT/CN2017/096784
Other languages
English (en)
Chinese (zh)
Inventor
周臣
董浩
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2018113317A1 publication Critical patent/WO2018113317A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0647—Migration mechanisms
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0673—Single storage device
    • G06F3/068—Hybrid storage device

Definitions

  • the present application relates to the field of communications technologies, and in particular, to a data migration method, apparatus, and system.
  • the use value of data has its own life cycle and can be followed regularly.
  • various types of metadata (such as part of the operating system data of a virtual machine) are of high value, are frequently accessed, and have high response speed requirements.
  • the frequency of newly generated data is usually accessed more frequently. Over time, the frequency of newly generated data will decrease and will not be accessed frequently.
  • Automatic tiered storage technology has thus emerged and is widely used in a variety of industries.
  • the automatic tiered storage technology stores data on different performance storage devices according to the characteristics of data access frequency, importance, retention time, etc.; and migrates data with low access frequency to lower performance based on the frequency at which data is accessed.
  • data that is frequently accessed is migrated to a storage device with high performance.
  • Data migration strategies include migrating low-access data to lower-performance storage devices and migrating frequently accessed data to high-performance storage devices.
  • data that is frequently accessed is preferentially migrated to a high-performance storage device.
  • the amount of data migrated is often determined according to the current available capacity of the high-performance storage device, and is generally performed by IO request for accessing data. Monitoring, the number of accessed IO requests reaches a certain amount of data and migrates to high-performance storage devices.
  • the embodiment of the present application provides a method, an apparatus, and a system for migrating data, which can accurately determine the amount of data of data that needs to be migrated from the first storage device to the second storage device, and migrate as much data as possible to the second storage device. At the same time, it can ensure that the performance of the second storage device is not affected, and the performance and efficiency of the storage system are improved.
  • a first aspect a method for migrating data in a storage system, the storage system first storage device and the second storage device, the method comprising: acquiring a read/write number of the second storage device per unit time; and according to the second storage device Obtaining the number of readings and the number of times per unit time, obtaining the intensity ratio of the second storage device; querying the intensity ratio to the capacity ratio correspondence table, obtaining a capacity ratio corresponding to the intensity ratio; obtaining the required ratio from the capacity ratio according to the a number of data blocks that a storage device migrates to the second storage device; and a corresponding data block is migrated from the first storage device to the second storage device according to the quantity.
  • the intensity ratio of the second storage device After obtaining the intensity ratio of the second storage device, according to the strength ratio and the capacity ratio correspondence table, it is required to migrate to the second storage.
  • the number of data blocks of the storage device In this way, the specific value of the migrated data can be determined according to the performance of the second storage device, which can ensure that the performance of the second storage device is not affected after the data is migrated, and the performance of the entire storage system is ensured.
  • the intensity ratio to capacity ratio correspondence table is obtained by pre-monitoring analysis.
  • the strength ratio to capacity ratio correspondence table can be directly queried to quickly obtain the number of data blocks that need to be migrated from the first storage device to the second storage device.
  • the query intensity ratio to capacity ratio correspondence table is obtained by a fuzzy matching rule to obtain a capacity ratio corresponding to the intensity ratio.
  • the ratio of the strength ratio calculated by the ratio of the intensity ratio to the capacity ratio is higher than the ratio of the intensity ratio of the higher one level to determine the corresponding capacity ratio, thereby improving the search efficiency.
  • the number of readings and writes per unit time in each storage block in the storage system is obtained, and the number of read and write times per unit time of each data block obtained is in order from high to low. arrangement.
  • the number of read and write times determines a data block corresponding to the capacity ratio; and the number of data blocks corresponding to the capacity ratio in the first storage device is confirmed.
  • the number of read and write times per unit time of each data block in the obtained storage system is arranged in order from high to low, so that The data block with higher read/write times per unit time can be migrated from the first storage device to the second storage device with higher performance in order according to the determined number of data blocks that need to be migrated, and the IO request can be responded more quickly.
  • the intensity ratio of the second storage device is the sum of the number of reading and writing times per unit time of the second storage device and the number of reading and writing times per unit time of the first storage device and the number of reading and writing times per unit time of the second storage device. proportion.
  • the number of reading and writing times of the first storage device per unit time is the product of the number of reading and writing times per unit time in the first storage device and the number of disks in the first storage device.
  • the first conversion factor is related to the ratio of the read request and the write request of the first storage device and the RAID level of the first storage device; the reading of the second storage device per unit time
  • the number of writes is a product of the number of reads and writes in a single disk unit time and the number of disks in the second storage device in the second storage device divided by a second conversion factor, the second conversion factor and the The ratio of the read request of the second storage device to the write request and the RAID level of the second storage device.
  • the number of readings and writes in a single disk unit time in the first storage device is related to the load characteristics of the first storage device and the response time; the single disk unit time in the second storage device The number of read and write times is related to the load characteristics of the second storage device and the response duration.
  • the device can guarantee the performance of the storage device and improve the efficiency and performance of the storage system.
  • a storage system for implementing data migration includes a first storage device, a second storage device, and a processor, and the processor is used in each step of the above method.
  • the processor is used in each step of the above method.
  • a storage system for implementing data migration includes a first storage device, a second storage device, and a processor.
  • the processor includes a data collection and analysis module and a data migration module.
  • the data collection and analysis module is configured to obtain the read/write times of the second storage device in a unit time; and obtain the intensity ratio of the second storage device according to the read/write times of the second storage device per unit time; the query intensity ratio and the capacity ratio Corresponding to the table, obtaining a capacity ratio corresponding to the intensity ratio; obtaining, according to the capacity ratio, the number of data blocks that need to be migrated from the first storage device to the second storage device; and obtaining the data block to be migrated The amount is sent to the data migration module.
  • the data migration module is configured to migrate the obtained number of data blocks from the first storage device to the second storage device.
  • the data collection and analysis module obtains the intensity ratio of the second storage device, and obtains the number of data blocks that need to be migrated to the second storage device according to the strength ratio and the capacity ratio correspondence table. In this way, the specific value of the migrated data can be determined according to the performance of the second storage device, which can ensure that the performance of the second storage device is not affected after the data is migrated, and the performance of the entire storage system is ensured.
  • the data acquisition and analysis module is further configured to: pre-collect an IO request of the analysis application to the storage system, and obtain a strength ratio to capacity ratio correspondence table.
  • the data collection and analysis module can directly query the strength ratio and capacity ratio correspondence table to quickly obtain the number of data blocks that need to be migrated from the first storage device to the second storage device.
  • the data acquisition and analysis module is configured to obtain a capacity ratio corresponding to the intensity ratio by using a query strength ratio and a capacity ratio correspondence table, and specifically: the data collection and analysis module is configured to query a strength ratio and a capacity ratio correspondence table.
  • a capacity ratio corresponding to the intensity ratio is obtained by a fuzzy matching rule.
  • the ratio of the strength ratio calculated by the ratio of the intensity ratio to the capacity ratio is higher than the ratio of the intensity ratio of the higher one level to determine the corresponding capacity ratio, thereby improving the search efficiency.
  • the data acquisition and analysis module is further configured to acquire the number of readings and writes per unit time in each data block in the storage system, and read and write the unit time of each data block obtained.
  • the number of data blocks from the first storage device to the second storage device is obtained by the data collection and analysis module according to the capacity ratio: The capacity ratio is obtained in order to obtain the number of data blocks that need to be migrated from the first storage device to the second storage device.
  • the data collection and analysis module presses the number of read and write times per unit time of each data block in the obtained storage system from high to low. Arrange sequentially, so that data blocks with higher read/write times per unit time can be sequentially migrated from the first storage device to the second storage device with higher performance according to the determined number of data blocks to be migrated, faster Respond to IO requests.
  • the data acquisition and analysis module is further configured to obtain the number of read and write times of the first storage device per unit time.
  • the intensity ratio of the second storage device is the proportion of the number of reading and writing times per unit time of the second storage device to the sum of the number of reading and writing times per unit time of the first storage device and the number of reading and writing times per unit time of the second storage device.
  • the number of readings and writes per unit time of the first storage device is the product of the number of readings and writes in a single disk unit time and the number of disks in the first storage device in the first storage device, and then divided by a conversion factor, wherein the first conversion factor is related to a ratio of a read request and a write request of the first storage device and a RAID level of the first storage device; and the number of read and write times of the second storage device per unit time is a single storage device
  • the product of the number of reads and writes in the unit time and the number of disks in the second storage device is divided by the second conversion factor, the second conversion factor and the read request of the second storage device It is related to the ratio of the write request and the RAID level of the second storage device.
  • the device can guarantee the performance of the storage device and improve the efficiency and performance of the storage system.
  • FIG. 1 is a schematic structural diagram of a storage system according to an embodiment of the present invention.
  • FIG. 2 is a schematic flowchart of a method for migrating data according to an embodiment of the present invention
  • FIG. 3 is a schematic structural diagram of another storage system according to an embodiment of the present invention.
  • An embodiment of the present invention provides a method for migrating data in a storage system, and calculates a size of data to be migrated by collecting and collecting load characteristics of an IO request to access the storage system, and estimating the performance of the storage device in the storage system. Maximize the performance of your storage system for the purpose of migrating data on demand
  • data whose access frequency reaches a predetermined value is referred to as hot data
  • data whose access frequency is lower than a predetermined value is referred to as cold data.
  • the storage system 1 includes a storage medium 101, a processor 103, and a cache 105.
  • the storage medium 101 in the storage system 1 can be various and divided into different performance layers according to the performance of the storage medium.
  • the storage medium in the storage system consists of an SSD disk and a common hard disk.
  • the SSD disk constitutes a high-performance layer storage device
  • the ordinary hard disk constitutes a storage device of a common performance layer.
  • the high performance layer and the common performance layer are included in the storage system as an example.
  • the common performance layer in the storage system is referred to as a first storage device, and the high performance in the storage system.
  • the layer is called the second storage device.
  • the hot data is moved to the high performance layer as an example, and the hot data cooling from the high performance layer to the common performance layer can also be processed by the same method.
  • the data can be processed in the same way when migrating between different performance layers.
  • the processor 103 is configured to execute the method of migrating data described below, and the cache 105 is configured to store required information.
  • the flow of the method for migrating data in the storage system provided by the embodiment of the present invention is as shown in FIG. 2, and is completed by a processor in the storage system.
  • the method embodiment of the present invention mainly describes an implementation manner of migrating data from a common performance layer to a high performance layer, which is described in detail below.
  • the common performance layer in the storage system is referred to as a first storage device
  • the high performance layer in the storage system is referred to as a second storage device.
  • the IO request in the embodiment of the present invention is an object of monitoring and analyzing the IO request for accessing the storage medium.
  • Step 201 Acquire the number of times of reading and writing of the second storage device per unit time.
  • the processor obtains the number of times of reading and writing of the second storage device in a unit time.
  • the unit time can be determined by the user according to the service type, which is not limited in the embodiment of the present invention.
  • the unit time is 1 second, the number of read/write times per second (IOPS) of the second storage device is obtained.
  • each disk in the same performance layer storage device has the same type.
  • Embodiments of the present invention provide A method for obtaining the number of reading and writing times per unit time of the second storage device.
  • the number of reads and writes per unit time of the second storage device (number of read/write times per unit time * number of disks) / RAID conversion factor.
  • the number of disks is the number of disks in the second storage device.
  • the RAID conversion factor is related to the ratio of read and write IO and the write penalty coefficient.
  • the number of reads and writes in a single disk unit is related to the load characteristics and response duration of the second storage device.
  • the data can be collected in advance and obtained statistically, as shown in Table 2.
  • the load characteristic of the second storage device here is the load characteristic of the IO request to access the second storage device.
  • the IOPS of the single disk in the second storage device can be directly found according to the load characteristic and the response time value of the IO request of the second storage device. The above method calculates the number of times of reading and writing in the second storage device time.
  • Table 2 Lookup table of the number of read/write times per unit time in the second storage device
  • the number of times of reading and writing in a single disk unit time can also be obtained according to the load characteristics and the response time length of the IO request for accessing the second storage device, and the value of the second table is not required to be obtained in advance.
  • Step 203 Acquire an intensity ratio of the second storage device according to the number of accesses of the second storage device per unit time.
  • the strength ratio of the second storage device is a ratio of the number of accesses per unit time of the second storage device to the number of accesses per unit time of the storage system.
  • the number of accesses of the second storage device per unit time has been described in step 201 and will not be further described herein.
  • the method for obtaining the number of accesses per unit time of the first storage device is the same as the method for obtaining the number of accesses per unit time of the second storage device, and is not further described.
  • the reading of the single disk unit time in the first storage device The number of writes can also be collected in advance and statistically obtained, as shown in Table 3.
  • Table 3 Lookup table for the number of read/write times per unit time in the first storage device
  • Tables 2 and 3 can be combined into one table storage, that is, the number of readings and writes in a single disk unit time in each performance layer in the storage system can be reflected in a table, as shown in Table 4. .
  • the number of readings and writes per unit time in a single disk in other performance layers may be reflected in one table, or may be embodied in different tables, in the implementation of the present invention. There is no limit in the example.
  • Table 4 Lookup table for the number of read and write times per unit time in the storage system
  • Step 205 Query a correspondence table between the intensity ratio and the capacity ratio, and obtain a capacity ratio corresponding to the intensity ratio.
  • the correspondence table between the intensity ratio and the capacity ratio is obtained by monitoring and analyzing the business data in advance.
  • the service data in the storage system (including the first storage device and the second storage device) is split into a plurality of data blocks, and the number of read and write times of the data blocks in a unit time, that is, the access strength of the data block is obtained.
  • Business data can be split into multiple data blocks according to the granularity of migration, or can be split into multiple data blocks according to the specified size.
  • the number of reads and writes of these data blocks is accessed during the statistical monitoring time, and the number of read and write times of the data blocks per unit time is calculated, that is, the access strength.
  • the data blocks in the storage system are sorted according to the access intensity from high to low, and then the data blocks are divided into groups according to a certain capacity ratio. For example, as shown in Table 3, data blocks are grouped by a capacity ratio of 2.5%, that is, data1 to dataA form a group having a capacity ratio of 2.5%, that is, data1 to data2A form a group having a capacity ratio of 5%... and so on. As shown in Table 5.
  • Capacity ratio Intensity ratio 0.0% 0.0% 2.5% 35.0% 5.0% 50.0% 7.5% 60.0% 10.0% 65.0% 12.5% 67.0% 15.0% 69.0% 20.0% 70.0% 25.0% 71.5% 30.0% 72.0% 100.0% 100.0%
  • Table 6 Correspondence table between intensity ratio and capacity ratio of a typical load
  • the number of I/O accesses is 65% of the total number of I/O accesses in the storage system.
  • the strength ratio of the data block is 70%
  • the corresponding capacity ratio is 20%
  • the P hot spot capacity ratio [70%] 20%, indicating that the number of I/O accesses accounts for all data blocks I/ in the storage system.
  • the number of O accesses 70% of the data blocks accounts for 20% of all data blocks in the storage system.
  • the data block access situation of the primary business model can be used as a statistical basis to obtain a strength ratio to capacity ratio correspondence table.
  • the capacity ratio corresponding to the strength ratio of the second storage device can be obtained by querying the capacity ratio correspondence table.
  • the query rule may adopt a fuzzy matching rule, that is, a query strength ratio and a capacity ratio correspondence table, and obtain a capacity ratio corresponding to the intensity ratio by a fuzzy matching rule. If the intensity ratio of the query needs to be inaccurately matched in the intensity ratio to the capacity ratio correspondence table, the corresponding capacity ratio is searched for according to the intensity ratio of the obtained second storage device and the intensity ratio of the higher one. If the data block to be migrated according to the capacity ratio obtained by the query satisfies the condition, the capacity ratio is calculated.
  • the strength ratio of the obtained second storage device is lower than the strength ratio of the lower one to find the corresponding capacity ratio.
  • Step 207 Obtain a quantity of data blocks that need to be migrated from the first storage device to the second storage device according to the capacity ratio.
  • the number of data blocks corresponding to the capacity ratio is obtained. Obtaining the number of readings and writes per unit time in each data block in the storage system, and reading the unit time of each data block obtained The number of writes is arranged in order from high to low; according to the obtained number of data blocks corresponding to the capacity ratio, a corresponding number of data blocks are selected according to the access intensity from high to low, and it is determined that the first storage device needs to be migrated to the second The number of data blocks of the storage device.
  • the following is an example in which the data block has not been moved in the second storage device. If the data block exists in the second storage device, the corresponding value is subtracted from the value already in the second storage device. , no longer explain one by one.
  • the amount of migrated data P capacity ratio [second storage device IOPS / (second storage device IOPS + first storage device IOPS)] * total data amount. If the calculated amount of data of the data block to be migrated is smaller than the total capacity of the second storage device, the number of data blocks that need to be migrated is the number of the calculated data blocks. If the calculated data volume of the data block to be migrated is greater than the total capacity of the second storage device, the number of data blocks that need to be migrated is based on the total capacity of the second storage device.
  • Step 209 Migrate the corresponding data block from the first storage device to the second storage device according to the quantity.
  • the number of data blocks that need to be migrated from the first storage device to the second storage device calculated according to step 207, and the corresponding number of data blocks are migrated from the first storage device to the second storage device according to the access intensity from high to low. in.
  • the data that accounts for 66% of the storage system access strength is migrated to the second storage device. Since the second storage device is a high-performance storage device, the user's request can be responded to in time. Moreover, the amount of data migrated does not exceed the capacity of the second storage device, and does not affect the performance of the second storage device, thereby ensuring the overall performance of the storage system.
  • the embodiment of the present invention further provides a storage system 3, which can implement the data migration method as described above, and the structure of the storage system 3 is as shown in FIG. 3.
  • the storage system 3 includes a first storage device 321 and a second storage device 323, and the performance of the first storage device 321 and the second storage device 323 are different.
  • the first storage device 321 is composed of a storage medium of a general performance, such as a conventional disk, and can store user data.
  • the second storage device 323 is composed of a high-performance storage medium, such as an SSD disk, and can be used to store hot data. .
  • a description will be made using the storage device.
  • the storage device in the embodiment of the present invention is only an exemplary description. In practical applications, storage media with different performances may also be added. The number of the storage media included in the storage device may also be set as needed, which is not limited in the embodiment of the present invention. In addition, the storage device may be composed of a storage medium of the same performance, or may be composed of a storage medium having similar performance.
  • the embodiment of the present invention is described by taking an example of migrating data in the first storage device 321 to the second storage device 323. In this embodiment, only the functions of the components in the storage system 3 are briefly described. For details of the method steps involved, refer to the description of the method embodiments in the foregoing.
  • the storage system 3 further includes a processor 31 including a data acquisition and analysis module 311 and a data migration module 313.
  • the data collection and analysis module 311 is configured to perform an analysis and calculation on the IO request of the application access storage device, obtain the number of data blocks that need to be migrated from the first storage device 321 to the second storage device 323, and notify the data migration module 313.
  • the data migration module 313 is configured to migrate the obtained number of data blocks from the first storage device 321 to the second storage device 323.
  • the data collection and analysis module 311 is configured to acquire the read/write times of the second storage device 323 in a unit time; and obtain the intensity ratio of the second storage device 323 according to the read/write times of the second storage device 323 per unit time.
  • a query intensity ratio to capacity ratio correspondence table obtaining a capacity ratio corresponding to the intensity ratio; obtaining a number of data blocks that need to be migrated from the first storage device 321 to the second storage device 323 according to the capacity ratio
  • the obtained number of data blocks that need to be migrated is sent to the data migration module 313.
  • the data collection and analysis module 311 obtains the intensity ratio of the second storage device 323, and obtains the number of data blocks that need to be migrated to the second storage device 323 according to the intensity ratio and capacity ratio correspondence table. In this way, the amount of migrated data can be determined according to the performance of the second storage device 323, and the performance of the second storage device 323 can be ensured after the data is migrated, thereby ensuring the performance of the entire storage system.
  • the data migration module 313 is configured to migrate the corresponding data block from the first storage device 321 to the second storage device 323 according to the received number of data blocks that need to be migrated.
  • the data collection and analysis module 311 is further configured to pre-collect an IO request of the analysis application to the storage system 3 to obtain an intensity ratio and capacity ratio correspondence table. After the intensity ratio of the second storage device 323 is obtained, the strength ratio and capacity ratio correspondence table can be directly queried, and the number of data blocks that need to be migrated from the first storage device 321 to the second storage device 323 can be quickly obtained.
  • the data collection and analysis module 311 is configured to obtain a capacity ratio corresponding to the intensity ratio by using the query strength ratio and the capacity ratio correspondence table.
  • the data collection and analysis module 311 is configured to query the intensity ratio and the capacity ratio correspondence table, and obtain the The intensity ratio is a corresponding capacity ratio.
  • the ratio of the strength ratio calculated by the ratio of the intensity ratio to the capacity ratio is higher than the ratio of the intensity ratio of the higher one level to determine the corresponding capacity ratio, thereby improving the search efficiency.
  • the data collection and analysis module 311 is further configured to acquire the number of reading and writing times per unit time in each data block in the storage system 3, and press the number of times of reading and writing in the unit time of each data block obtained from high to low. Arranging sequentially; determining a data block corresponding to the capacity ratio according to the capacity ratio and the number of read and write times per unit time of each acquired data block; confirming in the first storage device 321 in order
  • the capacity ratio is the number of corresponding data blocks.
  • the data collection and analysis module 311 will read and write the number of read and write times per unit time of each data block in the storage system 3. Arranged in order from high to low, so that data blocks with higher read/write times per unit time can be sequentially migrated from the first storage device 321 to the second higher performance module according to the determined number of data blocks to be migrated. In storage device 323, the IO request is responded more quickly.
  • the data collection and analysis module 311 is further configured to acquire the number of times of reading and writing of the first storage device 321 per unit time.
  • the intensity ratio of the second storage device 323 is the number of reading and writing times per unit time of the second storage device 323, and the number of reading and writing times per unit time of the first storage device 321 and the number of reading and writing times per unit time of the second storage device 323. And the proportion.
  • the number of times of reading and writing of the first storage device 321 per unit time is the product of the number of reading and writing times per unit time in the first storage device 321 and the number of disks in the first storage device 321 divided by a first conversion factor, the first conversion factor is related to a ratio of a read request and a write request of the first storage device 321 and a RAID level of the first storage device 321;
  • the number of times of reading and writing of the second storage device 323 per unit time is the product of the number of reading and writing times per unit time in the second storage device 323 and the number of disks in the second storage device 323 divided by a second conversion factor, the second conversion factor being related to a ratio of a read request to a write request of the second storage device 323 and a RAID level of the second storage device 323.
  • the storage device 323 can ensure the performance of the storage device 323 and improve the efficiency and performance of the storage system.
  • the embodiment of the present application further provides a computer storage medium for storing computer software instructions used by the storage system, which includes a program designed to execute the foregoing method embodiments. By executing a stored program, a method of migrating data between storage devices can be implemented.
  • the embodiment of the present application further provides a computer program, which includes instructions, when the computer program is executed by a computer, to enable the computer to execute the flow of the foregoing method embodiment.
  • embodiments of the present application can be provided as a method, apparatus (device), or computer program product.
  • the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment in combination of software and hardware.
  • the application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.
  • the computer program is stored/distributed in a suitable medium, provided with other hardware or as part of the hardware, or in other distributed forms, such as over the Internet or other wired or wireless telecommunication systems.
  • the computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device.
  • the apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device.
  • the instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Debugging And Monitoring (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

La présente invention concerne un procédé de migration de donnée d'un système de mémorisation. Le système de mémorisation comprend un dispositif de mémorisation source et un dispositif de mémorisation de sauvegarde. Le procédé comprend les étapes suivantes consistant à : obtenir le nombre d'opérations de lecture/écriture par unité de temps du second dispositif de mémorisation; obtenir, en fonction du nombre d'opérations de lecture/écriture par unité de temps du second dispositif de mémorisation, d'un rapport de résistance du second dispositif de mémorisation; rechercher une table de correspondance de rapport de force par rapport à la capacité, pour obtenir un rapport de capacité correspondant au rapport de force; obtenir, en fonction du rapport de capacité obtenu, le nombre de blocs de données qui doivent être migrés du premier dispositif de mémorisation au second dispositif de mémorisation; et faire migrer, en fonction du nombre, des blocs de données correspondants du premier dispositif de mémorisation au second dispositif de mémorisation Au moyen du procédé de l'invention, le nombre de blocs de données à migrer du premier dispositif de stockage au second dispositif de mémorisation peut être déterminé avec précision, ce qui permet d'assurer que les performances du second dispositif de mémorisation ne sont pas affectées tout en ayant autant de données que possible qui sont migrées vers le second dispositif de mémorisation, et d'améliorer les performances et l'efficacité du système de mémorisation.
PCT/CN2017/096784 2016-12-22 2017-08-10 Procédé de migration, appareil et système WO2018113317A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201611199849.1A CN107122126B (zh) 2016-12-22 2016-12-22 数据的迁移方法、装置和系统
CN201611199849.1 2016-12-22

Publications (1)

Publication Number Publication Date
WO2018113317A1 true WO2018113317A1 (fr) 2018-06-28

Family

ID=59717905

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/096784 WO2018113317A1 (fr) 2016-12-22 2017-08-10 Procédé de migration, appareil et système

Country Status (2)

Country Link
CN (1) CN107122126B (fr)
WO (1) WO2018113317A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110825908A (zh) * 2019-11-04 2020-02-21 安超云软件有限公司 一种对象的迁移方法、装置、电子设备及存储介质
CN111124281A (zh) * 2019-11-29 2020-05-08 浪潮(北京)电子信息产业有限公司 全闪存储系统的数据存储方法、装置、设备及存储介质
CN112015347A (zh) * 2020-08-28 2020-12-01 北京浪潮数据技术有限公司 一种数据迁移方法、装置及相关设备
CN113741810A (zh) * 2021-07-30 2021-12-03 苏州浪潮智能科技有限公司 一种数据迁移方法及装置

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107894873A (zh) * 2017-11-07 2018-04-10 长沙曙通信息科技有限公司 一种存储虚拟化系统数据处理实现方法
CN110413209B (zh) * 2018-04-28 2023-05-30 伊姆西Ip控股有限责任公司 管理存储系统的方法和设备
CN110018797B (zh) * 2019-04-11 2020-03-06 苏州浪潮智能科技有限公司 一种数据迁移方法、装置、设备及可读存储介质
CN111078160B (zh) * 2020-03-23 2020-06-23 上海飞旗网络技术股份有限公司 基于调用比例及响应时长的数据迁移存储方法及装置
CN114327253B (zh) * 2021-10-18 2024-05-28 杭州逗酷软件科技有限公司 数据迁移方法、装置、电子设备及存储介质

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104572684A (zh) * 2013-10-17 2015-04-29 深圳市腾讯计算机系统有限公司 数据存储系统的容量调整方法、装置及数据存储系统
CN104598495A (zh) * 2013-10-31 2015-05-06 南京中兴新软件有限责任公司 基于分布式文件系统的分级存储方法及系统

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101498673B1 (ko) * 2007-08-14 2015-03-09 삼성전자주식회사 반도체 드라이브, 그것의 데이터 저장 방법, 그리고 그것을포함한 컴퓨팅 시스템
US8700841B2 (en) * 2010-04-19 2014-04-15 International Business Machines Corporation Sub-LUN input/output profiling for SSD devices
CN102033717B (zh) * 2010-12-07 2013-05-08 清华大学 基于磁盘阵列的数据存储方法及系统
CN102541460B (zh) * 2010-12-20 2014-10-08 中国移动通信集团公司 一种多磁盘场景下的磁盘管理方法和设备
WO2012106923A1 (fr) * 2011-07-25 2012-08-16 华为技术有限公司 Procédé d'ajustement d'objet, dispositif de commande de migration, dispositif formant nœud et système de stockage
CN103106044B (zh) * 2012-12-13 2016-09-07 深圳先进技术研究院 分级存储节能方法
CN103106152B (zh) * 2012-12-13 2015-09-23 深圳先进技术研究院 基于层次存储介质的数据调度方法
CN103091707B (zh) * 2012-12-31 2016-06-22 中国石油集团川庆钻探工程有限公司地球物理勘探公司 地震数据分级存储装置及方法
CN104102454A (zh) * 2013-04-07 2014-10-15 杭州信核数据科技有限公司 一种自动实现分层存储的方法和管理分层存储的系统
JP5881900B2 (ja) * 2013-06-29 2016-03-09 華為技術有限公司Ｈｕａｗｅｉ Ｔｅｃｈｎｏｌｏｇｉｅｓ Ｃｏ．，Ｌｔｄ． ストレージアレイを管理する方法及び装置
CN103616944B (zh) * 2013-10-18 2016-05-11 杭州电子科技大学 云存储系统中基于预判性绿色数据分类策略的降耗方法
US10120601B2 (en) * 2014-03-24 2018-11-06 Hitachi, Ltd. Storage system and data processing method
CN104268099B (zh) * 2014-08-29 2017-06-13 浪潮(北京)电子信息产业有限公司 一种管理数据读写的方法及装置
CN105653524B (zh) * 2014-11-10 2021-07-30 创新先进技术有限公司 一种数据存储方法、装置和系统
CN104536904B (zh) * 2014-12-29 2018-01-09 杭州华为数字技术有限公司 一种数据管理的方法、设备与系统

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104572684A (zh) * 2013-10-17 2015-04-29 深圳市腾讯计算机系统有限公司 数据存储系统的容量调整方法、装置及数据存储系统
CN104598495A (zh) * 2013-10-31 2015-05-06 南京中兴新软件有限责任公司 基于分布式文件系统的分级存储方法及系统

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110825908A (zh) * 2019-11-04 2020-02-21 安超云软件有限公司 一种对象的迁移方法、装置、电子设备及存储介质
CN110825908B (zh) * 2019-11-04 2023-04-25 安超云软件有限公司 一种对象的迁移方法、装置、电子设备及存储介质
CN111124281A (zh) * 2019-11-29 2020-05-08 浪潮(北京)电子信息产业有限公司 全闪存储系统的数据存储方法、装置、设备及存储介质
CN111124281B (zh) * 2019-11-29 2023-11-03 浪潮(北京)电子信息产业有限公司 全闪存储系统的数据存储方法、装置、设备及存储介质
CN112015347A (zh) * 2020-08-28 2020-12-01 北京浪潮数据技术有限公司 一种数据迁移方法、装置及相关设备
CN113741810A (zh) * 2021-07-30 2021-12-03 苏州浪潮智能科技有限公司 一种数据迁移方法及装置
CN113741810B (zh) * 2021-07-30 2023-08-11 苏州浪潮智能科技有限公司 一种数据迁移方法及装置

Also Published As

Publication number Publication date
CN107122126A (zh) 2017-09-01
CN107122126B (zh) 2020-09-08

Similar Documents

Publication Publication Date Title
WO2018113317A1 (fr) Procédé de migration, appareil et système
US10168946B2 (en) Extent migration in multi-tier storage systems
US10031671B2 (en) Method, apparatus, and system for calculating identification threshold to distinguish cold data and hot data
CN111064808B (zh) 一种基于分布式存储系统的负载均衡方法及装置
US20120158799A1 (en) Automatically matching data sets with storage components
CN106469018B (zh) 分布式存储系统的负载监控方法及设备
CN111367469B (zh) 一种分层存储数据迁移方法和系统
CN106156283B (zh) 基于数据温度和节点性能的异构Hadoop存储方法
EP2494436A1 (fr) Allocation de mémoire de stockage sur la base d'estimations d'utilisations futures
WO2017162086A1 (fr) Procédé et dispositif de planification de tâche
US20170315924A1 (en) Dynamically Sizing a Hierarchical Tree Based on Activity
WO2013071726A1 (fr) Procédé et dispositif de préchauffage de données
CN105630810A (zh) 一种对于海量小文件在分布式存储系统中上载的方法
JP6823605B2 (ja) データ保持フレームワーク
US20150212744A1 (en) Method and system of eviction stage population of a flash memory cache of a multilayer cache system
CA2987731A1 (fr) Surveillance de memoire de base de donnees et defragmentation d'index de base de donnees
CN110688360A (zh) 分布式文件系统存储管理方法、装置、设备及存储介质
Kim et al. Selective I/O bypass and load balancing method for write-through SSD caching in big data analytics
CN109947667B (zh) 数据访问预测方法和装置
US9778854B2 (en) Computer system and method for controlling hierarchical storage therefor
US11636081B2 (en) Dynamic index management for computing storage resources
US20150149418A1 (en) Estimation of query input/output (i/o) cost in database
US8938479B1 (en) Systems and methods for dynamically selecting a logical location for an index
JP2018018133A (ja) 情報処理装置、ストリームストレージ制御プログラム、及びインデックスデータ参照方法
WO2016029481A1 (fr) Procédé et dispositif pour isoler des régions de disque

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17884759

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17884759

Country of ref document: EP

Kind code of ref document: A1