CN107122126B - Data migration method, device and system - Google Patents
Data migration method, device and system Download PDFInfo
- Publication number
- CN107122126B CN107122126B CN201611199849.1A CN201611199849A CN107122126B CN 107122126 B CN107122126 B CN 107122126B CN 201611199849 A CN201611199849 A CN 201611199849A CN 107122126 B CN107122126 B CN 107122126B
- Authority
- CN
- China
- Prior art keywords
- storage device
- data
- unit time
- ratio
- capacity ratio
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0647—Migration mechanisms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/068—Hybrid storage device
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
A method of migrating data in a storage system, the storage system including a first storage device and a second storage device, the method comprising the steps of: acquiring the read-write times of the second storage device in unit time; acquiring the intensity ratio of a second storage device according to the reading and writing times of the second storage device in unit time; inquiring a corresponding table of the intensity ratio and the capacity ratio to obtain the capacity ratio corresponding to the intensity ratio; obtaining the number of data blocks which need to be migrated from the first storage device to the second storage device according to the obtained capacity ratio; and migrating the corresponding data blocks from the first storage device to the second storage device according to the number. By the method, the number of the data blocks transferred from the first storage device to the second storage device can be accurately determined, the performance of the second storage device can be guaranteed not to be influenced while the data are transferred to the second storage device as much as possible, and the performance and the efficiency of the storage system are improved.
Description
Technical Field
The present application relates to the field of communications technologies, and in particular, to a method, an apparatus, and a system for migrating data.
Background
As networks evolve, more and more data is generated. Different data have different use values, some data are frequently accessed, some data are not accessed for a long time, and other data are frequently accessed at certain time.
The use value of the data has the life cycle of the data, and the data can be regularly circulated. In general, various types of metadata (e.g., part of the operating system data of a virtual machine) are valuable to use, frequently accessed, and have high requirements for response speed. The frequency with which newly generated data is accessed is also relatively high. Over time, however, newly generated data may be used less frequently and are no longer accessed frequently.
Automated hierarchical storage techniques have thus been developed and widely used in various industries. The automatic hierarchical storage technology stores data on storage equipment with different performances according to the characteristics of access frequency, importance, retention time and the like of the data; and migrating the data with low access frequency to the storage equipment with lower performance and migrating the data frequently accessed to the storage equipment with high performance based on the access frequency of the data.
The data migration policy includes migrating data that is accessed less frequently to lower performance storage devices and migrating data that is accessed frequently to higher performance storage devices. Data frequently accessed is preferentially selected to be migrated to the high-performance storage device during data migration, the migrated data volume is often determined according to the current available capacity of the high-performance storage device, and the data with the accessed IO request volume reaching a certain volume is migrated to the high-performance storage device generally by monitoring IO requests for accessing data.
The inventor finds that the data migration strategy of the existing automatic hierarchical storage technology is simple, the overall performance of the migrated storage system cannot be predicted, and the problem of overload of high-performance storage equipment is possibly caused.
Disclosure of Invention
The embodiment of the application provides a method, a device and a system for migrating data, which can accurately determine the data volume of data needing to be migrated from a first storage device to a second storage device, ensure that the performance of the second storage device is not affected while migrating as much data as possible to the second storage device, and improve the performance and efficiency of a storage system.
The embodiment of the application provides the following technical scheme:
in a first aspect, a method for migrating data in a storage system is provided, where the storage system includes a first storage device and a second storage device, and the method includes: acquiring the read-write times of the second storage device in unit time; acquiring the strength ratio of the second storage device according to the reading and writing times of the second storage device in unit time; inquiring a corresponding table of the intensity ratio and the capacity ratio to obtain the capacity ratio corresponding to the intensity ratio; obtaining the number of data blocks which need to be migrated from the first storage device to the second storage device according to the capacity ratio; and migrating the corresponding data blocks from the first storage device to the second storage device according to the number.
And after the intensity ratio of the second storage equipment is obtained, the number of the data blocks needing to be migrated to the second storage equipment is obtained according to the intensity ratio and capacity ratio corresponding table. Therefore, the specific numerical value of the migrated data can be determined according to the performance of the second storage device, the performance of the second storage device is not affected after the data is migrated, and the performance of the whole storage system is ensured.
In one possible design, the intensity ratio to capacity ratio correspondence table is obtained by a pre-monitoring analysis.
After the intensity ratio of the second storage device is obtained, the intensity ratio and capacity ratio corresponding table can be directly inquired, and the number of data blocks needing to be migrated from the first storage device to the second storage device can be quickly obtained.
In one possible design, the corresponding table of intensity ratio and capacity ratio is queried, and the capacity ratio corresponding to the intensity ratio is obtained through a fuzzy matching rule.
Therefore, when the specific numerical value of the calculated intensity ratio cannot be found in the intensity ratio and capacity ratio corresponding table, the corresponding capacity ratio can be determined according to the fuzzy matching rule by taking the value of the intensity ratio which is higher than the calculated intensity ratio by one level than the value of the intensity ratio in the intensity ratio and capacity ratio corresponding table, and the searching efficiency is improved.
In one possible design, the number of reads and writes per unit time of each data block in the storage system is obtained, and the obtained number of reads and writes per unit time of each data block is arranged in order from high to low. Obtaining the number of data blocks that need to be migrated from the first storage device to the second storage device according to the capacity ratio specifically includes: determining a data block corresponding to the capacity ratio according to the capacity ratio and the obtained read-write times of each data block in unit time; confirming the number of data blocks corresponding to the capacity ratio in the first storage device.
In order to determine that data with a large number of accesses is migrated from a first storage device to a second storage device, the number of read-write times of each data block in a unit time in the obtained storage system is arranged in sequence from high to low, so that the data block with the high number of read-write times in the unit time can be migrated from the first storage device to the second storage device with higher performance in sequence according to the determined number of the data blocks needing to be migrated, and an IO request can be responded more quickly.
In one possible design, the number of times of reading and writing of the first storage device in unit time is acquired;
the strength ratio of the second storage device is the proportion of the number of times of reading and writing of the second storage device in unit time to the sum of the number of times of reading and writing of the first storage device in unit time and the number of times of reading and writing of the second storage device in unit time.
In one possible design, the number of reads and writes per unit time of the first storage device is the product of the number of reads and writes per unit time of a single disk in the first storage device and the number of disks in the first storage device, and then divided by a first scaling factor, where the first scaling factor is related to the ratio of read requests and write requests of the first storage device and the RAID level of the first storage device; the number of read/write times in unit time of the second storage device is the product of the number of read/write times in unit time of a single disk in the second storage device and the number of disks in the second storage device, and then is divided by a second conversion coefficient, wherein the second conversion coefficient is related to the ratio of read requests and write requests of the second storage device and the RAID level of the second storage device.
In one possible design, the number of reads and writes per unit time of a single disk in the first storage device is related to the load characteristics and the response duration of the first storage device; and the read-write times in the unit time of the single disk in the second storage device are related to the load characteristics and the response time length of the second storage device.
Through setting and obtaining the specific values of the parameters, the number of data blocks which need to be migrated from the first storage device to the second storage device can be accurately obtained, the performance of the storage device can be ensured while hot data are migrated to the high-performance second storage device as much as possible, and the efficiency and the performance of the storage system are improved.
In a second aspect, a storage system for implementing data migration is provided, where the storage system includes a first storage device, a second storage device, and a processor, and the processor is used for each step in the above method. For implementation details of the respective steps and corresponding benefits, please refer to the related description in the first aspect.
In a third aspect, a storage system for implementing data migration is provided, where the storage system includes a first storage device, a second storage device, and a processor, and the processor includes a data acquisition and analysis module and a data migration module. The data acquisition and analysis module is used for acquiring the read-write times of the second storage device in unit time; acquiring the strength ratio of the second storage device according to the reading and writing times of the second storage device in unit time; inquiring a corresponding table of the intensity ratio and the capacity ratio to obtain the capacity ratio corresponding to the intensity ratio; obtaining the number of data blocks which need to be migrated from the first storage device to the second storage device according to the capacity ratio; and sending the obtained number of the data blocks needing to be migrated to the data migration module. The data migration module is used for migrating the obtained number of data blocks from the first storage device to the second storage device.
And the data acquisition and analysis module acquires the intensity ratio of the second storage equipment and obtains the number of data blocks needing to be transferred to the second storage equipment according to the intensity ratio and capacity ratio corresponding table. Therefore, the specific numerical value of the migrated data can be determined according to the performance of the second storage device, the performance of the second storage device is not affected after the data is migrated, and the performance of the whole storage system is ensured.
In one possible design, the data collection and analysis module is further configured to: and (4) acquiring and analyzing IO requests of the application to the storage system in advance to obtain a corresponding table of the intensity ratio and the capacity ratio.
Therefore, after the intensity ratio of the second storage device is obtained, the data acquisition and analysis module can directly inquire the intensity ratio and capacity ratio corresponding table to quickly obtain the number of data blocks needing to be migrated from the first storage device to the second storage device.
In one possible design, the data acquisition and analysis module is configured to query the intensity ratio-to-capacity ratio correspondence table to obtain a capacity ratio corresponding to the intensity ratio, specifically: the data acquisition and analysis module is used for inquiring the corresponding table of the intensity ratio and the capacity ratio and acquiring the capacity ratio corresponding to the intensity ratio through a fuzzy matching rule.
Therefore, when the specific numerical value of the calculated intensity ratio cannot be found in the intensity ratio and capacity ratio corresponding table, the corresponding capacity ratio can be determined according to the fuzzy matching rule by taking the value of the intensity ratio which is higher than the calculated intensity ratio by one level than the value of the intensity ratio in the intensity ratio and capacity ratio corresponding table, and the searching efficiency is improved.
In one possible design, the data acquisition and analysis module is further configured to acquire the number of read-write times of each data block in the storage system in unit time, and sequentially arrange the acquired number of read-write times of each data block in unit time from high to low; the data acquisition and analysis module obtains the number of data blocks which need to be migrated from the first storage device to the second storage device according to the capacity ratio, and specifically comprises: and the data acquisition and analysis module obtains the number of data blocks which need to be migrated from the first storage device to the second storage device in sequence according to the capacity ratio.
In order to determine that data with a large number of accesses is migrated from the first storage device to the second storage device, the data acquisition and analysis module sequentially arranges the read-write times of each data block in the acquired storage system in unit time from high to low, so that the data block with a high read-write time in unit time can be migrated from the first storage device to the second storage device with higher performance in sequence according to the determined number of the data blocks to be migrated, and an IO request can be responded more quickly.
In a possible design, the data acquisition and analysis module is further configured to obtain the number of times of reading and writing of the first storage device in a unit time. The strength ratio of the second storage device is the proportion of the number of times of reading and writing of the second storage device in unit time to the sum of the number of times of reading and writing of the first storage device in unit time and the number of times of reading and writing of the second storage device in unit time.
In one possible design, the number of reads and writes in a unit time of the first storage device is the product of the number of reads and writes in a unit time of a single disk in the first storage device and the number of disks in the first storage device, and then is divided by a first conversion coefficient, wherein the first conversion coefficient is related to the ratio of read requests and write requests of the first storage device and the RAID level of the first storage device; the number of read/write times in unit time of the second storage device is the product of the number of read/write times in unit time of a single disk in the second storage device and the number of disks in the second storage device, and the product is divided by a second conversion coefficient, wherein the second conversion coefficient is related to the ratio of read requests and write requests of the second storage device and the RAID level of the second storage device.
Through setting and obtaining the specific values of the parameters, the number of data blocks which need to be migrated from the first storage device to the second storage device can be accurately obtained, the performance of the storage device can be ensured while hot data are migrated to the high-performance second storage device as much as possible, and the efficiency and the performance of the storage system are improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below.
FIG. 1 is a schematic structural diagram of a memory system according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a method for migrating data according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of another memory system according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a method for migrating data in a storage system, which calculates the size of data to be migrated through the collection and statistics of load characteristics of an IO (input/output) request for accessing the storage system and the estimation of the performance of storage equipment in the storage system so as to achieve the aim of migrating data as required and give full play to the performance of the storage system
In the embodiment of the present invention, data whose access frequency reaches a predetermined value is referred to as hot data, and data whose access frequency is lower than the predetermined value is referred to as cold data.
A storage system to which embodiments of the present invention are applicable is shown in fig. 1. The storage system 1 includes a storage medium 101, a processor 103, and a cache 105. The storage medium 101 in the storage system 1 may be various and divided into different performance layers according to the performance of the storage medium. For example, the storage medium in the storage system is composed of an SSD disk and a normal hard disk, the SSD disk constitutes a high performance layer storage device, and the normal hard disk constitutes a storage device of a normal performance layer. In the embodiment of the present invention, an example is described in which the memory system includes a high performance layer and a normal performance layer, and for convenience of description, the normal performance layer in the memory system is referred to as a first memory device, and the high performance layer in the memory system is referred to as a second memory device. In the embodiments of the present invention, the migration of hot data to the high performance layer is described as an example, and the migration of cold data from the high performance layer to the normal performance layer may be handled by the same method. When other performance layers are added to the storage system, data can be processed in the same way when migrating between different performance layers. The processor 103 is configured to perform the method of migrating data described below, and the cache 105 is configured to store the needed information.
The flow of the method for migrating data in a storage system according to the embodiment of the present invention is shown in fig. 2 and is completed by a processor in the storage system. The embodiments of the method mainly describe the implementation manner of migrating data from the common performance layer to the high performance layer, and the detailed description is as follows. As described above, a general performance layer in the storage system is referred to as a first storage device, and a high performance layer in the storage system is referred to as a second storage device. In addition, the IO request in the embodiment of the present invention takes an IO request for accessing a storage medium as a monitoring and analysis object.
Step 201: and acquiring the read-write times of the second storage device in unit time.
The processor obtains the number of times of reading and writing of the second storage device in unit time, and the unit time can be determined by a user according to the service type, which is not limited in the embodiment of the invention. When the unit time is 1 second, the obtained number of times of reading and writing per second (Input/Output per second, IOPS) of the second storage device is obtained.
In the embodiment of the invention, the types of each disk in the same performance layer storage device are the same. The embodiment of the invention provides a method for acquiring the number of times of reading and writing of a second storage device in unit time. For example, the number of reads and writes per unit time of the second storage device is (number of reads and writes per unit time of single disk) per RAID conversion factor. Wherein the number of disks is the number of disks in the second storage device. For random write IO, the storage device adopts an RAID mode to calculate check data, so extra IO needs to be generated, the RAID conversion coefficient is related to the proportion of the read-write IO and a write penalty coefficient, the write penalty coefficient refers to a write IO amplification coefficient of a disk and is related to the level of the adopted RAID, and the corresponding relation between the value of the write penalty coefficient and the RAID level can refer to a first table. Specifically, the RAID scaling factor is the read request ratio + write request ratio write penalty factor.
| RAID level | Write penalty coefficient |
| 0 | 1 |
| 1 | 2 |
| 5 | 4 |
| 6 | 6 |
| 10 | 2 |
Table one: write penalty coefficient corresponding to RAID level
The number of times of reading and writing in the unit time of the single disk is related to the load characteristics and the response duration of the second storage device, and data can be collected in advance and obtained through statistics, as shown in table two. The load characteristic of the second storage device herein is a load characteristic of an IO request accessing the second storage device. Therefore, when the read-write times of the second storage device in unit time need to be obtained, the single-disk IOPS in the second storage device can be found directly according to the load characteristics of the IO request for accessing the second storage device and the value of the response time length, and the read-write times of the second storage device in unit time can be obtained through calculation by the method.
Table two: read-write times lookup table in unit time of single disk in second storage device
Optionally, the number of read/write times in a unit time of a single disk may also be obtained according to the load characteristics and response duration statistics of the IO request for accessing the second storage device, and it is not necessary to obtain the value of table two in advance.
Step 203: and acquiring the intensity ratio of the second storage equipment according to the access times of the second storage equipment in unit time.
The strength ratio of the second storage device is the proportion of the number of accesses of the second storage device in unit time to the number of accesses of the storage system in unit time.
In the embodiment of the present invention, the storage system includes storage devices of two performance layers, namely a first storage device and a second storage device, and the strength ratio of the second storage device is the number of accesses in the second storage device unit time/(the number of accesses in the second storage device unit time + the number of accesses in the first storage device unit time), that is, the strength ratio of the second storage device is the proportion of the number of reads and writes in the second storage device unit time to the sum of the number of reads and writes in the first storage device unit time and the number of reads and writes in the second storage device unit time. The number of accesses per unit time of the second storage device has already been described in step 201 and will not be further described here.
The method for acquiring the number of accesses per unit time of the first storage device is the same as the method for acquiring the number of accesses per unit time of the second storage device, and will not be described further. Similarly, the number of times of reading and writing of a single disk in the first storage device in unit time may also be acquired in advance and obtained by statistics, as shown in table three.
Table three: read-write times lookup table in unit time of single disk in first storage device
It should be noted that the contents in table two and table three may be merged into one table for storage, that is, the number of reads and writes per unit time of a single disk in each performance layer in the storage system may be embodied in one table, as shown in table four. When there are other storage devices with different performance layers in the storage system, the number of times of reading and writing in a unit time of a single disc in the other performance layers may be embodied in one table, or may be embodied by different tables, which is not limited in the embodiment of the present invention.
Table four: read-write times lookup table in unit time of single disk in storage system
Step 205: and inquiring a corresponding table of the intensity ratio and the capacity ratio to obtain the capacity ratio corresponding to the intensity ratio.
The corresponding table of the intensity ratio and the capacity ratio is obtained by monitoring, analyzing and calculating the service data in advance. The method comprises the steps of splitting service data in a storage system (comprising a first storage device and a second storage device) into a plurality of data blocks, and obtaining the read-write times of the data blocks in unit time, namely the access strength of the data blocks. The service data can be divided into a plurality of data blocks according to the migration granularity, and also can be divided into a plurality of data blocks according to the specified size. And counting the read-write times of accessing the data blocks in the monitoring time, and then calculating the read-write times IOPS (access strength) of the data blocks in unit time. The data blocks in the storage system are sorted from high to low according to the access strength, and then the data blocks are divided into a plurality of groups according to a certain capacity ratio. For example, as shown in table 3, the data blocks are grouped with the capacity ratio of 2.5%, that is, data 1-dataA form a group with the capacity ratio of 2.5%, that is, data 1-data 2A form a group … … with the capacity ratio of 5%, and so on, as shown in table five.
Table five: statistical table of hot spot data blocks
The number of times of reading and writing of each data block in unit time is calculated, and then the sum of the number of times of reading and writing of each data block in unit time corresponding to each capacity ratio is compared with the sum of the number of times of reading and writing of all data blocks in the storage system in unit time, so as to obtain an intensity ratio corresponding to the capacity ratio, i.e., a correspondence table of the intensity ratio and the capacity ratio, as shown in table six. The data in table six is an exemplary description, and the specific data may be different according to different values of the service. The value of the capacity ratio can also be set as desired.
| Volume ratio | Strength ratio |
| 0.0％ | 0.0％ |
| 2.5％ | 35.0％ |
| 5.0％ | 50.0％ |
| 7.5％ | 60.0％ |
| 10.0％ | 65.0％ |
| 12.5％ | 67.0％ |
| 15.0％ | 69.0％ |
| 20.0％ | 70.0％ |
| 25.0％ | 71.5％ |
| 30.0％ | 72.0％ |
| 100.0％ | 100.0％ |
Table six: intensity ratio to capacity ratio correspondence table for a typical load
As shown in table six, when the capacity ratio of the data blocks in the storage system is 10%, the corresponding intensity ratio is 65%, i.e., the P intensity ratio [ 10% ]is65%, which indicates that the I/O access number of the data blocks occupying 10% of the total data amount is 65% of the I/O access number of all the data blocks in the storage system. Conversely, when the intensity ratio of the data blocks is 70%, the corresponding capacity ratio is 20%, and the P hotspot capacity ratio [ 70% ]is20%, which means that the data blocks with the I/O access number accounting for 70% of the I/O access number of all the data blocks in the storage system account for 20% of all the data blocks in the storage system. From the data law in the table we can see that the intensity ratio for 10% data block is already as high as 65%; and the intensity ratio corresponding to more data blocks has little change, which shows that the thermal data of the scene is concentrated and is suitable for improving the performance by using a grading technology.
In order to improve the accuracy of the data, optionally, the table of the intensity ratio to the capacity ratio may be obtained by taking the data block access condition of the primary service model as a statistical basis.
After the intensity ratio of the second storage device obtained in step 203, the capacity ratio corresponding to the intensity ratio of the second storage device may be obtained by querying the capacity ratio correspondence table. The query rule may adopt a fuzzy matching rule, that is, a table of intensity ratio and capacity ratio is queried, and the capacity ratio corresponding to the intensity ratio is obtained through the fuzzy matching rule. And if the intensity ratio needing to be inquired cannot be accurately matched in the intensity ratio and capacity ratio corresponding table, searching the corresponding capacity ratio according to the obtained intensity ratio of the second storage device, which is higher than the intensity ratio of the first level. And if the data blocks needing to be migrated meet the conditions, which are determined according to the capacity ratio obtained by query, calculating according to the capacity ratio. And if the data block needing to be migrated is determined to not meet the condition according to the searched capacity ratio, searching the corresponding capacity ratio according to the strength ratio which is one level lower than the obtained strength ratio of the second storage device. Specific conditions are described below and will not be described in detail here.
Step 207: and obtaining the number of data blocks which need to be migrated from the first storage device to the second storage device according to the capacity ratio.
And obtaining the number of data blocks corresponding to the capacity ratio according to the capacity ratio obtained in the step 205. Acquiring the read-write times of each data block in the storage system in unit time, and sequentially arranging the acquired read-write times of each data block in unit time from high to low; and selecting a corresponding number of data blocks according to the number of the obtained data blocks corresponding to the capacity ratio from high to low according to the access strength, and determining the number of the data blocks which need to be migrated from the first storage device to the second storage device.
If some of the selected corresponding number of data blocks are already in the second storage device, then the number of data blocks in the first storage device among the number of data blocks is determined, and the number of data blocks that need to be migrated from the first storage device to the second storage device is determined.
In the following, it is described by taking an example that a data block has not been migrated into the second storage device, and if a data block exists in the second storage device, the calculated value is subtracted from the existing value in the second storage device, which is not described one by one. The migration data amount is P capacity ratio [ second storage device IOPS/(second storage device IOPS + first storage device IOPS) ], total data amount. And if the calculated data amount of the data blocks needing to be migrated is less than the total capacity of the second storage device, the number of the data blocks needing to be migrated is the calculated number of the data blocks. And if the calculated data amount of the data blocks needing to be migrated is larger than the total capacity of the second storage device, taking the total capacity of the second storage device as a standard for the number of the data blocks needing to be migrated.
Step 209: and migrating the corresponding data blocks from the first storage device to the second storage device according to the number. According to the number of the data blocks which need to be migrated from the first storage device to the second storage device and are obtained by the calculation in the step 207, migrating the corresponding number of the data blocks from the first storage device to the second storage device according to the access strength from high to low.
The method of the embodiments of the present invention is further illustrated by a specific example.
Through monitoring and analysis of typical traffic data, values for the following parameters were obtained, as shown in table seven:
TABLE VII: service parameter value-taking table
The second storage device IOPS is 3500 × 10/(0.7+0.3 × 4), the first storage device IOPS is 180 × 100/(0.7+0.3 × 4), and the second storage device strength ratio is 66%, the strength ratio and the capacity ratio correspondence table shown in the fuzzy matching rule lookup table six are used, and the higher-level strength ratio 67% of the strength ratio 66% is taken as the reference, the corresponding capacity ratio is determined to be 12.5%, and the migration data amount is 12.5% and 3.75 TB; because the high-performance layer capacity (6TB) >3.75TB, the final migration data volume is 3.75TB, and 3.75TB of data is migrated from the first storage device to the second storage device according to the access strength from high to low. Data accounting for 66% of the access strength of the storage system is migrated to the second storage device, and the second storage device is a high-performance storage device, so that the user request can be responded in time. And the migrated data volume does not exceed the capacity of the second storage device, so that the performance of the second storage device is not influenced, and the overall performance of the storage system is ensured.
An embodiment of the present invention further provides a storage system 3, which can implement the data migration method described above, and a structure of the storage system 3 is shown in fig. 3. The storage system 3 includes a first storage device 321 and a second storage device 323, and the first storage device 321 and the second storage device 323 have different capabilities. In the embodiment of the present invention, the first storage device 321 is composed of a storage medium with common performance, such as a conventional magnetic disk, and can store user data; the second storage device 323 is comprised of a high performance storage medium, such as an SSD disk, that may be used to store hot data. When it is necessary to point out the same features of the first storage device 321 and the second storage device 323, the description will be made using the storage devices. The storage device in the embodiment of the present invention is only an exemplary illustration, and in practical applications, storage media with different performances may also be added. The number of storage media contained in the storage device may also be set as needed, and is not limited in the embodiment of the present invention. In addition, the storage device may be composed of storage media of the same performance, or may be composed of storage media of close performance. The embodiment of the present invention will be described by taking an example of migrating data in the first storage device 321 to the second storage device 323. The function of each component in the storage system 3 is only briefly described in this embodiment, and the detailed features of the related method steps refer to the description in the previous embodiment of the method.
The storage system 3 further comprises a processor 31, which comprises a data acquisition and analysis module 311 and a data migration module 313. The data collection and analysis module 311 is configured to analyze and calculate an IO request for an application to access a storage device, obtain the number of data blocks that need to be migrated from the first storage device 321 to the second storage device 323, and notify the data migration module 313. The data migration module 313 is configured to migrate the obtained number of data blocks from the first storage device 321 to the second storage device 323.
Specifically, the data acquisition and analysis module 311 is configured to obtain the number of times of reading and writing of the second storage device 323 in unit time; acquiring the intensity ratio of the second storage device 323 according to the number of reading and writing times of the second storage device 323 in unit time; inquiring a corresponding table of the intensity ratio and the capacity ratio to obtain the capacity ratio corresponding to the intensity ratio; obtaining the number of data blocks required to be migrated from the first storage device 321 to the second storage device 323 according to the capacity ratio; the obtained number of data blocks to be migrated is sent to the data migration module 313. The detailed processing method is described in detail in the foregoing method, and is not described separately herein.
The data acquisition and analysis module 311 obtains the intensity ratio of the second storage device 323, and obtains the number of data blocks that need to be migrated to the second storage device 323 according to the intensity ratio and capacity ratio correspondence table. Therefore, the amount of the migrated data can be determined according to the performance of the second storage device 323, so that the performance of the second storage device 323 after the data is migrated can be ensured not to be affected, and the performance of the whole storage system is ensured.
The data migration module 313 is configured to migrate the corresponding data block from the first storage device 321 to the second storage device 323 according to the number of the received data blocks that need to be migrated.
The data collecting and analyzing module 311 is further configured to collect and analyze an IO request of the application to the storage system 3 in advance to obtain a corresponding table of the intensity ratio and the capacity ratio. After the intensity ratio of the second storage device 323 is obtained, the intensity ratio-capacity ratio correspondence table can be directly queried, and the number of data blocks that need to be migrated from the first storage device 321 to the second storage device 323 can be quickly obtained.
The data acquisition and analysis module 311 is configured to query the intensity ratio-to-capacity ratio correspondence table to obtain a capacity ratio corresponding to the intensity ratio, specifically: the data acquisition and analysis module 311 is configured to query the intensity ratio and capacity ratio correspondence table, and obtain a capacity ratio corresponding to the intensity ratio through a fuzzy matching rule.
Therefore, when the specific numerical value of the calculated intensity ratio cannot be found in the intensity ratio and capacity ratio corresponding table, the corresponding capacity ratio can be determined according to the fuzzy matching rule by taking the value of the intensity ratio which is higher than the calculated intensity ratio by one level than the value of the intensity ratio in the intensity ratio and capacity ratio corresponding table, and the searching efficiency is improved.
The data acquisition and analysis module 311 is further configured to obtain the number of read-write times of each data block in the storage system 3 in unit time, and arrange the obtained number of read-write times of each data block in unit time from high to low in sequence; determining a data block corresponding to the capacity ratio according to the capacity ratio and the obtained read-write times of each data block in unit time; the number of data blocks corresponding to the capacity ratio in the first storage device 321 is sequentially confirmed.
In order to determine that the hot data with a large number of accesses is migrated from the first storage device 321 to the second storage device 323, the data acquisition and analysis module 311 sequentially ranks the obtained read-write times of each data block in the storage system 3 in the unit time from high to low, so that the data block with a high read-write time in the unit time can be migrated from the first storage device 321 to the second storage device 323 with a higher performance in sequence according to the determined number of the data blocks to be migrated, and an IO request can be responded faster.
The data collecting and analyzing module 311 is further configured to obtain the number of times of reading and writing of the first storage device 321 in unit time. The intensity ratio of the second storage device 323 is a ratio of the number of times of reading and writing of the second storage device 323 in unit time to the sum of the number of times of reading and writing of the first storage device 321 in unit time and the number of times of reading and writing of the second storage device 323 in unit time.
The number of reads and writes in the first storage device 321 per unit time is a product of the number of reads and writes in the first storage device 321 per unit time of a single disk and the number of disks in the first storage device 321, and then divided by a first scaling factor, where the first scaling factor is related to a ratio of read requests and write requests of the first storage device 321 and a RAID level of the first storage device 321;
the number of reads and writes per unit time of the second storage device 323 is the product of the number of reads and writes per unit time of a single disk in the second storage device 323 and the number of disks in the second storage device 323, divided by a second scaling factor, where the second scaling factor is related to the ratio of read requests and write requests of the second storage device 323 and the RAID level of the second storage device 323.
By setting and acquiring the specific values of the parameters, the number of data blocks to be migrated from the first storage device 321 to the second storage device 323 can be accurately obtained, the performance of the storage device 323 can be ensured while hot data is migrated to the high-performance second storage device 323 as much as possible, and the efficiency and performance of the storage system can be improved.
The possible values of the parameters and the calculation method are described in detail in the foregoing method embodiments, and are not described in detail here.
Embodiments of the present application further provide a computer storage medium for storing computer software instructions for the storage system, which includes a program designed to execute the method embodiments. The method of migrating data between storage devices may be implemented by executing stored programs.
The embodiment of the present application further provides a computer program, which includes instructions, when the computer program is executed by a computer, the computer may execute the procedures of the above method embodiments.
While the present application has been described in connection with various embodiments, other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed application, from a review of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the word "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, apparatus (device), or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein. A computer program stored/distributed on a suitable medium supplied together with or as part of other hardware, may also take other distributed forms, such as via the Internet or other wired or wireless telecommunication systems.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (devices) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; such modifications and substitutions do not depart from the scope of the claims.
Claims (12)
1. A method of migrating data in a storage system, the storage system including a first storage device and a second storage device, the method comprising:
acquiring the read-write times of the second storage device in unit time;
acquiring the strength ratio of the second storage device according to the reading and writing times of the second storage device in unit time;
inquiring a corresponding table of the intensity ratio and the capacity ratio to obtain the capacity ratio corresponding to the intensity ratio;
obtaining the number of data blocks which need to be migrated from the first storage device to the second storage device according to the capacity ratio;
and migrating the corresponding data blocks from the first storage device to the second storage device according to the number.
2. The method of claim 1, wherein the intensity ratio to capacity ratio correspondence table is pre-monitored and analyzed.
3. The method according to claim 1 or 2, wherein the step of querying the intensity ratio-to-capacity ratio correspondence table to obtain the capacity ratio corresponding to the intensity ratio specifically comprises:
and inquiring a corresponding table of the intensity ratio and the capacity ratio, and acquiring the capacity ratio corresponding to the intensity ratio through a fuzzy matching rule.
4. The method according to claim 1 or 2, characterized in that the method further comprises:
acquiring the read-write times of each data block in the storage system in unit time, and sequentially arranging the acquired read-write times of each data block in unit time from high to low;
obtaining the number of data blocks that need to be migrated from the first storage device to the second storage device according to the capacity ratio specifically includes:
determining a data block corresponding to the capacity ratio according to the capacity ratio and the obtained read-write times of each data block in unit time;
confirming the number of data blocks corresponding to the capacity ratio in the first storage device.
5. The method according to claim 1 or 2, characterized in that the method further comprises:
acquiring the read-write times of the first storage device in unit time;
the strength ratio of the second storage device is the proportion of the number of times of reading and writing of the second storage device in unit time to the sum of the number of times of reading and writing of the first storage device in unit time and the number of times of reading and writing of the second storage device in unit time.
6. The method of claim 5, wherein the number of reads and writes per unit time of the first storage device is a product of the number of reads and writes per unit time of a single disk in the first storage device and the number of disks in the first storage device divided by a first scaling factor, the first scaling factor being related to a ratio of read requests to write requests of the first storage device and a RAID level of the first storage device;
the number of read/write times in unit time of the second storage device is the product of the number of read/write times in unit time of a single disk in the second storage device and the number of disks in the second storage device, and then is divided by a second conversion coefficient, wherein the second conversion coefficient is related to the ratio of read requests and write requests of the second storage device and the RAID level of the second storage device.
7. The method of claim 6, further comprising: the read-write times of the single disk in unit time in the first storage device are related to the load characteristics and the response duration of the first storage device;
and the read-write times in the unit time of the single disk in the second storage device are related to the load characteristics and the response time length of the second storage device.
8. A storage system implementing data migration, the storage system comprising a first storage device, a second storage device, and a processor configured to perform the method of claims 1-7.
9. A storage system for realizing data migration is characterized by comprising a first storage device, a second storage device and a processor, wherein the processor comprises a data acquisition and analysis module and a data migration module;
the data acquisition and analysis module is used for acquiring the read-write times of the second storage device in unit time; acquiring the strength ratio of the second storage device according to the reading and writing times of the second storage device in unit time; inquiring a corresponding table of the intensity ratio and the capacity ratio to obtain the capacity ratio corresponding to the intensity ratio; obtaining the number of data blocks which need to be migrated from the first storage device to the second storage device according to the capacity ratio; sending the obtained number of the data blocks to be migrated to a data migration module;
the data migration module is used for migrating the obtained number of data blocks from the first storage device to the second storage device.
10. The storage system of claim 9, wherein the data collection and analysis module is further configured to:
and (4) acquiring and analyzing IO requests of the application to the storage system in advance to obtain a corresponding table of the intensity ratio and the capacity ratio.
11. The storage system according to claim 9 or 10, wherein the data acquisition and analysis module is configured to query the intensity ratio-to-capacity ratio correspondence table to obtain the capacity ratio corresponding to the intensity ratio by: the data acquisition and analysis module is used for inquiring the corresponding table of the intensity ratio and the capacity ratio and acquiring the capacity ratio corresponding to the intensity ratio through a fuzzy matching rule.
12. The storage system according to claim 9 or 10, wherein the data acquisition and analysis module is further configured to acquire the number of times of reading and writing in unit time of each data block in the storage system, and arrange the acquired number of times of reading and writing in unit time of each data block in order from high to low;
the data acquisition and analysis module obtains the number of data blocks which need to be migrated from the first storage device to the second storage device according to the capacity ratio, and specifically comprises: and the data acquisition and analysis module obtains the number of data blocks which need to be migrated from the first storage device to the second storage device in sequence according to the capacity ratio.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611199849.1A CN107122126B (en) | 2016-12-22 | 2016-12-22 | Data migration method, device and system |
| PCT/CN2017/096784 WO2018113317A1 (en) | 2016-12-22 | 2017-08-10 | Data migration method, apparatus, and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611199849.1A CN107122126B (en) | 2016-12-22 | 2016-12-22 | Data migration method, device and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107122126A CN107122126A (en) | 2017-09-01 |
| CN107122126B true CN107122126B (en) | 2020-09-08 |
Family
ID=59717905
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611199849.1A Active CN107122126B (en) | 2016-12-22 | 2016-12-22 | Data migration method, device and system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN107122126B (en) |
| WO (1) | WO2018113317A1 (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107894873A (en) * | 2017-11-07 | 2018-04-10 | 长沙曙通信息科技有限公司 | A kind of memory virtualization system data processing implementation method |
| CN110413209B (en) * | 2018-04-28 | 2023-05-30 | 伊姆西Ip控股有限责任公司 | Method and apparatus for managing storage system |
| CN110018797B (en) * | 2019-04-11 | 2020-03-06 | 苏州浪潮智能科技有限公司 | Data migration method, device and equipment and readable storage medium |
| CN110825908B (en) * | 2019-11-04 | 2023-04-25 | 安超云软件有限公司 | Object migration method and device, electronic equipment and storage medium |
| CN111124281B (en) * | 2019-11-29 | 2023-11-03 | 浪潮(北京)电子信息产业有限公司 | Data storage method, device, equipment and storage medium of full flash memory system |
| CN111078160B (en) * | 2020-03-23 | 2020-06-23 | 上海飞旗网络技术股份有限公司 | Data migration and storage method and device based on call proportion and response duration |
| CN112015347A (en) * | 2020-08-28 | 2020-12-01 | 北京浪潮数据技术有限公司 | Data migration method and device and related equipment |
| CN113741810B (en) * | 2021-07-30 | 2023-08-11 | 苏州浪潮智能科技有限公司 | Data migration method and device |
| CN114327253B (en) * | 2021-10-18 | 2024-05-28 | 杭州逗酷软件科技有限公司 | Data migration method, device, electronic equipment and storage medium |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101369451A (en) * | 2007-08-14 | 2009-02-18 | 三星电子株式会社 | Solid state memory (ssm), computer system including an ssm, and method of operating an ssm |
| CN102033717A (en) * | 2010-12-07 | 2011-04-27 | 清华大学 | Data storage method and system based on RAID (Redundant Array of Inexpensive Disk) |
| CN102317901A (en) * | 2011-07-25 | 2012-01-11 | 华为技术有限公司 | Methods for object adjustment and devices for remove control, node and storage system |
| CN102541460A (en) * | 2010-12-20 | 2012-07-04 | 中国移动通信集团公司 | Multiple disc management method and equipment |
| CN103091707A (en) * | 2012-12-31 | 2013-05-08 | 中国石油集团川庆钻探工程有限公司地球物理勘探公司 | Device and method of seismic data hierarchical storage |
| CN103106044A (en) * | 2012-12-13 | 2013-05-15 | 深圳先进技术研究院 | Classification storage energy-saving method |
| CN103106152A (en) * | 2012-12-13 | 2013-05-15 | 深圳先进技术研究院 | Data scheduling method based on gradation storage medium |
| CN103616944A (en) * | 2013-10-18 | 2014-03-05 | 杭州电子科技大学 | Consumption reduction method in cloud storage system based on pre-judging green data classification strategy |
| CN103688248A (en) * | 2013-06-29 | 2014-03-26 | 华为技术有限公司 | Management method for storage arrays, device and controller |
| CN104102454A (en) * | 2013-04-07 | 2014-10-15 | 杭州信核数据科技有限公司 | Method for automatically realizing hierarchical storage and system for managing hierarchical storage |
| CN104268099A (en) * | 2014-08-29 | 2015-01-07 | 浪潮(北京)电子信息产业有限公司 | Management data reading and writing method and device |
| CN104536904A (en) * | 2014-12-29 | 2015-04-22 | 杭州华为数字技术有限公司 | Data management method, equipment and system |
| WO2015145532A1 (en) * | 2014-03-24 | 2015-10-01 | 株式会社日立製作所 | Storage system and data processing method |
| CN105653524A (en) * | 2014-11-10 | 2016-06-08 | 阿里巴巴集团控股有限公司 | Data storage method, device and system |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8700841B2 (en) * | 2010-04-19 | 2014-04-15 | International Business Machines Corporation | Sub-LUN input/output profiling for SSD devices |
| CN104572684B (en) * | 2013-10-17 | 2018-12-28 | 深圳市腾讯计算机系统有限公司 | Capacity adjustment method, device and the data-storage system of data-storage system |
| CN104598495A (en) * | 2013-10-31 | 2015-05-06 | 南京中兴新软件有限责任公司 | Hierarchical storage method and system based on distributed file system |
-
2016
- 2016-12-22 CN CN201611199849.1A patent/CN107122126B/en active Active
-
2017
- 2017-08-10 WO PCT/CN2017/096784 patent/WO2018113317A1/en active Application Filing
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101369451A (en) * | 2007-08-14 | 2009-02-18 | 三星电子株式会社 | Solid state memory (ssm), computer system including an ssm, and method of operating an ssm |
| CN102033717A (en) * | 2010-12-07 | 2011-04-27 | 清华大学 | Data storage method and system based on RAID (Redundant Array of Inexpensive Disk) |
| CN102541460A (en) * | 2010-12-20 | 2012-07-04 | 中国移动通信集团公司 | Multiple disc management method and equipment |
| CN102317901A (en) * | 2011-07-25 | 2012-01-11 | 华为技术有限公司 | Methods for object adjustment and devices for remove control, node and storage system |
| CN103106152A (en) * | 2012-12-13 | 2013-05-15 | 深圳先进技术研究院 | Data scheduling method based on gradation storage medium |
| CN103106044A (en) * | 2012-12-13 | 2013-05-15 | 深圳先进技术研究院 | Classification storage energy-saving method |
| CN103091707A (en) * | 2012-12-31 | 2013-05-08 | 中国石油集团川庆钻探工程有限公司地球物理勘探公司 | Device and method of seismic data hierarchical storage |
| CN104102454A (en) * | 2013-04-07 | 2014-10-15 | 杭州信核数据科技有限公司 | Method for automatically realizing hierarchical storage and system for managing hierarchical storage |
| CN103688248A (en) * | 2013-06-29 | 2014-03-26 | 华为技术有限公司 | Management method for storage arrays, device and controller |
| CN103616944A (en) * | 2013-10-18 | 2014-03-05 | 杭州电子科技大学 | Consumption reduction method in cloud storage system based on pre-judging green data classification strategy |
| WO2015145532A1 (en) * | 2014-03-24 | 2015-10-01 | 株式会社日立製作所 | Storage system and data processing method |
| CN104268099A (en) * | 2014-08-29 | 2015-01-07 | 浪潮(北京)电子信息产业有限公司 | Management data reading and writing method and device |
| CN105653524A (en) * | 2014-11-10 | 2016-06-08 | 阿里巴巴集团控股有限公司 | Data storage method, device and system |
| CN104536904A (en) * | 2014-12-29 | 2015-04-22 | 杭州华为数字技术有限公司 | Data management method, equipment and system |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2018113317A1 (en) | 2018-06-28 |
| CN107122126A (en) | 2017-09-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107122126B (en) | Data migration method, device and system | |
| CN107807796B (en) | Data layering method, terminal and system based on super-fusion storage system | |
| US10031671B2 (en) | Method, apparatus, and system for calculating identification threshold to distinguish cold data and hot data | |
| US10229129B2 (en) | Method and apparatus for managing time series database | |
| CN106469018B (en) | Load monitoring method and device for distributed storage system | |
| CN104978324B (en) | Data processing method and device | |
| CN112100293A (en) | Data processing method, data access method, data processing device, data access device and computer equipment | |
| CN107291539B (en) | Cluster program scheduler method based on resource significance level | |
| CN109783443A (en) | The cold and hot judgment method of mass data in a kind of distributed memory system | |
| CN113177050A (en) | Data balancing method, device, query system and storage medium | |
| JP2019204475A (en) | Method of caching plural files of 2 mb or less based upon hadoop | |
| CN110737717A (en) | database migration method and device | |
| CN107515807B (en) | Method and device for storing monitoring data | |
| CN110688360A (en) | Distributed file system storage management method, device, equipment and storage medium | |
| CN111125088B (en) | Multi-level data processing method and device | |
| CN104915376B (en) | A kind of archival compression method of file in cloud storage | |
| CN115952005B (en) | Metadata load balancing method, device, equipment and readable storage medium | |
| CN102546235A (en) | Performance diagnosis method and system of web-oriented application under cloud computing environment | |
| CN108664322A (en) | Data processing method and system | |
| CN108536759B (en) | Sample playback data access method and device | |
| CN109408302B (en) | Fault detection method and device and electronic equipment | |
| CN110334073A (en) | A kind of metadata forecasting method, device, terminal, server and storage medium | |
| CN115576924A (en) | Data migration method | |
| CN114416731A (en) | Data storage method, data reading method, data storage device, electronic device and medium | |
| CN109828718B (en) | Disk storage load balancing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |