CN113867645A

CN113867645A - Data migration and data read-write method and device, computer equipment and storage medium

Info

Publication number: CN113867645A
Application number: CN202111162203.7A
Authority: CN
Inventors: 张孙旻
Original assignee: Suzhou Inspur Intelligent Technology Co Ltd
Current assignee: Suzhou Inspur Intelligent Technology Co Ltd
Priority date: 2021-09-30
Filing date: 2021-09-30
Publication date: 2021-12-31
Anticipated expiration: 2041-09-30
Also published as: CN113867645B

Abstract

The invention discloses a data migration and data read-write method and device, computer equipment and a storage medium, and relates to the technical field of data storage. The method comprises the following steps: acquiring the heat value of each data block on a cold storage node and a hot storage node; according to each heat value, migrating a first data block with the heat value larger than a first preset heat threshold value on the cold storage node to the hot storage node; and detecting the use space of the hot storage node, and if the use space of the hot storage node reaches a preset space threshold, migrating the second data block with the heat value meeting the preset condition in the hot storage node to the cold storage node. By adopting the method, the migration of the data block from the cold storage node to the hot storage node and the migration from the hot storage node to the cold storage node can be completed, so that the data processing speed of the computer equipment is ensured, and the running performance of the computer equipment is ensured.

Description

Data migration and data read-write method and device, computer equipment and storage medium

Technical Field

The invention relates to the technical field of data storage, in particular to a data migration and data read-write method and device, computer equipment and a storage medium.

Background

In the big data era, the data volume is increasing day by day, the most used of the current traditional storage system is the HDD mechanical hard disk, although the mechanical hard disk is cheap and has long service life, the problem of slow read-write processing speed caused by long addressing time also exists. With the continuous development of cpu computing power, the read-write capability of a storage medium becomes a bottleneck of a system, and on the other hand, an SSD solid state disk has the advantages of low power consumption, high read-write speed, and the like, and becomes a substitute for a storage medium at the back end of a storage system.

In the prior art, storage systems that typically use a small number of SSD disks in conjunction with a large number of HDD disks are expected to achieve SSD performance at a cost approaching that of HDDs.

The significance of the hierarchical storage system is that hot data considered by the system can be placed in an SSD disk, and cold data is in an HDD disk, the processing mode improves the processing speed of reading and writing, data migration is needed in the SSD and the HDD to achieve the purpose, and how to perform the data migration becomes an important problem.

Disclosure of Invention

In view of this, embodiments of the present invention provide a data migration method, a data read-write method, a data migration apparatus, a computer device, and a storage medium, so as to solve the problem of data migration in an SSD and an HDD.

According to a first aspect, an embodiment of the present invention provides a data migration method, including: acquiring the heat value of each data block on a cold storage node and a hot storage node; according to each heat value, migrating a first data block with the heat value larger than a first preset heat threshold value on the cold storage node to the hot storage node; and detecting the use space of the hot storage node, and if the use space of the hot storage node reaches a preset space threshold, migrating the second data block with the heat value meeting the preset condition in the hot storage node to the cold storage node.

In this embodiment, the heat values of the data blocks on the cold storage node and the hot storage node are obtained, and the first data block with the heat value larger than the first preset heat threshold value on the cold storage node is migrated to the hot storage node according to the heat value of each data block. Therefore, the first data block is migrated from the cold storage node to the hot storage node, and the reading and writing speed of the first data block is accelerated. In addition, the use space of the thermal storage node can be detected, and if the use space of the thermal storage node reaches a preset space threshold, the speed of reading and writing data by the thermal storage node is ensured. The second data block with the heat value meeting the preset condition in the hot storage node can be migrated to the cold storage node. Therefore, a use space can be left for some thermal data from the thermal storage nodes, and the speed of reading and writing the data on the thermal storage nodes is further accelerated. Therefore, the method completes the migration of the first data block from the cold storage node to the hot storage node and completes the migration of the second data block from the hot storage node to the cold storage node under the condition of ensuring the data consistency, thereby ensuring the speed of processing the data and ensuring the running performance of the computer equipment.

With reference to the first aspect, in a first implementation manner of the first aspect, migrating a second data block, whose hot value satisfies a preset condition, in a hot storage node to a cold storage node includes: migrating a preset number of second data blocks with ranked hot values in the hot storage nodes to the cold storage nodes; or migrating the second data block with the heat value smaller than the second preset heat threshold value in the hot storage node to the cold storage node.

In this embodiment, a preset number of second data blocks with ranked hot values in the hot storage nodes are migrated to the cold storage nodes; or migrating the second data block with the heat value smaller than the second preset heat threshold value in the hot storage node to the cold storage node. Therefore, the second data block transferred from the hot storage node to the cold storage node can be guaranteed to be cold data, the accuracy of the transferred second data block is guaranteed, and hot data in the hot storage node cannot be transferred to the cold storage node. The speed of processing the data on the thermal storage nodes is further ensured.

With reference to the first aspect, in a second implementation manner of the first aspect, migrating a first data block with a heat value greater than a first preset heat threshold on a cold storage node to a hot storage node according to each heat value includes: putting each first data block into a first linked list, wherein the first linked list is a migration queue corresponding to the migration of the first data block from the cold storage node to the hot storage node; and migrating each first data block in the first linked list to the hot storage node.

In this embodiment, each first data block may be placed in the first linked list, and each first data block in the first linked list may be migrated to the hot storage node. The first linked list is a migration queue corresponding to the migration of the first data block from the cold storage node to the hot storage node. Therefore, the first data blocks can be guaranteed to be migrated to the hot storage nodes according to the preset sequence, and the situation that the data volume of the hot storage nodes is sharply increased and the operation of the hot storage nodes is influenced due to the fact that all the first data blocks are migrated to the hot storage nodes from the cold storage nodes at the same time is avoided.

With reference to the second embodiment of the first aspect, in a third embodiment of the first aspect, migrating each first data block in the first linked list to a hot storage node includes: for each first data block in the first linked list, splitting the first data block into a plurality of first data slices, wherein each first data slice has different identification information according to different positions of the first data block; the identification information comprises a source address of the first data slice at the cold storage node and a target address of the first data slice at the hot storage node; and migrating the first data pieces to the thermal storage nodes according to the preset sequence according to the identification information of the first data pieces.

In this embodiment, for each first data block in the first linked list, the first data block is split into a plurality of first data pieces, so that each first data block can be conveniently migrated, and the migration speed is increased. In addition, in order to prevent the first data pieces from being confused in the migration process, different identification information can be marked on the first data pieces according to different positions of the first data pieces in the first data block, and the first data pieces can be migrated to the hot storage node according to the identification information of the first data pieces and according to a preset sequence. Therefore, the situation that each first data slice has no migration error in the migration process can be ensured, and the migration accuracy of each first data block is ensured.

With reference to the third embodiment of the first aspect, in a fourth embodiment of the first aspect, the method further includes: and creating a save file based on the identification information of each first data slice and the migration process of each first data slice, and periodically updating the save file according to the migration process of each first data slice.

In this embodiment, the save file is created based on the identification information of each first data slice and the migration process of each first data slice, so that after the failure data migration process of the current storage system is interrupted, the migration of the first data block can be restarted according to the save file and the content recorded in the save file before the interruption of the data migration process after the failure is repaired. In addition, the saved files are periodically updated according to the migration process of each first data slice, and the accuracy and traceability of the saved files are guaranteed.

With reference to the second embodiment of the first aspect, in a fifth embodiment of the first aspect, before migrating each first data block in the first linked list to a thermal storage node, the method further includes: detecting whether each first data block in the first linked list is migrated within a preset time period; and if the first data block is migrated, removing the migrated first data block from the first linked list.

In this embodiment, whether each first data block in the first linked list is migrated within a preset time period is detected, and the migrated first data block is removed from the first linked list when the first data block is migrated. Therefore, misleading caused by severe change of the hot value of the first data block due to access randomness in a short time to the migration process can be prevented, and reduction of the service life of the area due to multiple times of erasing of the hot storage nodes is also protected.

In a second aspect, an embodiment of the present invention provides a data reading and writing method, where the method further includes: receiving a read-write instruction aiming at a target first data slice; carrying out data migration on a target first data slice by adopting the method of any one of the first aspect; detecting the current migration condition of a target first data slice; if the target first data piece is migrated to the thermal storage node, reading and writing data in the target first data piece according to the target address of the target first data piece in the thermal storage node; if the target first data slice is not migrated to the hot storage node, reading and writing data in the target first data slice according to the source address of the target first data slice at the cold storage node; and if the target first data slice is being migrated, detecting the content of the read-write instruction, and performing corresponding operation on the target first data slice according to the content of the read-write instruction.

In the embodiment, a read-write instruction for a target first data slice is received; detecting the current migration condition of a target first data slice; if the target first data piece is migrated to the thermal storage node, reading and writing data in the target first data piece according to the target address of the target first data piece in the thermal storage node; if the target first data slice is not migrated to the hot storage node, reading and writing data in the target first data slice according to the source address of the target first data slice at the cold storage node; and if the target first data slice is being migrated, detecting the content of the read-write instruction, and performing corresponding operation on the target first data slice according to the content of the read-write instruction. Therefore, the data migration can be guaranteed to be completed, and normal read-write processing of the data can be guaranteed not to be influenced. The performance of data processing is further ensured.

With reference to the second aspect, in the first embodiment of the second aspect, if the target first data slice is being migrated, detecting the content of the read/write command, and performing an operation on the target first data slice according to the content of the read/write command includes: if the content of the read-write instruction is to read the target first data slice, reading the data in the target first data slice according to the source address of the target first data slice at the cold storage node; and if the content of the read-write instruction is to write the target first data slice, recording the read-write instruction, and after the target first data slice is migrated, writing data into the target first data slice according to the target address of the target first data slice in the thermal storage node.

In the embodiment of the application, if the content of the read-write instruction is to read the target first data slice, reading the data in the target first data slice according to the source address of the target first data slice at the cold storage node; and if the content of the read-write instruction is to write the target first data slice, recording the read-write instruction, and after the target first data slice is migrated, writing data into the target first data slice according to the target address of the target first data slice in the thermal storage node. Therefore, the read-write task aiming at the target first data piece can be finished under the condition that the target first data piece is being migrated. Therefore, the data migration can be guaranteed to be completed, and normal read-write processing of the data can be guaranteed not to be influenced. The performance of data processing is further ensured.

According to a third aspect, an embodiment of the present invention provides a data migration apparatus, including:

the acquisition module is used for acquiring the heat values of all data blocks on the cold storage node and the hot storage node;

the migration module is used for migrating the first data blocks with the heat values larger than a first preset heat threshold value on the cold storage nodes to the hot storage nodes according to the heat values;

and the detection module is used for detecting the use space of the heat storage node, and if the use space of the heat storage node reaches a preset space threshold, migrating the second data block with the heat value meeting the preset condition in the heat storage node to the cold storage node.

With reference to the third aspect, in a first implementation manner of the third aspect, the detection module is specifically configured to migrate a preset number of second data blocks with ranked hot values in the hot storage nodes to the cold storage nodes; or migrating the second data block with the heat value smaller than the second preset heat threshold value in the hot storage node to the cold storage node.

With reference to the third aspect, in a second implementation manner of the third aspect, the migration module includes:

the storage unit is used for storing the first data blocks into a first linked list, wherein the first linked list is a migration queue corresponding to the first data blocks from the cold storage nodes to the hot storage nodes;

and the migration unit is used for migrating each first data block in the first linked list to the hot storage node.

With reference to the second embodiment of the third aspect, in a third implementation manner of the third aspect, the migration unit is specifically configured to, for each first data block in the first linked list, split the first data block into a plurality of first data slices, where each first data slice has different identification information according to a different position of the first data block; the identification information comprises a source address of the first data slice at the cold storage node and a target address of the first data slice at the hot storage node; and migrating the first data pieces to the thermal storage nodes according to the preset sequence according to the identification information of the first data pieces.

With reference to the third embodiment of the third aspect, in a fourth implementation of the third aspect, the migration unit is further configured to create a save file based on the identification information of each first data slice and the migration process of each first data slice, and periodically update the save file according to the migration process of each first data slice.

With reference to the second embodiment of the third aspect, in a fifth implementation of the third aspect, the migration module further includes:

the detection unit is used for detecting whether each first data block in the first linked list is migrated within a preset time period;

and the removing unit is used for removing the migrated first data block from the first linked list under the condition that the first data block is migrated.

In a fourth aspect, an embodiment of the present invention provides a data reading and writing apparatus, where the data reading and writing apparatus includes:

the receiving module is used for receiving a read-write instruction aiming at a target first data slice; carrying out data migration on a target first data slice by adopting the method of any one of the first aspect;

the detection model is used for detecting the current migration condition of the target first data slice;

the first read-write module is used for reading and writing data in the target first data piece according to the target address of the target first data piece in the thermal storage node if the target first data piece is migrated to the thermal storage node;

the second read-write module is used for reading and writing the data in the target first data sheet according to the source address of the target first data sheet in the cold storage node if the target first data sheet is not migrated to the hot storage node;

and the operation module detects the content of the read-write instruction if the target first data sheet is being migrated, and performs corresponding operation on the target first data sheet according to the content of the read-write instruction.

With reference to the fourth aspect, in a first implementation manner of the fourth aspect, the operation module is specifically configured to, if the content of the read-write instruction is to read the target first data slice, read data in the target first data slice according to a source address of the target first data slice at the cold storage node; and if the content of the read-write instruction is to write the target first data slice, recording the read-write instruction, and after the target first data slice is migrated, writing data into the target first data slice according to the target address of the target first data slice in the thermal storage node.

According to a fifth aspect, an embodiment of the present invention provides an electronic device/mobile terminal/server, including a memory and a processor, where the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions, so as to execute the data migration method in the first aspect or any one of the implementation manners of the first aspect and the data reading and writing method in any one of the implementation manners of the second aspect or the second aspect.

According to a sixth aspect, an embodiment of the present invention provides a computer-readable storage medium, where the computer-readable storage medium stores computer instructions for causing a computer to execute the data migration method in the first aspect or any one of the implementation manners of the first aspect and the data reading and writing method in the second aspect or any one of the implementation manners of the second aspect.

According to a seventh aspect, an embodiment of the present invention provides a computer program product, where the computer program product includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions, which, when executed by a computer, cause the computer to execute the data migration method in the first aspect or any one of the implementation manners of the first aspect and the data read-write method in any one of the implementation manners of the second aspect or the second aspect.

Drawings

The features and advantages of the present invention will be more clearly understood by reference to the accompanying drawings, which are illustrative and not to be construed as limiting the invention in any way, and in which:

FIG. 1 is a flow diagram that illustrates the steps of a data migration method in one embodiment;

FIG. 2 is a flow chart illustrating the steps of a data migration method in another embodiment;

FIG. 3 is a flow chart illustrating the steps of a data migration method in another embodiment;

FIG. 4 is a flow chart illustrating the steps of a data migration method in another embodiment;

FIG. 5 is a flow chart diagram illustrating the steps of a data migration and data read/write method in another embodiment;

FIG. 6 is a block diagram showing the configuration of a data migration apparatus in one embodiment;

FIG. 7 is a block diagram showing the configuration of a data migration apparatus in one embodiment;

FIG. 8 is a block diagram showing the configuration of a data migration apparatus in one embodiment;

FIG. 9 is a block diagram showing the structure of a data read/write apparatus in one embodiment;

FIG. 10 is a diagram illustrating an internal structure of one embodiment of a computer device in the form of a server;

FIG. 11 illustrates an internal block diagram of a computer device when the computer device is a terminal, according to an embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that in the method for data migration provided in the embodiment of the present application, an execution main body of the method may be a data migration device, and the data migration device may be implemented in a software, hardware, or a combination of software and hardware to become part or all of a computer device, where the computer device may be a server or a terminal, where the server in the embodiment of the present application may be one server or a server cluster composed of multiple servers, and the terminal in the embodiment of the present application may be another intelligent hardware device such as a smart phone, a personal computer, a tablet computer, a wearable device, and an intelligent robot. In the following method embodiments, the execution subject is a computer device as an example.

In an embodiment of the present application, as shown in fig. 1, a data migration method is provided, which is described by taking an example that the method is applied to a computer device, and includes the following steps:

step 101, acquiring the heat value of each data block on the cold storage node and the hot storage node.

Wherein the heat value is related to the read-write times of each data block. The cold storage nodes may be mechanical hard disks for storing cold data that is not commonly used by some computer devices, and the hot storage nodes may be solid state hard disks for storing hot data that is currently being used or frequently used by computer devices.

Specifically, the computer device may record the read-write frequency corresponding to each data block when the data blocks on the hot storage node and the cold storage node are read and written, and then the computer device may calculate the heat value corresponding to each data block according to the read-write frequency corresponding to each data block.

Specifically, the computer device may record the historical read-write times of each data block in at least one historical preset time period closest to the current time, and record the read-write times in the current preset time period, and then the computer device may perform weighted summation on the historical read-write times and the read-write times in the current preset time period to obtain the heat value of each data block in the current preset time period. The weight of the read-write times in the preset time period can be larger than the weight of the historical read-write times.

For example, assuming that the preset time period is one week, the computer device may calculate the number of times of reading and writing each data block in the previous week, then calculate the number of times of reading and writing each data block in the current week, then multiply the number of times of reading and writing obtained by the two calculations by corresponding weights, and add up the number of times of reading and writing to obtain the heat value corresponding to each data block in the current week. The weight corresponding to the reading and writing times in the week can be larger than the weight corresponding to the reading and writing times in the last week.

And step 102, migrating the first data block with the heat value larger than the first preset heat threshold value on the cold storage node to the hot storage node according to each heat value.

The number of the first data blocks may be one or multiple, and the number of the first data blocks is defined in the embodiment of the present application.

Specifically, after the computer device calculates the heat values corresponding to the data blocks on the cold storage node and the hot storage node, the computer device may compare the heat values corresponding to the data blocks on the cold storage node with a first preset heat threshold. When a first data block with a heat value greater than a first preset heat threshold exists on the thermal storage node, the computer device may migrate the first data block with a heat value greater than the first preset heat threshold to the thermal storage node.

In an alternative embodiment, the computer device may copy a first data block on the cold storage node having a heat value greater than a first preset heat threshold to the hot storage node, and then overwrite the first data block on the cold storage node with another data block.

In another optional embodiment, the computer device may further transmit a first data block with a heat value on the cold storage node greater than a first preset heat threshold to the hot storage node.

And 103, detecting the use space of the hot storage node, and if the use space of the hot storage node reaches a preset space threshold, migrating the second data block with the heat value meeting the preset condition in the hot storage node to the cold storage node.

The number of the second data blocks may be one or multiple, and the number of the second data blocks is defined in the embodiment of the present application.

Specifically, the computer device can also detect the usage space of the heat storage nodes in real time and can also periodically detect the usage space of the heat storage nodes. And the computer equipment compares the detected use space of the heat storage node with a preset space threshold value, and migrates a second data block with a heat value meeting a preset condition in the heat storage node to the cold storage node under the condition that the use space of the heat storage node reaches the preset space threshold value.

In an alternative embodiment, the computer device may copy a second data block on the hot storage node, whose hot value satisfies a preset condition, to the cold storage node, and then overwrite the second data block on the hot storage node with another data block.

In another alternative embodiment, the computer device may further transmit a second data block on the hot storage node, the second data block having a hot value satisfying a preset condition, to the cold storage node.

In this embodiment, the computer device obtains the heat values of the data blocks on the cold storage node and the hot storage node, and migrates the first data block, whose heat value is greater than the first preset heat threshold value, on the cold storage node to the hot storage node according to the heat values of the data blocks. Therefore, the first data block is migrated from the cold storage node to the hot storage node, and the reading and writing speed of the computer equipment on the first data block is increased. In addition, the computer device can also detect the use space of the thermal storage node, and if the use space of the thermal storage node reaches a preset space threshold, the speed of reading and writing data by the thermal storage node is ensured. The computer device may migrate a second data block, whose hot value satisfies a preset condition in the hot storage node, to the cold storage node. Therefore, a use space can be left for some thermal data from the thermal storage nodes, and the speed of reading and writing the data on the thermal storage nodes by the computer equipment is further accelerated. Therefore, in the above method, under the condition of ensuring the data consistency, the migration of the first data block from the cold storage node to the hot storage node is completed, and the migration of the second data block from the hot storage node to the cold storage node is also completed, so that the speed of processing the data by the computer device is ensured, and the operation performance of the computer device is ensured.

In an embodiment of the present application, the "migrating the second data block with the heat value satisfying the preset condition to the cold storage node in the hot storage node" in step 103 may include the following steps:

in one case, the second data blocks with the preset number and ranked according to the heat values in the hot storage nodes are migrated to the cold storage nodes.

Specifically, the computer device may sort the heat values of the data blocks in the thermal storage node from large to small, and then select a preset number of second data blocks with ranked heat values from the data blocks in the thermal storage node according to the sorting result. The computer device migrates a second data block selected from the data blocks in the hot storage node to the cold storage node. The preset number may be 3, 5, or an integer value thereof, and the preset number is not specifically limited in the embodiment of the present application.

In another case, the second data block with the heat value smaller than the second preset heat threshold value in the hot storage node is migrated to the cold storage node.

Specifically, the computer device may further compare the heat value of each data block in the thermal storage node with a second preset heat threshold, and select a second data block with a heat value smaller than the second preset heat threshold from each data block in the thermal storage node. The computer device then migrates a second data block selected from the data blocks in the hot storage node to the cold storage node. And the second preset heat threshold is smaller than the first preset heat threshold.

In this embodiment, the computer device migrates a preset number of second data blocks with ranked hot values in the hot storage nodes to the cold storage nodes; or migrating the second data block with the heat value smaller than the second preset heat threshold value in the hot storage node to the cold storage node. Therefore, the second data block transferred from the hot storage node to the cold storage node by the computer equipment is guaranteed to be cold data, the accuracy of the transferred second data block is guaranteed, and hot data in the hot storage node cannot be transferred to the cold storage node. The speed of processing the data on the thermal storage nodes by the computer equipment is further ensured.

In an embodiment of the present application, as shown in fig. 2, the "migrating a first data block with a heat value greater than a first preset heat threshold on a cold storage node to a hot storage node according to each heat value" in step 102 may include the following steps:

step 201, each first data block is put into a first linked list.

The first linked list is a migration queue corresponding to the migration of the first data block from the cold storage node to the hot storage node.

Specifically, after the computer device selects the first data block from the cold storage node according to the heat value of each data block, the computer device may place the selected first data block into the first data linked list.

In an alternative embodiment, when the number of the first data blocks is multiple, the computer device may sort the multiple first data blocks according to the magnitude of the heat value of each first data block. And then the computer device puts the plurality of first data blocks into the first linked list from large to small according to the heat value, so that the first data blocks with larger heat values in the first linked list can be preferentially migrated to the thermal storage node.

In another optional implementation manner, when the number of the first data blocks is multiple, the computer device may sort the multiple first data blocks according to the data size corresponding to each first data block. And then the computer device puts the plurality of first data blocks into the first linked list from small to large according to the data volume, so that the first data blocks with small data volume in the first linked list can be preferentially migrated to the hot storage node.

Step 202, migrating each first data block in the first linked list to a hot storage node.

Specifically, the computer device may migrate to the thermal storage node according to a preset order according to each first data block in the first linked list. The preset sequence may be time sequencing of putting each first data block into the first linked list, or sequencing of the heat value corresponding to each first data block, or sequencing of the data amount corresponding to each first data block, and the preset sequence is not specifically limited in the embodiment of the present application.

In an optional implementation manner, the computer device may further migrate a preset number of first data blocks in the first linked list to the hot storage node each time, and may migrate the next batch of first data blocks after the batch of first data blocks being migrated is completed. The preset number may be one, two, or other number, and the preset number is not specifically limited in the embodiments of the present application.

In an alternative embodiment, the computer device may migrate a first data block in the first linked list to the hot storage node each time, and may migrate the next first data block only after the migration of the first data block being migrated is completed; it is also possible to migrate the next first data block after the first data block being migrated completes the preset ratio. The preset ratio may be 70%, 80% or 90%, and the preset ratio is not specifically limited in the embodiment of the present application.

In this embodiment, the computer device may place each first data block into the first linked list, and migrate each first data block in the first linked list to the hot storage node. The first linked list is a migration queue corresponding to the migration of the first data block from the cold storage node to the hot storage node. Therefore, the first data blocks can be guaranteed to be migrated to the hot storage nodes according to the preset sequence, and the situation that the data volume of the hot storage nodes is sharply increased and the operation of the hot storage nodes is influenced due to the fact that all the first data blocks are migrated to the hot storage nodes from the cold storage nodes at the same time is avoided.

It should be noted that, in this embodiment of the application, when the computer device migrates the second data block in the hot storage node to the cold storage node, the computer device may further put the second data block into the second link, and migrate the second data block in the second link to the cold storage node. And the second link is a migration queue corresponding to the migration of the second data block from the hot storage node to the cold storage node.

In an alternative embodiment, when the number of the second data blocks is multiple, the computer device may sort the multiple second data blocks according to the magnitude of the heat value of each second data block. And then the computer equipment puts the second data blocks into a second linked list from small to large according to the heat value, so that the second data blocks with smaller heat values in the second linked list are preferentially migrated to the cold storage nodes.

In another optional implementation manner, when the number of the second data blocks is multiple, the computer device may sort the multiple second data blocks according to the data size corresponding to each second data block. And then the computer device puts the plurality of second data blocks into the second linked list from large to small according to the data volume, so that the second data blocks with large data volume in the second linked list can be preferentially migrated to the cold storage nodes.

The computer device may migrate to the thermal storage node according to a preset order based on each second data block in the second linked list. The preset sequence may be time sequencing of putting each second data block into the second linked list, or sequencing of the heat value corresponding to each second data block, or sequencing of the data size corresponding to each second data block, and the preset sequence is not specifically limited in the embodiments of the present application.

In an optional implementation manner, the computer device may further migrate a preset number of second data blocks in the second linked list to the cold storage node each time, and may migrate the next batch of second data blocks after the batch of second data blocks being migrated is completed. The preset number may be one, two, or other number, and the preset number is not specifically limited in the embodiments of the present application.

In an alternative embodiment, the computer device may migrate one second data block in the second linked list to the cold storage node each time, and may migrate the next second data block only after the migration of the second data block being migrated is completed; it is also possible to migrate the next second data block after the second data block being migrated completes the preset ratio. The preset ratio may be 70%, 80% or 90%, and the preset ratio is not specifically limited in the embodiment of the present application.

It should be noted that, each first data block in the first linked list and each second data block in the second linked list may be migrated at the same time to complement interference.

In an embodiment of the present application, as shown in fig. 3, the "migrating each first data block in the first linked list to a hot storage node" in step 202 may include the following steps:

step 301, for each first data block in the first linked list, splitting the first data block into a plurality of first data slices.

Wherein, each first data slice has different identification information according to the position of the first data block; the identification information includes a source address of the first data slice at the cold storage node and a destination address of the first data slice at the hot storage node.

Specifically, when the computer device migrates the first data blocks in the first linked list to the hot storage node, the computer device may split each first data block into a plurality of first data pieces according to the size of each first data block because the data volume of each first data block is large. In addition, the computer equipment adds the label information to each first data piece according to the position of each first data piece in the first data block.

And 302, migrating the first data pieces to the thermal storage nodes according to the identification information of the first data pieces according to a preset sequence.

Specifically, the computer device may migrate each first data piece into the thermal storage node according to the location of each first data piece in the first data block according to the identification information of each first data piece.

For example, assuming that the computer device splits the first data block into 5 data pieces, the reference numbers of the 5 first data pieces may be respectively marked as 1, 2, 3, 4, and 5 according to their positions in the first data block. Because the 5 first data slices are all split from the same first data block, the header addresses of the 5 first data slices are the same, and only the last label is different, wherein the identification information of each first data slice may include the header address and the label. The computer device may migrate each first data piece to the thermal storage node according to the header address and the label order of each first data piece.

Step 303, creating a save file based on the identification information of each first data slice and the migration process of each first data slice, and periodically updating the save file according to the migration process of each first data slice.

Specifically, the computer device may record the migration process of each first data slice by using the identification information of each first data slice, and create a save file according to the migration process of each first data slice and the identification information of each first data slice. And the guarantee file is used for saving the migration process of each first data block.

In the process of migrating the first data block, the computer device may periodically update the saved file according to the migration process of each first data piece, so that when a storage system of the computer device has a failure and the data migration process is interrupted, the migration of the first data block may be restarted according to the saved file and, after the failure is repaired, according to the content recorded in the saved file before the data migration process is interrupted.

For example, the computer device may record an offset PBA1(physical block address) of each first data block on a storage disk of the storage system, and obtain an unused storage space from the hot storage node, where a starting address is PBA2, set a pointer to a head address of a first data slice in the first data block currently migrated, read data of a storage sector corresponding to the head address from the disk, write the read data to a destination address on the hot storage node synchronously, and after the first data slice is completely written, update the pointer to a position of a next first data slice, where the pointer writes a file once when updating the first data slice each time. The storage file records information of a first linked list to be migrated, information of a first data block being migrated and a pointed first data slice position.

It should be noted that, in the implementation of the present application, when migrating each second data block in the second linked list to a cold data node, the computer device may also split each second data block into a plurality of second data slices, and migrate the plurality of second data slices to the cold storage node. The specific definition of the computer device migrating the plurality of second data pieces to the cold storage nodes may refer to the definition of the computer device migrating the plurality of first data pieces to the hot storage nodes, and the present application will not be repeated for the sake of brevity.

It should be noted that, in the implementation of the present application, saving the file also saves a process of migrating each second data block in the second linked list to the cold data node by the computer device, and specific limitations will not be repeated.

In this embodiment, for each first data block in the first linked list, the computer device splits the first data block into a plurality of first data pieces, so that each first data block can be conveniently migrated, and the migration speed is increased. In addition, in order to prevent a chaotic situation that each first data piece has a migration error in the migration process, the computer device may further label, according to the position of each first data piece in the first data block, different identification information on each first data piece, and migrate, according to the identification information of each first data piece, each first data piece to the hot storage node according to a preset sequence. Therefore, the situation that each first data slice has no migration error in the migration process can be ensured, and the migration accuracy of each first data block is ensured. In addition, in this embodiment, the computer device creates a save file based on the identification information of each first data piece and the migration process of each first data piece, so that when a failure data migration process interruption occurs in the storage system of the computer device, the migration of the first data block can be restarted according to the save file and the content recorded in the save file before the data migration process interruption after the failure is repaired. In addition, the computer equipment periodically updates the saved files according to the migration process of each first data slice, and the accuracy and traceability of the saved files are guaranteed.

In an embodiment of the present application, as shown in fig. 4, before and after the step 202 "migrating each first data block in the first linked list to the hot storage node", the data migration method may further include the following steps:

step 401, detecting whether each first data block in the first linked list is migrated within a preset time period.

Specifically, after the computer device puts the first data blocks into the first linked list, it may detect whether the first data blocks are migrated within a preset time period. The method comprises the steps that each first data block is migrated from a cold storage node to a hot storage node, and each first data block is migrated from the hot storage node to the cold storage node. The preset time period may be one week or one month, and the preset time period is not specifically limited in the embodiment of the present application.

In step 402, if the first data block is migrated, the migrated first data block is removed from the first linked list.

Specifically, if the first data block that has been migrated exists in the first linked list, in order to prevent misleading to the migration process due to a drastic change in the hot value of the first data block caused by the random access in a short time, the computer device may remove the first data block that has been migrated from the first linked list.

Similarly, the computer device may further detect whether each second data block in the second linked list has been migrated within a preset time period, and remove the migrated second data block from the second linked list if the second data block has been migrated. The method comprises the steps that each second data block is migrated from a cold storage node to a hot storage node, and each second data block is migrated from the hot storage node to the cold storage node.

In this embodiment, the computer device detects whether each first data block in the first linked list has been migrated within a preset time period, and removes the migrated first data block from the first linked list when the first data block has been migrated. Therefore, misleading caused by severe change of the hot value of the first data block due to access randomness in a short time to the migration process can be prevented, and reduction of the service life of the area due to multiple times of erasing of the hot storage nodes is also protected.

In an embodiment of the present application, a data reading and writing method is provided, which is applied to a computer device for description:

the computer equipment receives a read-write instruction aiming at the target first data piece, and detects the current migration condition of the target first data piece. And the target first data slice adopts any one of the data migration methods to perform data migration.

Specifically, the computer device may receive a read-write instruction for the target first piece of data. The computer may detect a current migration condition of the target first piece of data from the saved file.

In one case, the target first data slice is migrated to the thermal storage node, and the data in the target first data slice is read and written according to the target address of the target first data slice at the thermal storage node.

Specifically, if the read-write instruction indicates that the target first data piece to be read and written has been migrated to the thermal storage node, that is, the target first data piece has been migrated, the computer device may perform data reading and writing on the target first data piece according to a destination address of the target first data piece in the thermal storage node included in the read-write instruction.

In one case, if the target first data slice is not migrated to the hot storage node, the data in the target first data slice is read and written according to the source address of the target first data slice at the cold storage node.

Specifically, if the read-write instruction indicates that the target first data piece to be read-written has not been migrated to the hot storage node, that is, the target first data piece has not been migrated yet, and the target first data piece is currently still in the cold storage node, the computer device may perform data reading and writing on the target first data piece according to a destination address, included in the read-write instruction, of the target first data piece in the cold storage node.

In another case, if the target first data slice is being migrated, detecting the content of the read-write command, and performing corresponding operation on the target first data slice according to the content of the read-write command.

Specifically, if there is an intersection between the read-write start address and the read-write end address included in the read-write instruction and the target first data slice being migrated, it indicates that the target first data slice to be read-written in the read-write instruction is being migrated, and the computer device needs to confirm the content in the read-write instruction again, and perform corresponding operation on the target first data slice according to the content in the read-write instruction.

In the embodiment, the computer device receives a read-write instruction for a target first data slice; detecting the current migration condition of a target first data slice; if the target first data piece is migrated to the thermal storage node, reading and writing data in the target first data piece according to the target address of the target first data piece in the thermal storage node; if the target first data slice is not migrated to the hot storage node, reading and writing data in the target first data slice according to the source address of the target first data slice at the cold storage node; and if the target first data slice is being migrated, detecting the content of the read-write instruction, and performing corresponding operation on the target first data slice according to the content of the read-write instruction. Therefore, the data migration can be guaranteed to be completed, and normal read-write processing of the data can be guaranteed not to be influenced. The data processing performance of the computer equipment is further ensured.

In an embodiment of the present application, if the target first data slice is being migrated and corresponding operation is performed on the target first data slice according to the content of the read-write instruction, the following contents may be included:

in one case, if the content of the read-write command is to read the target first data slice, the data in the target first data slice is read according to the source address of the target first data slice at the cold storage node.

Specifically, as can be seen from the above embodiments, in the case that the target first data piece is being migrated, the computer device needs to read the content in the read-write instruction. If the content of the read-write instruction is to read the target first data slice, the computer device can read the data in the target first data slice by using the source address of the target first data slice at the cold storage node because the data of the first data slice can also be read at the cold storage node.

In another case, if the content of the read-write instruction is to write the target first data slice, the read-write instruction is recorded, and after the target first data slice is migrated, data is written into the target first data slice according to the target address of the target first data slice at the thermal storage node.

Specifically, if the content of the read-write instruction is to write the target first data slice, because the target first data slice is being migrated and cannot write data, the computer device may hang the read-write task corresponding to the read-write instruction in the system task linked list, and after the target first data slice is migrated, write data in the target first data slice according to the target address of the target first data slice at the thermal storage node.

In the embodiment of the application, if the content of the read-write instruction is to read the target first data slice, reading the data in the target first data slice according to the source address of the target first data slice at the cold storage node; and if the content of the read-write instruction is to write the target first data slice, recording the read-write instruction, and after the target first data slice is migrated, writing data into the target first data slice according to the target address of the target first data slice in the thermal storage node. Therefore, the read-write task aiming at the target first data piece can be finished under the condition that the target first data piece is being migrated. Therefore, the data migration can be guaranteed to be completed, and normal read-write processing of the data can be guaranteed not to be influenced. The data processing performance of the computer equipment is further ensured.

It should be noted that the methods in the above two embodiments are also applicable to reading and writing the target second data slice, and specific contents will not be repeated in the embodiments of the present application.

For better understanding of the data migration and data reading method provided in the embodiment of the present application, as shown in fig. 5, the embodiment of the present application provides an overall flow of data migration and data reading, which may specifically include the following steps:

step 501, obtaining the heat value of each data block on the cold storage node and the hot storage node.

Step 502, according to each heat value, a first data block with a heat value larger than a first preset heat threshold value on a cold storage node is placed into a first linked list.

Step 503, detecting whether each first data block in the first linked list is migrated within a preset time period.

In step 504, if the first data block is migrated, the migrated first data block is removed from the first linked list.

Step 505, for each first data block in the first linked list, splitting the first data block into a plurality of first data slices.

Step 506, migrating the first data pieces to the thermal storage nodes according to the preset sequence according to the identification information of the first data pieces.

Step 507, creating a save file based on the identification information of each first data slice and the migration process of each first data slice, and periodically updating the save file according to the migration process of each first data slice.

In step 508, the computer device receives a read-write command for the target first piece of data.

In step 509, the current migration situation of the target first data slice is detected.

Step 510, migrating the target first data slice to the thermal storage node, and reading and writing data in the target first data slice according to the target address of the target first data slice at the thermal storage node.

In step 511, the target first data slice is migrated to the thermal storage node, and the data in the target first data slice is read and written according to the target address of the target first data slice at the thermal storage node.

In step 512, if the target first data slice is being migrated, the content of the read/write command is detected.

Step 513, if the content of the read-write instruction is to read the target first data slice, reading the data in the target first data slice according to the source address of the target first data slice at the cold storage node.

And 514, if the content of the read-write instruction is to write the target first data slice, recording the read-write instruction, and after the target first data slice is migrated, writing data in the target first data slice according to the target address of the target first data slice in the thermal storage node.

It should be understood that although the various steps in the flow charts of fig. 1-5 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 1-5 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed in turn or alternately with other steps or at least some of the other steps.

Accordingly, referring to fig. 6, an embodiment of the present invention provides a data migration apparatus 600, where the data migration apparatus 600 includes: an acquisition module 610, a migration module 620, and a detection module 630, wherein:

an obtaining module 610 is configured to obtain a hot value of each data block on the cold storage node and the hot storage node.

And the migration module 620 is configured to migrate, according to each hot value, the first data block with the hot value greater than the first preset hot threshold on the cold storage node to the hot storage node.

The detecting module 630 is configured to detect a usage space of the thermal storage node, and migrate the second data block, whose heat value satisfies a preset condition in the thermal storage node, to the cold storage node if the usage space of the thermal storage node reaches a preset space threshold.

In an embodiment of the present application, the detecting module 630 is specifically configured to migrate the second data blocks with the ranked hot values in the hot storage nodes to the cold storage nodes in a preset number; or migrating the second data block with the heat value smaller than the second preset heat threshold value in the hot storage node to the cold storage node.

In an embodiment of the present application, as shown in fig. 7, the migration module 620 includes: drop-in unit 621 and migration unit 622

The placing unit 621 is configured to place each first data block into a first linked list, where the first linked list is a migration queue corresponding to a migration of the first data block from a cold storage node to a hot storage node;

and a migration unit 622, configured to migrate each first data block in the first linked list to a hot storage node.

In an embodiment of the present application, the migration unit 622 is specifically configured to, for each first data block in the first linked list, split the first data block into a plurality of first data slices, where each first data slice has different identification information according to a position of the first data block; the identification information comprises a source address of the first data slice at the cold storage node and a target address of the first data slice at the hot storage node; and migrating the first data pieces to the thermal storage nodes according to the preset sequence according to the identification information of the first data pieces.

In an embodiment of the application, the migration unit 622 is further configured to create a save file based on the identification information of each first data slice and the migration process of each first data slice, and periodically update the save file according to the migration process of each first data slice.

In an embodiment of the present application, as shown in fig. 8, the migration module 620 further includes: a detection unit 623 and a removal unit 634, wherein:

a detecting unit 623, configured to detect whether each first data block in the first linked list has been migrated within a preset time period;

and a removing unit 634, configured to remove the migrated first data block from the first linked list if the first data block is migrated.

In an embodiment of the present application, a data reading and writing apparatus 900 is provided, where the data reading and writing apparatus 900 includes: a receiving module 910, a detection model 920, a first read/write module 930, a second read/write module 940, and an operation module 950, where:

a receiving module 910, configured to receive a read-write instruction for a target first data slice; and the target first data slice adopts the data migration method in any one of the above embodiments to perform data migration.

And the detection model 920 is used for detecting the current migration condition of the target first data slice.

The first reading/writing module 930 is configured to, if the target first data slice is migrated to the thermal storage node, read and write data in the target first data slice according to a target address of the target first data slice at the thermal storage node.

The second read/write module 940 is configured to, if the target first data piece is not migrated to the hot storage node, read and write data in the target first data piece according to a source address of the target first data piece at the cold storage node.

The operation module 950 detects the content of the read-write command if the target first data slice is being migrated, and performs corresponding operation on the target first data slice according to the content of the read-write command.

In an embodiment of the application, the operation module 950 is specifically configured to, if the content of the read-write instruction is to read the target first data slice, read data in the target first data slice according to a source address of the target first data slice at the cold storage node; and if the content of the read-write instruction is to write the target first data slice, recording the read-write instruction, and after the target first data slice is migrated, writing data into the target first data slice according to the target address of the target first data slice in the thermal storage node.

For specific limitations and beneficial effects of the data migration apparatus and the data reading/writing apparatus, reference may be made to the above limitations on the data migration method and the data reading/writing method, which are not described herein again. All or part of each module in the data migration device and the data reading and writing device can be realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 10. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a data migration and data read-write method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.

In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 11. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing data migration and data reading and writing data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a data migration and data read-write method.

Those skilled in the art will appreciate that the configurations shown in fig. 10 and 11 are merely block diagrams of portions of configurations related to aspects of the present application, and do not constitute limitations on the computing devices to which aspects of the present application may be applied, as a particular computing device may include more or fewer components than shown, or combine certain components, or have a different arrangement of components.

In one embodiment of the present application, there is provided a computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the following steps when executing the computer program: acquiring the heat value of each data block on a cold storage node and a hot storage node; according to each heat value, migrating a first data block with the heat value larger than a first preset heat threshold value on the cold storage node to the hot storage node; and detecting the use space of the hot storage node, and if the use space of the hot storage node reaches a preset space threshold, migrating the second data block with the heat value meeting the preset condition in the hot storage node to the cold storage node.

In one embodiment of the application, the processor when executing the computer program further performs the following steps: migrating a preset number of second data blocks with ranked hot values in the hot storage nodes to the cold storage nodes; or migrating the second data block with the heat value smaller than the second preset heat threshold value in the hot storage node to the cold storage node.

In one embodiment of the application, the processor when executing the computer program further performs the following steps: putting each first data block into a first linked list, wherein the first linked list is a migration queue corresponding to the migration of the first data block from the cold storage node to the hot storage node; and migrating each first data block in the first linked list to the hot storage node.

In one embodiment of the application, the processor when executing the computer program further performs the following steps: for each first data block in the first linked list, splitting the first data block into a plurality of first data slices, wherein each first data slice has different identification information according to different positions of the first data block; the identification information comprises a source address of the first data slice at the cold storage node and a target address of the first data slice at the hot storage node; and migrating the first data pieces to the thermal storage nodes according to the preset sequence according to the identification information of the first data pieces.

In one embodiment of the application, the processor when executing the computer program further performs the following steps: and creating a save file based on the identification information of each first data slice and the migration process of each first data slice, and periodically updating the save file according to the migration process of each first data slice.

In one embodiment of the application, the processor when executing the computer program further performs the following steps: detecting whether each first data block in the first linked list is migrated within a preset time period; and if the first data block is migrated, removing the migrated first data block from the first linked list.

In one embodiment of the application, the processor when executing the computer program further performs the following steps: receiving a read-write instruction aiming at a target first data slice; detecting the current migration condition of a target first data slice; if the target first data piece is migrated to the thermal storage node, reading and writing data in the target first data piece according to the target address of the target first data piece in the thermal storage node; if the target first data slice is not migrated to the hot storage node, reading and writing data in the target first data slice according to the source address of the target first data slice at the cold storage node; and if the target first data slice is being migrated, detecting the content of the read-write instruction, and performing corresponding operation on the target first data slice according to the content of the read-write instruction.

In one embodiment of the application, the processor when executing the computer program further performs the following steps: if the content of the read-write instruction is to read the target first data slice, reading the data in the target first data slice according to the source address of the target first data slice at the cold storage node; and if the content of the read-write instruction is to write the target first data slice, recording the read-write instruction, and after the target first data slice is migrated, writing data into the target first data slice according to the target address of the target first data slice in the thermal storage node.

In one embodiment of the present application, there is provided a computer readable storage medium having a computer program stored thereon, the computer program when executed by a processor implementing the steps of: acquiring the heat value of each data block on a cold storage node and a hot storage node; according to each heat value, migrating a first data block with the heat value larger than a first preset heat threshold value on the cold storage node to the hot storage node; and detecting the use space of the hot storage node, and if the use space of the hot storage node reaches a preset space threshold, migrating the second data block with the heat value meeting the preset condition in the hot storage node to the cold storage node.

In one embodiment of the application, the computer program when executed by the processor further performs the steps of: migrating a preset number of second data blocks with ranked hot values in the hot storage nodes to the cold storage nodes; or migrating the second data block with the heat value smaller than the second preset heat threshold value in the hot storage node to the cold storage node.

In one embodiment of the application, the computer program when executed by the processor further performs the steps of: putting each first data block into a first linked list, wherein the first linked list is a migration queue corresponding to the migration of the first data block from the cold storage node to the hot storage node; and migrating each first data block in the first linked list to the hot storage node.

In one embodiment of the application, the computer program when executed by the processor further performs the steps of: for each first data block in the first linked list, splitting the first data block into a plurality of first data slices, wherein each first data slice has different identification information according to different positions of the first data block; the identification information comprises a source address of the first data slice at the cold storage node and a target address of the first data slice at the hot storage node; and migrating the first data pieces to the thermal storage nodes according to the preset sequence according to the identification information of the first data pieces.

In one embodiment of the application, the computer program when executed by the processor further performs the steps of: and creating a save file based on the identification information of each first data slice and the migration process of each first data slice, and periodically updating the save file according to the migration process of each first data slice.

In one embodiment of the application, the computer program when executed by the processor further performs the steps of: detecting whether each first data block in the first linked list is migrated within a preset time period; and if the first data block is migrated, removing the migrated first data block from the first linked list.

In one embodiment of the application, the computer program when executed by the processor further performs the steps of: receiving a read-write instruction aiming at a target first data slice; detecting the current migration condition of a target first data slice; if the target first data piece is migrated to the thermal storage node, reading and writing data in the target first data piece according to the target address of the target first data piece in the thermal storage node; if the target first data slice is not migrated to the hot storage node, reading and writing data in the target first data slice according to the source address of the target first data slice at the cold storage node; and if the target first data slice is being migrated, detecting the content of the read-write instruction, and performing corresponding operation on the target first data slice according to the content of the read-write instruction.

In one embodiment of the application, the computer program when executed by the processor further performs the steps of: if the content of the read-write instruction is to read the target first data slice, reading the data in the target first data slice according to the source address of the target first data slice at the cold storage node; and if the content of the read-write instruction is to write the target first data slice, recording the read-write instruction, and after the target first data slice is migrated, writing data into the target first data slice according to the target address of the target first data slice in the thermal storage node.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a Flash Memory (Flash Memory), a Hard Disk (Hard Disk Drive, abbreviated as HDD), a Solid State Drive (SSD), or the like; the storage medium may also comprise a combination of memories of the kind described above.

Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A method of data migration, the method comprising:

acquiring the heat value of each data block on a cold storage node and a hot storage node;

according to each heat value, migrating a first data block with the heat value larger than a first preset heat threshold value on the cold storage node to the hot storage node;

and detecting the use space of the hot storage node, and if the use space of the hot storage node reaches a preset space threshold, migrating a second data block with a heat value meeting a preset condition in the hot storage node to the cold storage node.

2. The method of claim 1, wherein migrating the second data block with the hot storage node hot value satisfying a predetermined condition to the cold storage node comprises:

migrating a preset number of second data blocks with ranked hot values in the hot storage nodes to the cold storage nodes; or

And migrating the second data block with the heat value smaller than a second preset heat threshold value in the hot storage node to the cold storage node.

3. The method of claim 1, wherein migrating the first data blocks with the heat value greater than a first preset heat threshold on the cold storage node to the hot storage node according to each of the heat values comprises:

putting each first data block into a first linked list, wherein the first linked list is a migration queue corresponding to the first data block which is migrated from the cold storage node to the hot storage node;

and migrating each first data block in the first linked list to the hot storage node.

4. The method of claim 3, wherein migrating each of the first data chunks in the first linked list to the thermal storage node comprises:

for each first data block in the first linked list, splitting the first data block into a plurality of first data slices, wherein each first data slice has different identification information according to different positions of the first data block; the identification information comprises a source address of the first data slice at the cold storage node and a target address of the first data slice at the hot storage node;

and migrating the first data pieces to the thermal storage nodes according to the identification information of the first data pieces and a preset sequence.

5. The method of claim 4, further comprising:

creating a save file based on the identification information of each first data slice and the migration process of each first data slice, and periodically updating the save file according to the migration process of each first data slice.

6. The method of claim 3, wherein prior to migrating each of the first data chunks in the first linked list to the hot storage node, the method further comprises:

detecting whether each first data block in the first linked list is migrated within a preset time period;

and if the first data block is migrated, removing the migrated first data block from the first linked list.

7. A method for reading and writing data, comprising:

receiving a read-write instruction aiming at a target first data slice, wherein the target first data slice carries out data migration by adopting the method of any one of claims 1 to 6;

detecting the current migration condition of the target first data slice;

if the target first data slice is migrated to the thermal storage node, reading and writing data in the target first data slice according to a target address of the target first data slice at the thermal storage node;

if the target first data slice is not migrated to the hot storage node, reading and writing data in the target first data slice according to a source address of the target first data slice at the cold storage node;

and if the target first data slice is being migrated, detecting the content of the read-write instruction, and performing corresponding operation on the target first data slice according to the content of the read-write instruction.

8. The method of claim 7, wherein the detecting the content of the read/write command and operating the target first data slice according to the content of the read/write command if the target first data slice is being migrated comprises:

if the content of the read-write instruction is to read the target first data slice, reading the data in the target first data slice according to the source address of the target first data slice at the cold storage node;

and if the content of the read-write instruction is to write the target first data slice, recording the read-write instruction, and after the target first data slice is migrated, writing data in the target first data slice according to the target address of the target first data slice in the hot storage node.

9. An apparatus for data migration, the apparatus comprising:

the acquisition module is used for acquiring the heat values of all data blocks on the cold storage node and the hot storage node; the migration module is used for migrating the first data blocks with the heat values larger than a first preset heat threshold value on the cold storage nodes to the hot storage nodes according to the heat values;

and the detection module is used for detecting the use space of the hot storage node, and if the use space of the hot storage node reaches a preset space threshold, migrating the second data block with the heat value meeting a preset condition in the hot storage node to the cold storage node.

10. A computer device comprising a memory and a processor, wherein the memory and the processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the data migration method according to any one of claims 1 to 8.

11. A computer-readable storage medium storing computer instructions for causing a computer to perform the data migration method of any one of claims 1-8.