CN110083312B - Disk capacity expansion method and device and computer equipment - Google Patents
Disk capacity expansion method and device and computer equipment Download PDFInfo
- Publication number
- CN110083312B CN110083312B CN201910350207.4A CN201910350207A CN110083312B CN 110083312 B CN110083312 B CN 110083312B CN 201910350207 A CN201910350207 A CN 201910350207A CN 110083312 B CN110083312 B CN 110083312B
- Authority
- CN
- China
- Prior art keywords
- data
- disk
- hash value
- hash
- stored
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0647—Migration mechanisms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Signal Processing For Digital Recording And Reproducing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a disk capacity expansion method, a device and computer equipment, after a distributed storage system receives data to be stored, the data to be stored is mapped into a hash ring according to a hash value of the data to be stored, so as to determine the position of the data to be stored in the hash ring, further preliminarily determine which two disks mapped into the hash ring can be subjected to disk capacity expansion, and then calculate the disk hash value of a capacity expansion disk by using the data hash value between the two disks and the data hash value of the data to be stored, namely determine the position of the hash ring to which the capacity expansion disk should be mapped, ensure the load balance of each disk, and reduce the quantity of data to be transferred and the quantity of disks influenced after the data to be stored is added.
Description
Technical Field
The present application relates to the field of communications technologies, and in particular, to a disk capacity expansion method and apparatus, and a computer device.
Background
In the application process of the distributed storage system, with the increase of the traffic, the disk often has a situation of insufficient space, and disk expansion needs to be performed, that is, data of an old disk is migrated to a new disk so as to vacate the disk space and continue to perform service operation.
In the disk capacity expansion process, the migration data volume of the old disk, the number of disks needing data migration, and the distribution condition of data in each disk often affect the normal operation of the service, and further affect the working efficiency.
In view of this, the prior art proposes to perform a remainder operation on the number of disks by calculating the hash value of the data and using the hash value to scatter the data into the disks. However, in the disk capacity expansion method, each time one data is added, the number of disks affected by the method is large, and thus the amount of migrated data is large, which is not favorable for service operation.
Disclosure of Invention
In view of this, the present application provides a disk capacity expansion method, device and computer device, which greatly reduce the migration data volume and disk number affected by disk capacity expansion, and ensure balanced data access of each disk after capacity expansion as much as possible, thereby ensuring reliable and efficient operation of services.
In order to achieve the above object, the present application provides the following technical solutions:
a disk capacity expansion method comprises
Acquiring data to be stored, and calculating a data hash value of the data to be stored;
mapping the data to be stored to a hash ring according to the data hash value of the data to be stored, and obtaining the data hash value of target data in the hash ring, wherein the hash ring is mapped with a first number of disks and each data in the disks, the target data comprises the data mapped to a first disk, and the first disk is a disk in the first number of disks and provided with a disk hash value matched with the data hash value of the data to be stored;
determining a disk hash value mapped by the capacity expansion disk in the hash ring by using the data hash value of the target data and the data hash value of the data to be stored;
determining migration data in the target data according to the disk hash value of the capacity expansion disk and the data hash value of the target data, and mapping the migration data and the data to be stored into the capacity expansion disk.
Optionally, the method further includes:
according to a random hash algorithm, mapping data stored in a distributed storage system to a hash ring, wherein the hash ring comprises a data hash value of the data;
averagely mapping a first number of disks in the distributed storage system to the hash ring, and determining the disk hash value of each disk;
and mapping the data mapped to the hash ring to a corresponding disk according to the first rotation direction of the hash ring.
Optionally, the mapping the first number of disks in the distributed storage system to the hash ring on average, and determining the disk hash value of each disk includes:
acquiring respective binary codes of a first number of disks in the distributed storage system, wherein the first number of binary codes are continuous binary numbers;
zero padding is carried out on the binary number, and the obtained 32-bit binary number is used as a disk hash value of a corresponding disk;
mapping the first number of disks into the hash ring using the disk hash value, so that the first number of disks are evenly distributed in the hash ring.
Optionally, the method further includes:
determining the hash value ranges corresponding to the first number of disks according to the first rotation direction by using the respective disk hash values of the first number of disks;
the obtaining of the data hash value of the target data in the hash ring includes:
determining a target hash value range to which the data hash value of the data to be stored belongs from hash value ranges corresponding to the first number of disks respectively;
and taking the disk corresponding to the target hash value range as a first disk, and acquiring a data hash value of target data mapped to the first disk.
Optionally, the determining, by using the data hash value of the target data and the data hash value of the data to be stored, the disk hash value of the capacity expansion disk mapped on the hash ring includes:
calculating to obtain a hash expected value of the data mapped to the first disk by using the data hash value of the target data and the data hash value of the data to be stored;
and taking the hash expected value as a disk hash value of a capacity expansion disk, and mapping the capacity expansion disk into the hash ring.
Optionally, after acquiring the data to be stored, the method further includes:
detecting whether a second disk with a storage space allowance exists in a first number of disks of the distributed storage system;
if so, writing the device to be stored into the corresponding second disk according to the size of the storage space of the second disk;
and if not, executing the step of calculating the data hash value of the data to be stored.
Optionally, the determining migration data in the target data according to the disk hash value of the capacity expansion disk and the data hash value of the target data includes:
determining fixed data of a data hash value between the disk hash value of the first disk and the disk hash value of the capacity expansion disk in the target data;
and taking the target data except the fixed data in the target data as migration data, and clearing the mapping relation between the migration data and the first disk.
A disk capacity expansion apparatus, the apparatus comprising:
the data hash value calculation module is used for acquiring data to be stored and calculating the data hash value of the data to be stored;
the first mapping module is used for mapping the data to be stored to a hash ring according to the data hash value of the data to be stored, and acquiring the data hash value of target data in the hash ring, wherein the hash ring is mapped with a first number of disks and each data in the disks, the target data comprises the data mapped to a first disk, and the first disk is a disk in the first number of disks and has a disk hash value matched with the data hash value of the data to be stored;
the capacity expansion disk determining module is used for determining a disk hash value of the capacity expansion disk mapped on the hash ring by using the data hash value of the target data and the data hash value of the data to be stored;
and the data migration module is used for determining migration data in the target data according to the disk hash value of the capacity expansion disk and the data hash value of the target data, and mapping the migration data and the data to be stored into the capacity expansion disk.
Optionally, the apparatus further comprises:
a hash value range determining module, configured to determine, according to a first rotation direction of the hash ring, hash value ranges corresponding to the first number of disks by using disk hash values of the first number of disks;
the first mapping module comprises:
a first determining unit, configured to determine, from hash value ranges corresponding to the first number of disks, a target hash value range to which a data hash value of the data to be stored belongs;
and the first acquisition unit is used for taking the disk corresponding to the target hash value range as a first disk and acquiring the data hash value of the target data mapped to the first disk.
A computer device, comprising:
a plurality of disks for storing data;
a memory for storing a program for implementing the disk capacity expansion method;
and the processor is used for loading and executing the program stored in the memory to realize the steps of the disk capacity expansion method.
Therefore, compared with the prior art, the disk capacity expansion method, device and computer equipment provided by the application have the advantages that after the distributed storage system receives the data to be stored, the data to be stored is mapped into the hash ring according to the hash value of the data to be stored, so that the position of the data to be stored in the hash ring is determined, further, the disk capacity expansion between two disks mapped into the hash ring is preliminarily determined, then, the disk hash value of the capacity expansion disk is calculated by using the data hash value between the two disks and the data hash value of the data to be stored, namely, the position of the capacity expansion disk to be mapped into the hash ring is determined, the load balance of each disk is ensured, and the data quantity to be transferred and the quantity of disks influenced after the data to be stored is added are reduced.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 illustrates a flowchart of a disk capacity expansion method according to an embodiment of the present application;
fig. 2 illustrates a schematic diagram of mapping data onto a hash ring in a disk capacity expansion method provided by an embodiment of the present application;
fig. 3 is a schematic diagram illustrating mapping of data to a disk in a disk capacity expansion method according to an embodiment of the present application;
fig. 4 is a schematic diagram illustrating migration data is mapped to a capacity expansion disk in a disk capacity expansion method according to an embodiment of the present application;
fig. 5 is a flowchart illustrating another disk capacity expansion method according to an embodiment of the present application;
fig. 6 shows a hash tree diagram for obtaining a disk hash value in a disk capacity expansion method provided in an embodiment of the present application;
fig. 7 shows a hash tree diagram for obtaining a disk hash value of an expanded disk in a disk expansion method provided in an embodiment of the present application
Fig. 8 is a schematic structural diagram illustrating a disk capacity expansion apparatus according to an embodiment of the present application;
fig. 9 is a schematic structural diagram of another disk capacity expansion apparatus according to an embodiment of the present application;
fig. 10 is a schematic structural diagram of another disk capacity expansion apparatus according to an embodiment of the present application;
fig. 11 is a schematic structural diagram of another disk capacity expansion apparatus according to an embodiment of the present application;
fig. 12 shows a hardware structure diagram of a computer device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In order to make the aforementioned objects, features and advantages of the present application more comprehensible, the present application is described in further detail with reference to the accompanying drawings and the detailed description.
Referring to fig. 1, a flowchart of a disk capacity expansion method provided in an embodiment of the present application is shown, where the method may include, but is not limited to, the following steps:
step S101, acquiring data to be stored;
step S102, calculating a data hash value of the data to be stored;
the data to be stored in this embodiment may need to be stored in a disk of the computer device, and specifically may be service data of any service that the computer device can provide.
The embodiment is suitable for the situation that the disk space is insufficient and the disk needs to be expanded, and in order to reduce the number of migrated disks and the data volume and ensure the load balance of each disk after the disk is expanded, the expanded disk can be positioned by utilizing the Hash ring.
Based on this, the embodiment may use a random hash algorithm to calculate the hash value of the data to be stored, and a specific implementation process is not described in detail. In general, random hash calculation is performed on data to be stored, and a value range of an obtained data hash value may be represented by a hexadecimal number, where the value range may specifically be: (000000000000-0 xFFFFFFFFFFFFFFFFFFFFFFF).
In this embodiment, a consistent hash algorithm, which is modulo 2^32, i.e., the whole hash value space is organized into a virtual ring, can be used to implement data caching, and the hash value space of the hash value in the hash value range of the above embodiment can also be 0 ^ 2^32-1 (i.e., the hash value is a 32-bit unsigned integer).
Step S103, mapping the data to be stored to a hash ring according to the data hash value of the data to be stored;
step S104, acquiring a data hash value of the target data in the hash ring;
in practical application of the embodiment, for data stored in the first number of disks in the distributed storage system, a random hash algorithm may be used to map the data into the hash ring generated above, such as the hash ring shown in fig. 2, which is schematically illustrated by only taking mapping results of data hash values (e.g., hash (data1) to hash (data8) in fig. 2) of data1 to data8 as an example, and is not limited to the hash values mapped onto the hash values in fig. 2, and the embodiment does not limit the number of disks in the distributed storage system, that is, specific numerical values of the first number.
Similarly, in this embodiment, the respective disk hash values of the first number of disks may also be calculated and mapped into the hash ring, so that the mapped disks can be evenly distributed in the hash ring, and the specific implementation method is not limited in this application. Therefore, the hash ring constructed in this embodiment is mapped with the first number of disks and the data in the disks, and the first number of disks mapped onto the hash ring may divide the hash value range of the whole hash ring into the first number of sub-ranges, and then, the hash value range corresponding to each disk may be determined according to the mapping relationship between the data and the disks, that is, the disk corresponding to the hash value range between two adjacent disks is determined according to the first rotation direction.
Based on this, in this embodiment, after determining the hash value ranges corresponding to the first number of disks according to the first rotation direction by using the disk hash values of the first number of disks, the data hash value of the data to be stored may be compared with the hash value ranges, a target hash value range to which the data hash value of the data to be stored belongs is determined, the disk corresponding to the target hash value range is used as the first disk, and the data mapped to the first disk is used as the target data, so as to obtain the data hash value of the target data. Therefore, the first disk of this embodiment may be a disk of the first number of disks, which has a disk hash value matching the data hash value of the data to be stored (i.e., a disk hash value belonging to the hash range to which the data hash value belongs).
Referring to the schematic diagram of the hash ring mapped with disks and their data shown in fig. 3, the data mapped on the hash ring may be mapped to the first Disk according to the same rotation direction, as shown in fig. 3, the data may be mapped to the first Disk according to a counterclockwise direction, as shown in fig. 3, for an example of mapping relationships between data1 to data8 and disks 0 to Disk3, but is not limited to the counterclockwise mapping manner shown in fig. 3.
In addition, according to the counterclockwise mapping manner shown in fig. 3, on the hash ring, the hash value range from the Disk hash value of the Disk0 to the Disk hash value of the Disk1 is actually composed of data hash values that can be mapped to the data in the Disk0, and therefore, the hash value range corresponds to the Disk0, and includes the Disk hash value of the Disk0 and does not include the Disk hash value of the Disk 1; similarly, the hash value range from the Disk hash value of the Disk1 to the Disk hash value of the Disk2 may be determined, and corresponds to the Disk 1; the hash value range from the Disk hash value of the Disk2 to the Disk hash value of the Disk3 corresponds to the Disk 2; the hash value range from the Disk hash value of the Disk3 to the Disk hash value of the Disk0 corresponds to the Disk 3.
Then, after mapping the data to be stored to the hash ring according to the mapping manner shown in fig. 3, taking the data to be stored as data9 as an example, mapping the data hash value of the data to be stored to the hash ring, and comparing the data hash value with the existing hash value range of each Disk, it is known that, as shown in fig. 3, the data hash value of the data to be stored is located between the Disk hash values of the Disk0 and the Disk1, specifically, is located in the hash value range corresponding to the Disk0, and according to the mapping manner described above, when the storage space of the Disk0 is sufficient, the data to be stored should be mapped to the Disk 0. If the storage space of the Disk0 is not enough, the data to be stored cannot be written into the Disk0, and in order to ensure reliable storage of the data and normal operation of business operations, a Disk expansion operation is often required, that is, a new Disk is expanded.
In this case, according to the mapping relationship between the data and the disk described above, it is often necessary to migrate data in some old disks to new disks in order to free up disk space for continuing business operations. However, in the conventional disk capacity expansion method based on the remainder removing algorithm, the number of disks used is 1 each time a disk is expanded, but the migration data volume and the number of disk nodes affected by the disk capacity expansion are large, as shown in the following table 1, as the result of the influence of disk capacity expansion on the migration data volume and the number of disk nodes, the number of disk nodes is affected by 3.5 disks every time one data is added, and the migration data volume is 3.
TABLE 1
In order to improve the disk capacity expansion method based on the basic remainder removing algorithm, the basic remainder removing algorithm is improved to obtain the stable remainder removing algorithm, and in the method, when the disk capacity is expanded each time, the number of the used disks is usually the number of the disks before the capacity expansion, namely the total number of the disks is twice of the number of the original disks. In this scheme, even if the disk capacity is expanded at the best time of disk capacity expansion, that is, under the condition that the data to be stored is added to the disk with the most distributed data, the migration data volume and the number of the disk nodes which are influenced by the disk capacity expansion are also large, specifically, as shown in the following table 2, the influence of the disk capacity expansion on the migration data volume and the number of the disk nodes is a result, in this improvement scheme, when one data is added, 2.4 disk nodes are influenced, and the influenced migration data volume is 1.2, and when the disk capacity expansion is performed once when one data is added, the disk nodes and the migration data volume influenced by the two schemes will be more, which seriously influences the reliable operation of the service.
TABLE 2
In contrast, according to the present application, it is desirable to expand the disk at the worst time of disk expansion (i.e., perform disk expansion once every time data is added as described above), so that the amount of migrated data and the number of disks can be reduced as much as possible, and load balance of each disk is ensured, thereby reducing adverse effects on services caused by data migration during disk expansion.
Specifically, according to the manner described in the above steps, the disk and the data stored in the disk are mapped onto the hash ring, and after the data to be stored that needs to be added this time is received, the data is also mapped into the hash ring, and the position of the data in the hash ring is determined, that is, the size relationship between the data hash value of the data to be stored and the disk hash value of the existing disk is determined, that is, the position relationship between the data to be stored and each disk after being mapped onto the hash ring is determined.
After that, while reducing the amount of migrated data and the number of disks, and ensuring load balance of each disk as much as possible, in this embodiment, the addition of the data to be stored may be considered, only the mapping relationship between the data adjacent to the position of the data to be stored on the hash ring and the disk is affected, and only the disk with the closest hash value on the hash ring is affected, so after the data to be stored is mapped onto the hash ring, the disk hash value adjacent to the data hash value of the data to be stored may be determined, the disk with the disk hash value is marked as a first disk, and the data mapped to the first disk is acquired and marked as target data.
As shown in the hash ring shown in fig. 3, data mapped to the Disk0 between the Disk0 and the Disk1 may be recorded as target data, such as data6 and data8, in practical applications, the data volume of the target data is often large, and the specific data volume may be determined according to the size of the Disk storage space and the specific service operation condition.
Step S105, determining a disk hash value of the expansion disk mapped on the hash ring by using the data hash value of the target data and the data hash value of the data to be stored;
after the foregoing analysis, the present application expects that the load of each disk can be balanced as much as possible after the disk expansion is completed, so when the position of the expanded disk is determined in the hash ring, the data hash value of the target data obtained above, that is, the data in the disk that may be affected by the disk expansion this time, may be used, in combination with the data hash value of the data to be stored that is added this time, to determine where the expanded disk should be located in the hash ring, where the affected migration number and the disk number are the smallest, and balance of the data amount in each disk is maintained as much as possible, which is not limited by the specific implementation method in this embodiment.
Optionally, in this embodiment, after the data to be stored is added, an expected value of a data hash value of data in a disk affected by the data to be stored, that is, an average value of data hash values of data in the first disk, is calculated, and the expected value is used as the disk hash value of the expansion disk, that is, the position of the expansion disk in the hash ring is determined, but the method is not limited to this method for determining the position of the expansion disk.
And step S106, determining migration data in the target data according to the disk hash value of the capacity expansion disk and the data hash value of the target data, and mapping the migration data and the data to be stored into the capacity expansion disk.
After the mapping position of the expanded disk on the hash ring is determined, the mapping relationship between the data currently mapped onto the hash ring and the disk needs to be adjusted, and especially, due to the addition of the expanded disk, the data in the disk affected by the addition of the expanded disk often needs to be mapped into the expanded disk.
Specifically, since the manner in which the data is mapped to the disk is not adjusted, after the mapping position of the capacity expansion disk is determined, the data located between the capacity expansion disk and the first disk, that is, the data whose data hash value is between the disk hash values of the first disk and the capacity expansion disk, is determined as non-migration data, then other data in the first disk is migration data, this embodiment needs to migrate this portion of migration data to the capacity expansion disk, that is, remove the mapping relationship between the migration data and the first disk, and remap the migration data to the capacity expansion disk again.
As shown in fig. 4, the newly added data to be stored data9 may still be directly mapped to the capacity expansion disk in the counterclockwise direction of the hash ring, and the data6 originally mapped to the first disk will be migrated and mapped to the capacity expansion disk. By combining the disk capacity expansion method provided by the present application shown in table 3, it is obvious that the influence on the migration data volume and the number of disks is serious, after the disk capacity expansion is completed, the influenced migration data volume is 1 part, and the influenced number of disks is 1, so it is obvious that the adverse effect is reduced to the minimum compared with the existing disk capacity expansion method, and the application also considers the load balance condition of each disk after the data migration, thereby improving the reliability of service operation.
Optionally, referring to fig. 5, which is a flowchart illustrating a further embodiment of the disk capacity expansion method provided by the present application, this embodiment may be a refinement scheme of the disk capacity expansion method described in the foregoing embodiment, but in practical application, the refinement scheme is not limited to be described in this embodiment, and as shown in fig. 5, the disk capacity expansion method provided by this embodiment may include, but is not limited to, the following steps:
step S201, mapping data stored in the distributed storage system to a hash ring according to a random hash algorithm;
in this embodiment, as to the process of constructing the hash ring, reference may be made to the description of the corresponding parts of the above embodiments.
Step S202, acquiring respective binary codes of a first number of disks in the distributed storage system;
step S203, zero padding is carried out on the binary number, and the obtained 32-bit binary number is used as a disk hash value of a corresponding disk;
step S204, mapping a first number of disks into a hash ring by using disk hash values so as to enable the first number of disks to be evenly distributed in the hash ring;
in this embodiment, the first number is 4, and referring to a schematic diagram of a position relationship between a disk and data mapped on a hash ring shown in fig. 2, in order to ensure that 4 disks are evenly distributed in the hash ring, a new hash algorithm is proposed in this embodiment to calculate a disk hash value of each disk.
In this embodiment, a hash tree is taken as a data structure to describe the Disk hash values of the disks, specifically, four disks shown in fig. 2 are taken as an example for description, since the four disks are evenly distributed in the hash ring, if the Disk0 is denoted as D0, the encoding may be 00; disk1 is denoted as D1, which may be encoded as 01; disk2 is denoted as D2, which may be encoded as 10; the Disk3 is denoted as D3, and its code may be 11, so that a hash tree as shown in fig. 6 can be obtained, in the hash tree, if a child node is the same as a parent node, its corresponding number is 0, otherwise, it is 1, that is, the number corresponding to a newly appearing child node may be 1, as shown in fig. 6, when viewed from the root node downwards, the child nodes whose numbers correspond to the child nodes increasing on the left side are all new nodes whose corresponding numbers are 0, and the leaf nodes are found from the root node, and the numbers sequentially obtained from the path can constitute the code of the Disk. In this way, the present embodiment may expand the hash tree, determine the number of the added new node (i.e., the expanded disk), and further obtain the hash value of the new node.
Since the hash value of the hash ring constructed in the present application is 32 bits, in order to facilitate calculation, in this embodiment, a first occurring disk (i.e., a leaf node) is searched from a root node, 32 bits are filled right after a corresponding encoding-use 0 is obtained, and the obtained value may be used as the disk hash value of the disk. Therefore, the disk hash value of the disk D0 obtained in this embodiment may be 00000000000000000000000000000000 — 0; the disk hash value of disk D1 may be 01000000000000000000000000000000 ^ 2^ 30; the disk hash value of disk D2 may be 10000000000000000000000000000000 ^ 2^ 31; the disk hash value for disk D3 may be 11000000000000000000000000000000 ═ 3 × 2^ 30.
Step S205, according to the first rotation direction of the hash ring, mapping the data mapped to the hash ring to a corresponding disk;
the present application mainly uses the first rotation direction as the counterclockwise direction for example to explain, and certainly, the first rotation direction may also be a clockwise direction, and no matter which rotation direction is used, the method of mapping data into the disk is similar, and the detailed description is not repeated in the present application.
Step S206, acquiring data to be stored, and calculating a data hash value of the data to be stored;
step S207, detecting whether a second disk with the surplus of the storage space exists in the first quantity of disks, and if so, entering step S208; if not, go to step S209;
step S208, writing the device to be stored into the corresponding second disk according to the size of the storage space of the second disk;
step S209, according to the data hash value of the data to be stored, mapping the data to be stored into a hash ring;
it can be seen that, after receiving new data to be stored, the distributed storage system of this embodiment may first determine whether the storage space of each disk currently exists is sufficient, and if so, may directly write the data to be stored into a disk with sufficient storage space, and specifically may write the data to be stored into a disk with the largest storage space allowance, but is not limited thereto; if the storage space of each disk is insufficient, a new disk needs to be expanded so as to write the data to be stored into the new disk, and meanwhile, the storage capacity of other disks can be adjusted so that the other disks have the surplus of the storage space to store the new data.
Step S210, determining the data hash value of the target data in the hash ring according to the data hash value of the data to be stored and the disk hash values mapped to the disks in the hash ring;
in this embodiment, in order to ensure load balancing of each disk in the system after capacity expansion of the disk is performed as much as possible, a mean (or expected) calculation manner may be used to determine the position of the capacity expansion disk, so that in this embodiment, a data hash value of a calculated object, that is, target data, may be determined first, here, in order to reduce a calculation amount, the embodiment may determine a disk hash value range where the data hash value of the data to be stored is located first, record a disk having the disk hash value range as a first disk, and record data mapped to the first disk as the target data, where a specific implementation process may refer to descriptions of corresponding parts in the foregoing embodiments.
Step S211, calculating to obtain a hash expected value of the data mapped to the first disk by using the data hash value of the target data and the data hash value of the data to be stored;
step S212, using the hash expected value as a disk hash value of the capacity expansion disk, and mapping the capacity expansion disk into a hash ring;
it should be understood that, in the current stage of this embodiment, the data mapped to the first disk (that is, the disk corresponding to the hash value range where the data hash value of the data to be stored is located) includes the data to be stored, and the data mapped to the first disk before receiving the data to be stored, in this embodiment, the expected value may be calculated by using the respective data hash values of the data to obtain the expected hash value; of course, the present application may also directly calculate an average value of the data hash values of the data, and use the average value as the expected hash value of the data of the first disk.
For example, suppose that the data hash value of the data to be stored data9 is obtained by calculation according to the method described in this embodiment, and is 855294057, the data hash value is converted into a 32-bit binary number of 00110010111110101100000001101001, and the hash value is compared with the disk hash values of other disks in the hash ring, specifically, the hash value ranges corresponding to other disks are compared, it is known that the data hash value of the data to be stored is located in the hash value range corresponding to the disk D0, that is, the mapping position of the data to be stored in the hash ring is between the mapping positions of the disk D0 and the disk D1, and the data to be stored should be mapped to the disk D0 according to the mapping relationship in the counterclockwise direction, but because the disk D0 has insufficient storage space, the data cannot be directly stored, and this embodiment may expand a new disk between the two disks.
According to the method described in the above steps, after the data to be stored is mapped to the disk D0 (i.e., the first disk), the total expected hash value of the data mapped by the first disk is calculated, for example, after the calculation, the expected hash value is 00110010111110100100010001100000, and this embodiment may directly use it as the disk hash value of the expansion disk, and thus characterize the mapping position of the disk hash value in the hash ring.
Still, the search path of the Disk hash value of the expanded Disk is described by using the hash tree structure in the above embodiment, and according to the hash tree expansion rule described above, referring to the expanded hash value tree shown in fig. 7, in the expanded hash value tree, the Disk (node) of the ith bit in the tree may be searched from back to front, and the code of the search path may be used as the Disk hash value of the expanded Disk, and if the expanded Disk4 is denoted as D4, the hash value may be 00110010111110100100010001100000. It should be noted that fig. 7 does not show all nodes of the search capacity expansion disk path, and when a node of the next layer is expanded, the node which is the same as the parent node and is numbered as 0 may be added to the left, and a new node which is different from the parent node and is numbered as 0 may be added to the right, so that the expanded hash tree structure is not described in detail in this embodiment. .
Step S213, determining fixed data of the data hash value between the disk hash value of the first disk and the disk hash value of the expansion disk from the target data;
step S214, using the target data except the fixed data in the target data as migration data, and clearing the mapping relationship between the migration data and the first disk.
After the mapping position of the capacity expansion disk in the hash ring is determined, data to be migrated due to adding the capacity expansion disk may be further determined, and since the migrated data is included in the target data determined above and the manner of determining the mapping relationship between the data and the disk is not changed in the process, the embodiment may determine, based on the manner, data that does not need to be migrated from the target data, and record the data as fixed data, and then determine other data in the target data as migrated data.
Still as illustrated in the example of fig. 4, in the hash ring shown in fig. 4, because the expanded Disk4 is added between the Disk D0 and the Disk D1, so that the Disk closest to the mapping position of the data between the Disk D4 and the Disk D1 is no longer the Disk D0 but becomes the Disk D4 in the counterclockwise manner of the hash ring, based on the above-described method for determining the mapping relationship between the data and the Disk, the part of the data needs to be migrated from the Disk D0 to the Disk D4, that is, the part of the data is migrated data affected by the expanded Disk. The data between the disk D0 and the disk D4 is still mapped to the disk D0 according to the counterclockwise mapping, and the mapping relationship does not need to be changed, so that the data is recorded as fixed data. In this embodiment, the disk D0 is the first disk of this embodiment.
It should be noted that the determination method regarding the above migration data and fixed data is not limited to the method described in the present embodiment.
In summary, even under the worst condition that disk expansion is performed every time data is added, the disk hash value of the expanded disk is obtained according to the new hash value calculation method provided in this embodiment, the amount of data to be migrated and the number of disks are both greatly reduced each time disk expansion is performed, for example, only one disk is migrated if the amount of data migrated is affected by this embodiment, and meanwhile, when determining the disk hash value of the expanded disk, load feedback is introduced in consideration of the data hash values of other data mapped to the disk.
Referring to fig. 8, which is a schematic structural diagram of a disk capacity expansion apparatus provided in an embodiment of the present application, the apparatus may include:
the data hash value calculation module 11 is configured to acquire data to be stored, and calculate a data hash value of the data to be stored;
in practical application of this embodiment, in order to reduce unnecessary disc expansion operations, the apparatus may further include:
the detection module is used for detecting whether a second disk with the surplus of storage space exists in a first number of disks of the distributed storage system, and if not, triggering the data hash value calculation module 11 to calculate the data hash value of the data to be stored, or triggering other subsequent modules to execute the functions of the modules to realize disk expansion;
and the data writing module is used for writing the equipment to be stored into the corresponding second disk according to the size of the storage space of the second disk under the condition that the detection result of the detection module is yes.
The first mapping module 12 is configured to map the data to be stored to a hash ring according to the data hash value of the data to be stored, and obtain a data hash value of target data in the hash ring;
the hash ring is mapped with a first number of disks and each data in the disks, the target data comprises data mapped to a first disk, and the first disk is a disk in the first number of disks and provided with a disk hash value matched with the data hash value of the data to be stored;
a capacity expansion disk determining module 13, configured to determine a disk hash value mapped by the capacity expansion disk in the hash ring by using the data hash value of the target data and the data hash value of the data to be stored;
optionally, as shown in fig. 9, the capacity expansion disk determining module 13 may include:
a first calculating unit 131, configured to calculate, by using the data hash value of the target data and the data hash value of the data to be stored, a hash expected value of the data mapped to the first disk;
a first mapping unit 132, configured to use the expected hash value as a disk hash value of a capacity expansion disk, and map the capacity expansion disk into the hash ring.
And the data migration module 14 is configured to determine migration data in the target data according to the disk hash value of the expansion disk and the data hash value of the target data, and map the migration data and the data to be stored to the expansion disk.
Optionally, referring to fig. 9, the data migration module 14 may include:
a fixed data determining unit 141, configured to determine, in the target data, fixed data whose data hash value is between the disk hash value of the first disk and the disk hash value of the capacity expansion disk;
a migration data determining unit 142, configured to use, as migration data, target data of the target data except the fixed data, and clear a mapping relationship between the migration data and the first disk.
In summary, the disk capacity expansion method provided in this embodiment greatly reduces the migration data volume and the number of disks affected by increased capacity expansion disks, and ensures the balance of data access to each disk, thereby ensuring reliable and efficient operation of services.
Optionally, as shown in fig. 10, the apparatus may further include:
the second mapping module 15 is configured to map, according to a random hash algorithm, data stored in the distributed storage system to a hash ring, where the hash ring includes a data hash value of the data;
a third mapping module 16, configured to map a first number of disks in the distributed storage system to the hash ring on average, and determine a disk hash value of each disk;
in practical applications of this embodiment, the third mapping module 16 may include:
the first acquisition unit is used for acquiring the binary codes of the first number of disks in the distributed storage system;
the first number of binary codes is a continuous binary number, and the specific numerical value of the first number is not limited in the application.
A disk hash value obtaining unit, configured to perform zero padding on the binary number, and use the obtained 32-bit binary number as a disk hash value of a corresponding disk;
a first mapping unit, configured to map the first number of disks into the hash ring by using the disk hash value, so that the first number of disks are evenly distributed in the hash ring.
A fourth mapping module 17, configured to map the data mapped to the hash ring to a corresponding disk according to the first rotation direction of the hash ring.
It should be noted that, regarding the calculation manner of the data hash value and the disk hash value, the construction manner of the hash ring, and the process of mapping the data and the disk into the hash ring, reference may be made to the description of the corresponding parts of the above method embodiments, and this embodiment is not described again.
Optionally, on the basis of the foregoing embodiment, as shown in fig. 11, the apparatus may further include:
a hash value range determining module 18, configured to determine, according to the first rotation direction, hash value ranges corresponding to the first number of disks by using disk hash values of the first number of disks;
based on this, the first mapping module 12 may include:
a second mapping unit 121, configured to map the data to be stored to a hash ring according to a data hash value of the data to be stored;
a first determining unit 122, configured to determine, from hash value ranges corresponding to the first number of disks, a target hash value range to which a data hash value of the data to be stored belongs;
a first obtaining unit 123, configured to take the disk corresponding to the target hash value range as a first disk, and obtain a data hash value of target data mapped to the first disk.
The embodiment of the present application further provides a storage medium, where a program is stored, and when the program is executed by a processor, the method for expanding a disk capacity includes implementing each step of the disk capacity expanding method, and a specific implementation process may refer to the description of the corresponding part of the embodiment of the method.
The embodiment of the present application provides a processor, where the processor is configured to run a program, where the program executes each step of the disk capacity expansion method when running, and a specific implementation process may refer to descriptions of corresponding parts of the method embodiment.
Referring to fig. 12, a hardware structure diagram of a computer device provided in this embodiment of the present application is a computer device, where the computer device may be a terminal device or a server, and the present application does not limit a product type of the computer device, and the computer device is suitable for a distributed storage system, and in this embodiment, the computer device may include: a plurality of disks 21, a memory 22, and a processor 23.
In this embodiment, the memory 22 and the processor 23 may communicate with each other through a communication bus, and the number of the memory 22, the processor 23 and the communication bus may be at least one.
The magnetic disk 21 is a data storage device, and is mainly used for storing data, the specific structure of the magnetic disk is not limited in the present application, the plurality of magnetic disks 21 in the present embodiment form a distributed storage structure, and the specific implementation process of the present embodiment is not described in detail.
In practical applications, the computer device may further include a communication interface, which may specifically be an interface of a wireless communication module and/or a wired communication module, such as an interface of a communication module, such as a WIFI module, a GPRS module, a GSM module, and the like.
The memory 22 is used for storing a program for implementing the above-described disk expansion method;
in the embodiment of the present application, the memory may also be used to store various intermediate data, such as hash values, generated in the disk capacity expansion process, as needed.
Optionally, the memory may store program codes for implementing the functional modules included in the virtual device, and may specifically be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk memory.
The processor 23 may be a central processing unit CPU, or an application Specific Integrated circuit asic (application Specific Integrated circuit), or one or more Integrated circuits configured to implement the embodiments of the present application, and the present application does not limit the composition of the processor 23.
In the embodiment of the present application, the processor 23 is configured to load and execute the program stored in the memory 23, where the program is mainly configured to:
acquiring data to be stored, and calculating a data hash value of the data to be stored;
mapping the data to be stored to a hash ring according to the data hash value of the data to be stored, and obtaining the data hash value of target data in the hash ring, wherein the hash ring is mapped with a first number of disks and each data in the disks, the target data comprises the data mapped to a first disk, and the first disk is a disk in the first number of disks and provided with a disk hash value matched with the data hash value of the data to be stored;
determining a disk hash value mapped by the capacity expansion disk in the hash ring by using the data hash value of the target data and the data hash value of the data to be stored;
determining migration data in the target data according to the disk hash value of the capacity expansion disk and the data hash value of the target data, and mapping the migration data and the data to be stored into the capacity expansion disk.
It should be noted that the processor executing the program may also implement other steps of the disk capacity expansion method described in the foregoing method embodiment, and the specific implementation process may refer to the description of the corresponding part of the foregoing method embodiment, which is not described herein again.
Finally, it should be noted that, in the embodiments, relational terms such as first, second and the like may be used solely to distinguish one operation, unit or module from another operation, unit or module without necessarily requiring or implying any actual such relationship or order between such units, operations or modules. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method or system that comprises the element.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device and the computer equipment disclosed by the embodiment correspond to the method disclosed by the embodiment, so that the description is relatively simple, and the relevant points can be referred to the method part for description.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (9)
1. A disk capacity expansion method comprises
Acquiring data to be stored, and calculating a data hash value of the data to be stored;
mapping the data to be stored to a hash ring according to the data hash value of the data to be stored, and obtaining the data hash value of target data in the hash ring, wherein the hash ring is mapped with a first number of disks and each data in the disks, the target data comprises the data mapped to a first disk, and the first disk is a disk in the first number of disks and provided with a disk hash value matched with the data hash value of the data to be stored;
determining the disk hash value of the expansion disk mapped on the hash ring by using the data hash value of the target data and the data hash value of the data to be stored, including: calculating by using the data hash value of the target data and the data hash value of the data to be stored by adopting a mean algorithm to obtain a hash expected value of the data mapped to the first disk; taking the hash expected value as a disk hash value of a capacity expansion disk, mapping the capacity expansion disk into the hash ring to determine the position of the capacity expansion disk, and ensuring the load balance of each disk in the system after the capacity expansion disk is expanded;
determining migration data in the target data according to the disk hash value of the capacity expansion disk and the data hash value of the target data, and mapping the migration data and the data to be stored into the capacity expansion disk.
2. The method of claim 1, further comprising:
according to a random hash algorithm, mapping data stored in a distributed storage system to a hash ring, wherein the hash ring comprises a data hash value of the data;
averagely mapping a first number of disks in the distributed storage system to the hash ring, and determining the disk hash value of each disk;
and mapping the data mapped to the hash ring to a corresponding disk according to the first rotation direction of the hash ring.
3. The method of claim 2, wherein mapping the first number of disks in the distributed storage system to the hash ring on average and determining the disk hash value for each disk comprises:
acquiring respective binary codes of a first number of disks in the distributed storage system, wherein the first number of binary codes are continuous binary numbers;
zero padding is carried out on the binary number, and the obtained 32-bit binary number is used as a disk hash value of a corresponding disk;
mapping the first number of disks into the hash ring using the disk hash value, so that the first number of disks are evenly distributed in the hash ring.
4. The method of claim 2, further comprising:
determining the hash value ranges corresponding to the first number of disks according to the first rotation direction by using the respective disk hash values of the first number of disks;
the obtaining of the data hash value of the target data in the hash ring includes:
determining a target hash value range to which the data hash value of the data to be stored belongs from hash value ranges corresponding to the first number of disks respectively;
and taking the disk corresponding to the target hash value range as a first disk, and acquiring a data hash value of target data mapped to the first disk.
5. The method of any of claims 2-4, after obtaining the data to be stored, the method further comprising:
detecting whether a second disk with a storage space allowance exists in a first number of disks of the distributed storage system;
if so, writing the data to be stored into the corresponding second disk according to the size of the storage space of the second disk;
and if not, executing the step of calculating the data hash value of the data to be stored.
6. The method according to any one of claims 1 to 4, wherein the determining migration data in the target data according to the disk hash value of the flash disk and the data hash value of the target data includes:
determining fixed data of a data hash value between the disk hash value of the first disk and the disk hash value of the capacity expansion disk in the target data;
and taking the target data except the fixed data in the target data as migration data, and clearing the mapping relation between the migration data and the first disk.
7. A disk capacity expansion apparatus, the apparatus comprising:
the data hash value calculation module is used for acquiring data to be stored and calculating the data hash value of the data to be stored;
the first mapping module is used for mapping the data to be stored to a hash ring according to the data hash value of the data to be stored, and acquiring the data hash value of target data in the hash ring, wherein the hash ring is mapped with a first number of disks and each data in the disks, the target data comprises the data mapped to a first disk, and the first disk is a disk in the first number of disks and has a disk hash value matched with the data hash value of the data to be stored;
a capacity expansion disk determining module, configured to determine, by using the data hash value of the target data and the data hash value of the data to be stored, a disk hash value of a capacity expansion disk mapped in the hash ring, where the capacity expansion disk determining module includes: calculating by using the data hash value of the target data and the data hash value of the data to be stored by adopting a mean algorithm to obtain a hash expected value of the data mapped to the first disk; taking the hash expected value as a disk hash value of a capacity expansion disk, mapping the capacity expansion disk into the hash ring to determine the position of the capacity expansion disk, and ensuring the load balance of each disk in the system after the capacity expansion disk is expanded;
and the data migration module is used for determining migration data in the target data according to the disk hash value of the capacity expansion disk and the data hash value of the target data, and mapping the migration data and the data to be stored into the capacity expansion disk.
8. The apparatus of claim 7, further comprising:
a hash value range determining module, configured to determine, according to a first rotation direction of the hash ring, hash value ranges corresponding to the first number of disks by using disk hash values of the first number of disks;
the first mapping module comprises:
a first determining unit, configured to determine, from hash value ranges corresponding to the first number of disks, a target hash value range to which a data hash value of the data to be stored belongs;
and the first acquisition unit is used for taking the disk corresponding to the target hash value range as a first disk and acquiring the data hash value of the target data mapped to the first disk.
9. A computer device, comprising:
a plurality of disks for storing data;
a memory for storing a program for implementing the disk capacity expansion method according to any one of claims 1 to 6;
a processor for loading and executing the program stored in the memory to implement the steps of the disk capacity expansion method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910350207.4A CN110083312B (en) | 2019-04-28 | 2019-04-28 | Disk capacity expansion method and device and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910350207.4A CN110083312B (en) | 2019-04-28 | 2019-04-28 | Disk capacity expansion method and device and computer equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110083312A CN110083312A (en) | 2019-08-02 |
| CN110083312B true CN110083312B (en) | 2021-01-15 |
Family
ID=67417382
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910350207.4A Active CN110083312B (en) | 2019-04-28 | 2019-04-28 | Disk capacity expansion method and device and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110083312B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111680019B (en) * | 2020-04-29 | 2023-11-24 | 杭州趣链科技有限公司 | Block chain data expansion method and device |
| CN111563070A (en) * | 2020-05-15 | 2020-08-21 | 苏州浪潮智能科技有限公司 | Method and device for storing result of Hash algorithm |
| CN111736763B (en) * | 2020-05-25 | 2022-05-13 | 瑞芯微电子股份有限公司 | Storage data reading control method and device |
| CN111756828B (en) * | 2020-06-19 | 2023-07-14 | 广东浪潮大数据研究有限公司 | Data storage method, device and equipment |
| CN111930716A (en) * | 2020-07-31 | 2020-11-13 | 中国工商银行股份有限公司 | Database capacity expansion method, device and system |
| CN112463050A (en) * | 2020-11-26 | 2021-03-09 | 新华三技术有限公司成都分公司 | Storage system capacity expansion method, device, equipment and machine-readable storage medium |
| CN113904999B (en) * | 2021-10-29 | 2023-08-08 | 北京知道创宇信息技术股份有限公司 | Data expansion method and programmable switch |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107391040A (en) * | 2017-07-28 | 2017-11-24 | 郑州云海信息技术有限公司 | A kind of method and device of storage array disk I O scheduling |
| CN108600321A (en) * | 2018-03-26 | 2018-09-28 | 中国科学院计算技术研究所 | A kind of diagram data storage method and system based on distributed memory cloud |
| CN108810041A (en) * | 2017-04-27 | 2018-11-13 | 华为技术有限公司 | A kind of data write-in of distributed cache system and expansion method, device |
| CN109271391A (en) * | 2018-09-29 | 2019-01-25 | 武汉极意网络科技有限公司 | Date storage method, server, storage medium and device |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9246810B2 (en) * | 2011-03-11 | 2016-01-26 | Broadcom Corporation | Hash-based load balancing with per-hop seeding |
| EP3376395A1 (en) * | 2012-12-27 | 2018-09-19 | Huawei Technologies Co., Ltd. | Partition extension method and apparatus |
| CN109617986B (en) * | 2018-12-27 | 2020-08-07 | 华为技术有限公司 | Load balancing method and network equipment |
-
2019
- 2019-04-28 CN CN201910350207.4A patent/CN110083312B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108810041A (en) * | 2017-04-27 | 2018-11-13 | 华为技术有限公司 | A kind of data write-in of distributed cache system and expansion method, device |
| CN107391040A (en) * | 2017-07-28 | 2017-11-24 | 郑州云海信息技术有限公司 | A kind of method and device of storage array disk I O scheduling |
| CN108600321A (en) * | 2018-03-26 | 2018-09-28 | 中国科学院计算技术研究所 | A kind of diagram data storage method and system based on distributed memory cloud |
| CN109271391A (en) * | 2018-09-29 | 2019-01-25 | 武汉极意网络科技有限公司 | Date storage method, server, storage medium and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110083312A (en) | 2019-08-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110083312B (en) | Disk capacity expansion method and device and computer equipment | |
| US9921773B2 (en) | Range-based data deduplication using a hash table with entries replaced based on address alignment information | |
| KR100873943B1 (en) | System and method of reading non-volatile computer memory | |
| CN111125447A (en) | Metadata access method, device and equipment and readable storage medium | |
| CN111309258B (en) | B + tree access method and device and computer readable storage medium | |
| CN110032470B (en) | Method for constructing heterogeneous partial repeat codes based on Huffman tree | |
| CN112099725A (en) | Data processing method and device and computer readable storage medium | |
| US20160285476A1 (en) | Method for encoding and decoding of data based on binary reed-solomon codes | |
| US20220035546A1 (en) | Base and compressed difference data deduplication | |
| CN112363958A (en) | Cache replacement method and device | |
| CN110109867B (en) | Method, apparatus and computer program product for improving online mode detection | |
| CN117075821B (en) | Distributed storage method and device, electronic equipment and storage medium | |
| US10564848B2 (en) | Information storage device and method for deduplication | |
| US10997040B1 (en) | System and method for weight based data protection | |
| JP7355616B2 (en) | Distributed storage systems and how to update parity in distributed storage systems | |
| CN115803814A (en) | Storage medium reading method and related equipment | |
| CN111857603A (en) | Data processing method and related device | |
| CN111880745A (en) | Data processing method, device and equipment based on solid-state disk array and storage medium | |
| CN107943415B (en) | Method and system for searching free cluster based on FAT file system | |
| CN110658994A (en) | Data processing method and device based on HDD (hard disk drive) and SSD (solid State disk) hybrid disk array | |
| CN114003174A (en) | Erasure code storage method and system and electronic equipment | |
| CN114791912A (en) | Data processing method, system, electronic equipment and storage medium | |
| CN112394873B (en) | Data management method, system, electronic equipment and storage medium | |
| JP6694141B2 (en) | Array control program, array control method, array control device | |
| JP2010191903A (en) | Distributed file system striping class selecting method and distributed file system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |