CN110096227A - Date storage method, data processing method, device, electronic equipment and computer-readable medium - Google Patents
Date storage method, data processing method, device, electronic equipment and computer-readable medium Download PDFInfo
- Publication number
- CN110096227A CN110096227A CN201910245119.8A CN201910245119A CN110096227A CN 110096227 A CN110096227 A CN 110096227A CN 201910245119 A CN201910245119 A CN 201910245119A CN 110096227 A CN110096227 A CN 110096227A
- Authority
- CN
- China
- Prior art keywords
- disk
- data
- dummy node
- target
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
- G06F16/9014—Indexing; Data structures therefor; Storage structures hash tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0662—Virtualisation aspects
- G06F3/0665—Virtualisation aspects at area level, e.g. provisioning of virtual or logical volumes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0662—Virtualisation aspects
- G06F3/0667—Virtualisation aspects at data level, e.g. file, record or object virtualisation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The embodiment of the present application discloses date storage method, data processing method, device, electronic equipment and computer-readable medium.The embodiment of the method includes: according to consistency hash algorithm, and respectively each disk distributes the dummy node of preset quantity, and each dummy node distributed and data to be stored are respectively mapped in Hash ring；Mapping position based on the data in the Hash ring determines at least two destination virtual nodes in the Hash ring；The data are stored respectively into the corresponding disk of each destination virtual node.The embodiment helps to increase the capacity of logical volume.
Description
Technical field
The invention relates to field of computer technology, and in particular to date storage method, data processing method, dress
It sets, electronic equipment and computer-readable medium.
Background technique
Data storage, is the process recorded data on computer-internal or exterior storage medium.In electronic equipment sheet
When ground carries out data storage, usually store data in the disk that electronic equipment is installed.Since independent disk storage is held
It measures limited, it usually needs organize together the independent disk of muti-piece to form logical volume, to improve storage performance and provide number
According to backup functionality.
Existing mode usually utilizes RAID (Redundant Array of Independent Disks, disk battle array
Column) muti-piece disk combination is logical volume (Logical Volume) by technology, and then data are divided into multiple data blocks (Block)
It is written in parallel to multiple disks.However, when some disk failure, data in disk to calculate damage, this mode is needed
Read the full dose data in other disks.Due to the limitation of the factors such as read and write rate, disk size is bigger, and disk is each when rebuilding
The performance of disk is lower.Therefore, in the case where guaranteeing disk performance, the logical volume formed by means of which, capacity is usual
By larger limitation.
Summary of the invention
The embodiment of the present application proposes date storage method, data processing method, device, electronic equipment and computer-readable
Medium, the technical issues of to solve in the prior art while guaranteeing disk performance, the logical volume of larger capacity can not be created.
In a first aspect, the embodiment of the present application provides a kind of date storage method, this method comprises: according to consistency Hash
Algorithm, respectively each disk distribute one or more dummy nodes, and by each dummy node distributed and data to be stored
It is respectively mapped in Hash ring, wherein in Hash ring, the dummy node of the same disk is adjacent；Based on data in Hash ring
In mapping position, determine at least two destination virtual nodes in Hash ring, wherein the corresponding disk of each destination virtual node
It is different；Data are stored respectively into the corresponding disk of each destination virtual node.
In some embodiments, the mapping position based on data in Hash ring determines at least two mesh in Hash ring
Mark dummy node, comprising: using mapping position of the data in Hash ring as target position, from target position, according to presetting
Search direction, first dummy node nearest with target position is searched in Hash ring；By the first dummy node in Hash ring
In mapping position as the initial position searched, according to search direction, at least one second virtual section is searched in Hash ring
Point, wherein the first dummy node and each second dummy node respectively correspond different disks；First dummy node and second is empty
Quasi- node is determined as destination virtual node.
In some embodiments, the quantity of the dummy node distributed by each disk is equal to preset quantity；And by first
Mapping position of the dummy node in Hash ring as search initial position, according to search direction, searched in Hash ring to
Few second dummy node, comprising: be based on preset quantity, determine target interval number；By the first dummy node in Hash ring
Mapping position as search initial position successively searched in Hash ring according to search direction, and according to target interval number
The dummy node found is determined as the second dummy node by dummy node, wherein the number for searching the second dummy node is equal to
Preset backup quantity of data.
In some embodiments, after storing data respectively into the corresponding disk of each destination virtual node, the party
Method further include: using the corresponding disk of the first dummy node as the master disk for storing above-mentioned data, by each second dummy node pair
The disk answered is identified as backup disk corresponding with master disk.
In some embodiments, this method further include: in response to receiving the inquiry request to target data, according to consistent
Property hash algorithm, determines target mapping position of the target data in Hash ring；Based on target mapping position, determine in Hash ring
Dummy node to be checked；Target data is inquired from the corresponding disk of dummy node to be checked.
In some embodiments, dummy node to be checked includes the first dummy node to be checked and at least one is second to be checked
Ask dummy node；And target data is inquired from the corresponding disk of dummy node to be checked, comprising: by the first void to be checked
Master disk of the quasi- corresponding disk of node as storage target data, inquires number of targets from the master disk of storage target data
According to；Target data is not inquired in response to determination, using the corresponding disk of the second dummy node to be checked as storing target data
Backup disk, from storage target data backup disk in inquire target data.
Second aspect, the embodiment of the present application provide a kind of data storage device, which includes: map unit, are matched
It is set to according to consistency hash algorithm, respectively each disk distributes one or more dummy nodes, and each virtual by what is distributed
Node and data to be stored are respectively mapped in Hash ring, wherein in Hash ring, the dummy node phase of the same disk
It is adjacent；First determination unit is configured to the mapping position based on data in Hash ring, determines at least two mesh in Hash ring
Mark dummy node, wherein the corresponding disk of each destination virtual node is different；Storage unit is configured to respectively store data
Into the corresponding disk of each destination virtual node.
In some embodiments, the first determination unit, comprising: the first searching module is configured to data in Hash ring
In mapping position as target position, from target position, according to preset search direction, searched in Hash ring and target
The first nearest dummy node of position；Second searching module is configured to the mapped bits by the first dummy node in Hash ring
The initial position as lookup is set, according to search direction, at least one second dummy node is searched in Hash ring, wherein the
One dummy node and each second dummy node respectively correspond different disks；Determining module is configured to the first dummy node
It is determined as destination virtual node with the second dummy node.
In some embodiments, the quantity of the dummy node distributed by each disk is equal to preset quantity；And second look into
Module is looked for, is further configured to: based on preset quantity, determining target interval number；By the first dummy node in Hash ring
Mapping position is as the initial position searched, and according to search direction, and according to target interval number, successively searches in Hash ring empty
Quasi- node, is determined as the second dummy node for the dummy node found, wherein the number for searching the second dummy node is equal in advance
If backup quantity of data.
In some embodiments, device further include: the second determination unit is configured to the first dummy node is corresponding
The corresponding disk of each second dummy node is identified as corresponding with master disk by disk as the master disk for storing above-mentioned data
Backup disk.
In some embodiments, device further include: third determination unit is configured in response to receive to number of targets
According to inquiry request target mapping position of the target data in Hash ring is determined according to consistency hash algorithm；4th determines
Unit is configured to determine the dummy node to be checked in Hash ring based on target mapping position；Query unit is configured to
Target data is inquired from the corresponding disk of dummy node to be checked.
In some embodiments, dummy node to be checked includes the first dummy node to be checked and at least one is second to be checked
Ask dummy node；And query unit, comprising: the first enquiry module is configured to the first dummy node to be checked is corresponding
Master disk of the disk as storage target data, inquires target data from the master disk of storage target data；Second inquiry mould
Block is configured in response to determination and does not inquire target data, using the corresponding disk of the second dummy node to be checked as storing
The backup disk of target data inquires target data from the backup disk of storage target data.
The third aspect, the embodiment of the present application provide a kind of data processing method, this method comprises: in response to detecting magnetic
Disk replacement, using the disk being replaced as target master disk and target backup disk, by the corresponding backup of target master disk
Disk is determined as the first disk to be copied, and master disk corresponding with target backup disk is determined as the second disk to be copied,
In, each disk carries out data storage using the method as described in the embodiment in above-mentioned first aspect；By the first magnetic to be copied
Data in disk and the second disk to be copied are replicated in the new disk replaced
Fourth aspect, the embodiment of the present application provide a kind of data processing equipment, which comprises determining that unit, are matched
It is set in response to detecting that disk is replaced, using the disk being replaced as target master disk and target backup disk, by mesh
The corresponding backup disk of mark master disk is determined as the first disk to be copied, and master disk corresponding with target backup disk is determined as
Second disk to be copied, wherein each disk carries out data using the method as described in the embodiment in above-mentioned first aspect and deposits
Storage；Copied cells, be configured to for the data in the first disk to be copied and the second disk to be copied being replicated in replaced it is new
In disk.
5th aspect, the embodiment of the present application provide a kind of electronic equipment, comprising: one or more processors；Storage dress
Set, be stored thereon with one or more programs, when one or more programs are executed by one or more processors so that one or
Multiple processors realize the method such as any embodiment in above-mentioned first aspect and the third aspect.
6th aspect, the embodiment of the present application provide a kind of computer-readable medium, are stored thereon with computer program, should
The method such as any embodiment in above-mentioned first aspect and the third aspect is realized when program is executed by processor.
Date storage method provided by the embodiments of the present application, data processing method, device, electronic equipment and computer-readable
Medium, first according to consistency hash algorithm, respectively each disk distributes one or more dummy nodes, to be distributed
Each dummy node and data to be stored are respectively mapped in Hash ring.Then, the mapping position based on data in Hash ring,
Determine at least two destination virtual nodes of the correspondence different disk in Hash ring.Finally, data are stored respectively to each target
In the corresponding disk of dummy node.To which it is different can to store data at least two respectively for this data storage method
Disk.Due to using consistency hash algorithm in data storage procedure, thus while data are stored to a certain disk,
Also can will the data backup at least one disk relevant to the disk.In turn, when some disk failures, it is only necessary to from
It backs up in the disk for the data for having damage disk and reads data, do not need to read the full dose data in other disks, work as magnetic as a result,
When disk quantity increases (i.e. logic Volume Space increases), disk performance will not be impacted.Therefore, patrolling for larger capacity is supported
Collect the creation of volume.
Detailed description of the invention
By reading a detailed description of non-restrictive embodiments in the light of the attached drawings below, the application's is other
Feature, objects and advantages will become more apparent upon:
Fig. 1 is the flow chart according to one embodiment of the date storage method of the application；
Fig. 2 is the schematic diagram of the Hash ring after maps virtual node；
Fig. 3 is the flow chart according to another embodiment of the date storage method of the application；
Fig. 4 is the structural schematic diagram according to one embodiment of the data storage device of the application；
Fig. 5 is the structural schematic diagram according to one embodiment of the data processing method of the application；
Fig. 6 is the structural schematic diagram according to one embodiment of the data processing equipment of the application；
Fig. 7 is adapted for the structural schematic diagram for the computer system for realizing the electronic equipment of the embodiment of the present application.
Specific embodiment
The application is described in further detail with reference to the accompanying drawings and examples.It is understood that this place is retouched
The specific embodiment stated is used only for explaining related invention, rather than the restriction to the invention.It also should be noted that in order to
Convenient for description, part relevant to related invention is illustrated only in attached drawing.
It should be noted that in the absence of conflict, the features in the embodiments and the embodiments of the present application can phase
Mutually combination.The application is described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.
Referring to FIG. 1, it illustrates the processes 100 according to one embodiment of the date storage method of the application.The number
According to storage method, comprising the following steps:
Step 101, according to consistency hash algorithm, respectively each disk distributes one or more dummy nodes, and by institute
Each dummy node and data to be stored of distribution are respectively mapped in Hash ring.
In the present embodiment, can pacify in the executing subject (such as server for storing data) of date storage method
Equipped with multiple disks.In general, disk may include floppy disk and hard disk.It is understood that the memory capacity due to hard disk is larger
And it is more suitable for data storage, thus, multiple disks herein can be multiple hard disks.
In the present embodiment, above-mentioned executing subject can be according to consistency hash algorithm, and respectively each disk distribution is default
The dummy node of quantity, and each dummy node distributed and data to be stored are respectively mapped in Hash ring.Wherein, exist
In Hash ring, the dummy node of the same disk is adjacent.Specifically, it according to consistency hash algorithm, can be executed sequentially as follows
Operation:
The first step can be constructed Hash ring (Hash ring) based on the range of the value of preset hash function.Wherein, Hash
Ring is an annular numerical space, by the value (such as integer in 0 to (2^32) -1 range) of hash function according to up time
Needle direction is arranged successively composition from small to large.Each value of hash function can characterize it in the position of the Hash ring.
Second step can be respectively the dummy node of each disk distribution preset quantity (such as 2), and by each dummy node
It maps in above-mentioned Hash ring.Herein, each dummy node can correspond to a cryptographic Hash.Virtual section is distributed for each disk
Point, as each disk distribute cryptographic Hash.Mapping position of each dummy node in Hash ring, the as Kazakhstan of the dummy node
Position of the uncommon value in above-mentioned Hash ring.It should be noted that the same disk in Hash ring can be made when distribution dummy node
Dummy node it is adjacent.In addition, for each disk distribution dummy node quantity can be it is identical.
As an example, Fig. 2 shows the schematic diagrames of the Hash ring after maps virtual node.In Hash ring shown in Fig. 2,
It rises from (i.e. cryptographic Hash be 0 position) at position 0, is respectively included virtually according to the dummy node that successively maps clockwise
Node 0, dummy node 1, dummy node 2, dummy node 3, dummy node 4, dummy node 5, dummy node 6, dummy node 7.
Wherein, dummy node 0 and dummy node 1 are the dummy nodes for disk A distribution.Dummy node 2 and dummy node 3 are for disk
The dummy node of B distribution.Dummy node 4 and dummy node 5 are the dummy nodes for disk C distribution.Dummy node 6 and virtual section
Point 7 is the dummy node for disk D distribution.It can be seen that the dummy node of the same hard disk in Hash ring is adjacent.It needs
It is bright, it is illustrated, is not intended to restrict the invention as disk for distributing two dummy nodes for each disk in figure
Distribute dummy node quantity.
Third step can use the cryptographic Hash that above-mentioned function calculates data to be stored, so that above-mentioned data be mapped
To in above-mentioned Hash ring.Herein, the mapping position of above-mentioned data is position of the cryptographic Hash of the data in above-mentioned Hash ring.
Step 102, the mapping position based on above-mentioned data in Hash ring determines that at least two targets in Hash ring are empty
Quasi- node.
In the present embodiment, above-mentioned executing subject can be based on mapping position of the above-mentioned data in Hash ring, above-mentioned
In Hash ring, at least two destination virtual nodes for respectively corresponding different disk are determined.It should be noted that destination virtual node
Quantity can be preset according to actual needs.For example, when only needing the data backup wait store one time, then it can be true
Fixed two destination virtual nodes.When needing the data backup wait store twice, then three destination virtual nodes can be determined.
When needing the data backup wait store three times, then four destination virtual nodes can be determined.And so on.
Herein, the corresponding disk of each destination virtual node is different.Specifically, above-mentioned executing subject can be in accordance with the following steps
Destination virtual node is determined one by one:
The first step can determine that first aim virtually saves based on the mapping position of data to be stored in Hash ring
The position of point.
As an example, can be from data to be stored be in the mapping position in Hash ring, according to preset search direction
(such as clockwise), first dummy node found from above-mentioned Hash ring is virtually saved as first aim
Point.
It should be noted that determine that the mode of first aim dummy node is not limited to above-mentioned example, it can also be according to it
He determines rule, is not construed as limiting herein.For example, can be dummy node as first using find second or third
A destination virtual node can also carry out empty according to other search directions (such as counter clockwise direction) or preset lookup order
The lookup of quasi- node.
Second step determines the corresponding hard disk of first aim dummy node, in the corresponding dummy node of other hard disks,
Determine other one or more destination virtual nodes.
As an example, can virtually be saved according to above-mentioned search direction (such as clockwise direction) from above-mentioned first aim
It is risen at point, successively searches dummy node.After finding a dummy node, it can be determined that the dummy node whether with have determined that
Destination virtual node correspond to identical hard disk, if so, ignoring the dummy node, continue looking into for next dummy node
It looks for.If it is not, the dummy node can be then determined as to destination virtual node.It is preset when the quantity of destination virtual node reaches
Quantity after, then can stop searching.
As another example, due to the quantity of the dummy node distributed for each disk can be it is identical.Therefore, above-mentioned to hold
Row main body can also be according to above-mentioned search direction (such as clockwise direction), from above-mentioned first aim dummy node, often
Every the dummy node of preset quantity, a destination virtual node is determined.As an example, being saved if each disk is corresponding there are two virtual
Point then can determine a destination virtual node every a dummy node.At this point, the destination virtual node determined
Just different disks is corresponded to.
As another example, the corresponding hard disk of above-mentioned first aim dummy node can be considered as the first hard disk, according to
Above-mentioned search direction (such as clockwise), from above-mentioned first aim dummy node, search successively it is adjacent at least
One hard disk.Later, a dummy node in hard disk that each time finds can be determined as destination virtual node.Herein,
The quantity for the hard disk searched is preset backup quantity.
For example, it is desired to then search an adjacent hard disk when backing up one time.When needing to back up twice, then search successively
Two adjacent hard disks.It should be noted that the neighbouring relations of hard disk, it can be based on the neighbouring relations of dummy node in Hash ring
To determine.By taking Fig. 2 as an example, since the dummy node 0 of disk A and the dummy node 7 of disk D are adjacent, and the dummy node of disk A
1 is adjacent with the dummy node 2 of disk B, it is therefore contemplated that disk A is adjacent with disk D and disk B respectively.Similarly, it is believed that
Disk B is adjacent with disk A and disk C respectively；Disk C is adjacent respectively at disk B and disk D.Further, since disk A and disk B
Adjacent, disk B is adjacent with disk C, it is therefore contemplated that disk A, disk B, disk C are successively adjacent.Similarly, disk B, disk
C, disk D is successively adjacent；Disk C, disk D, disk A are successively adjacent；Disk D, disk A, disk B are successively adjacent.
It should be noted that determine the mode of other one or more destination virtual nodes be not limited to it is above-mentioned enumerate show
Example can also use other modes to determine, be not construed as limiting herein.
Step 103, above-mentioned data are stored respectively into the corresponding disk of each destination virtual node.
In the present embodiment, after determination there emerged a destination virtual node, above-mentioned executing subject can be by above-mentioned wait store
Data stored respectively into the corresponding disk of each destination virtual node.By taking Fig. 2 as an example, however, it is determined that the two destination virtual sections gone out
Point is respectively dummy node 1 and 3, then can respectively store above-mentioned data (can be described as data M) to be stored to disk A respectively
In disk B.It herein, can be using disk A as the master disk for storing above-mentioned data M, using disk B as corresponding with the master disk
Backup disk.Data M is stored in disk A and disk B as a result,.In addition, for other data (can be described as data N),
If the two destination virtual nodes determined in the storing process of data N are respectively node 7 and node 1, can divide
Data N is not stored in disk D and disk A.At this point it is possible to using disk D as the master disk for storing above-mentioned data N, it will
Disk A is as backup disk corresponding with master disk D.Data N is stored in disk D and disk A as a result,.It can be seen that
Disk A can simultaneously as storing certain data (such as data M) master disk and for back up other data (such as
Data N) backup disk.Each disk can be used as the storage that master disk carries out a part of data as a result, meanwhile,
It can be used as the backup disk of some or multiple disks, carry out the backup of another part data.
It, thus, can be with after storing data in each disk since each destination virtual node respectively corresponds different disks
Realize that the data for treating storage store and backed up simultaneously.
It is understood that data can be stored to a disk, while when any data write-in according to corresponding
Regular backup is into other one or more disks (can be described as corresponding backup disk).Therefore, event occurs for some disk
When barrier, by replicating the data in the corresponding backup disk of the failed disk, a part of data can be carried out to the failed disk
Reconstruction.Meanwhile the failed disk may also can be as the backup disk of another disk, at this point, there is also another in failed disk
A part of data (i.e. the Backup Data of another disk).Therefore, the data in another disk can be replicated simultaneously, it can be right
The failed disk carries out the reconstruction of above-mentioned another part data.As an example, when the quantity of destination virtual node is two, when
Some disk (such as disk A) breaks down when need to carry out disk reconstruction, and a part of data in disk A are by adjacent discs B
Backup, and another part data in disk A are the Backup Datas of another adjacent discs D.Therefore, can directly by disk B and
Data in disk D are copied in new disk A, do not need the data in reading disk C.
This data storage method as a result, does not need to extract full dose number from whole disks when disk failures
According to, thus disk can be improved and rebuild speed, and can guarantee the performance of disk.When further, being rebuild due to disk
Performance is not limited by disk size, thus, it is possible to realize the creation of the logical volume of larger capacity, increases the capacity of logical volume.
The method provided by the above embodiment of the application, first according to consistency hash algorithm, respectively each disk distribution
One or more dummy nodes, so that each dummy node distributed and data to be stored to be respectively mapped in Hash ring.
Then, the mapping position based on above-mentioned data in above-mentioned Hash ring determines correspondence different disk in above-mentioned Hash ring extremely
Few two destination virtual nodes.Finally, being stored above-mentioned data respectively into the corresponding disk of above-mentioned each destination virtual node.From
And this data storage method, at least two different disks can be stored data into respectively.Due in data storage procedure
In use consistency hash algorithm, thus while data are stored to a certain disk, also can by the data backup to
In at least one relevant disk of the disk.In turn, when some disk failures, it is only necessary to have the data of damage disk from backup
Data are read in disk, do not need to read the full dose data in other disks, and when number of disks increases, (i.e. logical volume holds as a result,
Amount increases) when, disk performance will not be impacted.Therefore, the creation of the logical volume of larger capacity is supported.
With further reference to Fig. 3, it illustrates the processes 300 of another embodiment of date storage method.Data storage
The process 300 of method, comprising the following steps:
Step 301, according to consistency hash algorithm, respectively each disk distributes one or more dummy nodes, and by institute
Each dummy node and data to be stored of distribution are respectively mapped in Hash ring.
In the present embodiment, can pacify in the executing subject (such as server for storing data) of date storage method
Equipped with multiple disks.Above-mentioned executing subject can be according to consistency hash algorithm, the void of respectively each disk distribution preset quantity
Quasi- node, and each dummy node distributed and data to be stored are respectively mapped in Hash ring.Wherein, in Hash ring
In, the dummy node of the same disk is adjacent.
It should be noted that the operation of step 301 and the operation of above-mentioned steps 101 are essentially identical, details are not described herein again.
Step 302, the mapping position using data in Hash ring is as target position, from target position, according to default
Search direction, first dummy node nearest with target position is searched in Hash ring.
In the present embodiment, above-mentioned executing subject can be by mapped bits of the above-mentioned data to be stored in above-mentioned Hash ring
It sets as target position, from above-mentioned target position, according to preset search direction (such as clockwise direction), is looked into Hash ring
The dummy node nearest with target position is looked for, as the first dummy node.
By taking Fig. 2 as an example, if mapping position of the above-mentioned data to be stored in above-mentioned Hash ring is located at dummy node 0 to void
Between quasi- node 1 (herein, the position including dummy node 0 and dummy node 1), then according to being searched clockwise, with
The nearest dummy node of the mapping position is dummy node 1, then the dummy node 1 can be determined as the first dummy node.
Step 303, the mapping position using the first dummy node in Hash ring is as the initial position searched, according to lookup
At least one second dummy node is searched in direction in Hash ring.
In the present embodiment, above-mentioned executing subject can be by mapped bits of above-mentioned first dummy node in above-mentioned Hash ring
The initial position as lookup is set, according to above-mentioned search direction, at least one second dummy node is searched in above-mentioned Hash ring.
Wherein, above-mentioned first dummy node and each second dummy node respectively correspond different disks.
Herein, above-mentioned executing subject can carry out the lookup of the second dummy node in various manners.As an example, can be with
Dummy node is successively searched from above-mentioned first dummy node according to above-mentioned search direction.When finding a dummy node
Afterwards, it can be determined that the dummy node identical hard disk whether corresponding with the first dummy node or fixed second dummy node,
If so, ignoring the dummy node, continue the lookup of next dummy node.If it is not, then can be true by the dummy node
It is set to the second dummy node.After the quantity of the second dummy node reaches preset backup quantity of data, then it can stop
It searches.
In some optional implementations of the present embodiment, the quantity of the dummy node distributed by each disk is equal to pre-
If quantity (such as 2).Above-mentioned executing subject can be primarily based on above-mentioned preset quantity, determine target interval number.It is above-mentioned in practice
Target interval number can be equal to the difference of above-mentioned preset quantity with 1.For example, can determine target interval if preset quantity is 2
Number is 1.I.e. every a dummy node, a destination virtual node is determined.At this point, each destination virtual node necessarily corresponds to not
Same hard disk.
At this point, above-mentioned executing subject can be using mapping position of above-mentioned first dummy node in above-mentioned Hash ring as looking into
It is successively searched in above-mentioned Hash ring virtual according to above-mentioned search direction, and according to above-mentioned target interval number the initial position looked for
The dummy node found is determined as the second dummy node by node.I.e. every the dummy node of above-mentioned target interval number, determine
Second dummy node out.Wherein, the number for searching the second dummy node is equal to preset backup quantity of data.
Step 304, above-mentioned data are stored respectively into the corresponding disk of each destination virtual node.
In the present embodiment, after determination there emerged a destination virtual node, above-mentioned executing subject can be by above-mentioned wait store
Data stored respectively into the corresponding disk of each destination virtual node.By taking Fig. 2 as an example, two destination virtual nodes determining
Respectively dummy node 1 and 3 then can respectively store above-mentioned data to be stored into disk A and disk B.
It, thus, can be with after storing data in each disk since each destination virtual node respectively corresponds different disks
Realize that the data for treating storage store and backed up simultaneously.
Step 305, virtual by each second using the corresponding disk of the first dummy node as the master disk for storing above-mentioned data
The corresponding disk of node is identified as backup disk corresponding with master disk.
In the present embodiment, above-mentioned executing subject can be using the corresponding disk of the first dummy node as the above-mentioned data of storage
Master disk, the corresponding disk of each second dummy node is identified as backup disk corresponding with the master disk.
It is understood that data can be stored to a corresponding master disk, be deposited simultaneously when any data write-in
Storage is into the corresponding backup disk of one or more master disks.Therefore, each disk can simultaneously as master disk and
Some other or multiple disks backup disk.
It is understood that since each disk can be simultaneously as master disk and some other or multiple disks
Backup disk thus for each disk, the data in the disk can be considered as and be made of two parts data.Wherein,
A part of data be the disk as master disk when the data that are stored；Another part data be the disk as backup disk when
The data stored.When some disk failures, by replicating the data in the corresponding backup disk of the failed disk,
The reconstruction of a part of data can be carried out to the failed disk.Meanwhile the failed disk may also can be as the standby of other disks
Part disk, at this point, there is also another part data (i.e. the Backup Datas of other disks) in failed disk.Therefore, Ke Yitong
When replicate data in other disks, the reconstruction of above-mentioned another part data can be carried out to the failed disk.As an example,
When the quantity of destination virtual node is two, when some disk (such as disk A), which breaks down, to carry out disk, disk A
In master data a (data stored using disk A as master disk) backed up by adjacent discs B, and in disk A backup have
The master data d (data stored using disk D as master disk) of another adjacent discs D.It therefore, can be directly by disk B
It is copied to the data in disk D in new disk A, does not need the data in reading disk C.
Step 306, in response to receiving the inquiry request to target data, according to consistency hash algorithm, target is determined
Target mapping position of the data in Hash ring.
In the present embodiment, in response to receiving the inquiry request to target data (i.e. current data to be checked), on
Target mapping of the above-mentioned target data in above-mentioned Hash ring can be determined according to above-mentioned consistency hash algorithm by stating executing subject
Position.Herein, the concrete operations for determining target mapping position can refer to and determine data to be stored in step 201 or step 301
Mapping position associated description, details are not described herein again.
Step 307, it is based on target mapping position, determines the dummy node to be checked in Hash ring.
In the present embodiment, above-mentioned executing subject can be based on above-mentioned target mapping position, determine in above-mentioned Hash ring
Dummy node to be checked.Herein, the operation that dummy node to be checked is determined based on target mapping position, with data storage procedure
In, determine that the operation (reference can be made to content described in step 102 or step 302-303) of destination virtual node is identical, herein
It repeats no more.
Due to determining the mode of dummy node to be checked when carrying out data query, destination virtual is determined when storing with data
The mode of node is identical, it is thus possible to guarantee that inquired disk is identical as the disk that the target data is stored, from without
It accesses each disk and carries out the inquiry of above-mentioned target data, to reduce the access times to disk.
Step 308, target data is inquired from the corresponding disk of dummy node to be checked.
In the present embodiment, any dummy node pair to be checked that above-mentioned executing subject can be determined from step 307
Target data is inquired in the disk answered.Can also mesh be inquired into the corresponding disk of each node to be checked determined respectively
Mark data.
In some optional implementations of the present embodiment, above-mentioned dummy node to be checked may include first to be checked
Dummy node and at least one second dummy node to be checked.At this point, above-mentioned executing subject can be by the above-mentioned first void to be checked
The quasi- corresponding disk of node is as the master disk for storing above-mentioned target data, the first main magnetic from the above-mentioned target data of above-mentioned storage
Above-mentioned target data is inquired in disk.By taking Fig. 2 as an example, however, it is determined that two dummy nodes to be checked gone out are respectively 1 He of dummy node
3, then know that above-mentioned target data has been stored in disk A and disk B.Since dummy node 1 is first to be checked virtual
Node, dummy node 3 is the second dummy node to be checked, so as to using disk A as the main magnetic for storing above-mentioned target data
Disk, using disk B as backup disk corresponding with the master disk.After receiving to the inquiry request of above-mentioned data, Ke Yizhi
It connects and inquires above-mentioned data from master disk A.It should be noted that determining that above-mentioned first dummy node to be checked can be with step
Determine that the mode of the first dummy node is identical, determines that the mode of the above-mentioned second dummy node to be checked can be with step 303 in 302
The mode of middle the second dummy node of determination is identical.
In above-mentioned implementation, above-mentioned target data (such as the master disk damages) is not inquired in response to determining, on
Stating executing subject can be using the corresponding disk of the above-mentioned second dummy node to be checked as the backup magnetic for storing above-mentioned target data
Disk inquires above-mentioned target data from the backup disk of the above-mentioned target data of above-mentioned storage.It continues the example presented above, as master disk A
When being damaged, then above-mentioned target data can not be read from master disk A, at this point it is possible to from the backup disk B of master disk A
In read above-mentioned target data.
From figure 3, it can be seen that compared with the corresponding embodiment of Fig. 1, the process of the date storage method in the present embodiment
300 relate to the determination step of master disk and backup disk.The scheme of the present embodiment description as a result, when disk failures,
It does not need to extract full dose data from whole disks, thus disk can be improved and rebuild speed, and can guarantee disk
Performance.Further, performance when being rebuild due to disk is not limited by disk size, thus, it is possible to realize patrolling for larger capacity
The creation for collecting volume, increases the capacity of logical volume.In addition, the process 300 of the date storage method in the present embodiment further relates to
The operation of data query.When carrying out data query, it is first determined target mapping position of the target data in Hash ring, then
The corresponding dummy node to be checked of the target mapping position is determined, to inquire from the corresponding disk of dummy node to be checked
The target data does not need to access each disk, reduces disk access amount.
With further reference to Fig. 4, as the realization to method shown in above-mentioned each figure, this application provides a kind of storages of data to fill
The one embodiment set, the Installation practice is corresponding with embodiment of the method shown in FIG. 1, which specifically can be applied to respectively
In kind electronic equipment.
As shown in figure 4, data storage device 400 described in the present embodiment includes: map unit 401, it is configured to basis
Consistency hash algorithm, respectively each disk distribute one or more dummy nodes, and by each dummy node distributed and to
The data of storage are respectively mapped in Hash ring, wherein in above-mentioned Hash ring, the dummy node of the same disk is adjacent；The
One determination unit 402 is configured to the mapping position based on above-mentioned data in above-mentioned Hash ring, determines in above-mentioned Hash ring
At least two destination virtual nodes, wherein each destination virtual node corresponds to different disks；Storage unit 403, be configured to by
Above-mentioned data are stored respectively into the corresponding disk of above-mentioned each destination virtual node.
In some optional implementations of the present embodiment, above-mentioned first determination unit may include the first lookup mould
Block, the second searching module and determining module (not shown).Wherein, above-mentioned first searching module is configured to above-mentioned number
According to the mapping position in above-mentioned Hash ring as target position, from above-mentioned target position, according to preset search direction,
First dummy node nearest with target position is searched in above-mentioned Hash ring.Above-mentioned second searching module, being configured to will be above-mentioned
Mapping position of first dummy node in above-mentioned Hash ring is as the initial position searched, according to above-mentioned search direction, upper
It states and searches at least one second dummy node in Hash ring, wherein above-mentioned first dummy node and each second dummy node difference
Corresponding different disk.Above-mentioned determining module is configured to determine above-mentioned first dummy node and above-mentioned second dummy node
For destination virtual node.
In some optional implementations of the present embodiment, the quantity of the dummy node distributed by each disk is equal to pre-
If quantity；And above-mentioned second searching module can be further configured to: being based on above-mentioned preset quantity, determined target interval
Number；Using mapping position of above-mentioned first dummy node in above-mentioned Hash ring as the initial position searched, according to above-mentioned lookup
Direction, and according to above-mentioned target interval number, dummy node successively is searched in above-mentioned Hash ring, the dummy node found is true
It is set to the second dummy node, wherein the number for searching the second dummy node is equal to preset backup quantity of data.
In some optional implementations of the present embodiment, which can also be including third determination unit, four really
Order member and query unit (not shown).Wherein, the third determination unit, is configured in response to receive to target
The inquiry request of data determines target mapping position of the target data in Hash ring according to consistency hash algorithm；4th really
Order member is configured to determine the dummy node to be checked in Hash ring based on target mapping position；Query unit is configured
Target data is inquired at from the corresponding disk of dummy node to be checked.
In some optional implementations of the present embodiment, dummy node to be checked may include first to be checked virtual
Node and at least one second dummy node to be checked；And query unit may include: the first enquiry module, be configured to
Using the corresponding disk of the first dummy node to be checked as the master disk of storage target data, from the master disk of storage target data
Middle inquiry target data；Second enquiry module is configured in response to determination and does not inquire target data, by the second void to be checked
Backup disk of the quasi- corresponding disk of node as storage target data, inquires target from the backup disk of storage target data
Data.
In some optional implementations of the present embodiment, which can also include that the first query unit and second is looked into
Ask unit (not shown).Wherein, above-mentioned first query unit is configured in response to receive the inquiry to above-mentioned data
Request, inquires above-mentioned data from above-mentioned master disk；Second query unit (not shown).Wherein, above-mentioned to be configured to ring
Above-mentioned data should not be inquired in determination, inquire above-mentioned data from the corresponding backup disk of above-mentioned master disk.
The device provided by the above embodiment of the application, map unit 401 is according to consistency hash algorithm first, respectively
Each disk distributes one or more dummy nodes, so that each dummy node distributed and data to be stored to be respectively mapped to
In Hash ring.Then, mapping position of first determination unit 402 based on above-mentioned data in above-mentioned Hash ring, determines above-mentioned Kazakhstan
At least two destination virtual nodes of the correspondence different disk in uncommon ring.Finally, storage unit 403 stores above-mentioned data respectively
Into the corresponding disk of above-mentioned each destination virtual node.To which this data storage method can be determined according to certain rule
The disk of data storage and backup.Therefore, when some disk failures, it is only necessary in the disk for having the data of damage disk from backup
Data are read, does not need to read the full dose data in other disks, ensure that disk performance, to support patrolling for larger capacity
Collect the creation of volume.
Fig. 5 is referred to, it illustrates the processes 500 of one embodiment of data processing method provided by the present application.The number
It may comprise steps of according to processing method:
Step 501, standby using the disk being replaced as target master disk and target in response to detecting that disk is replaced
Part disk, is determined as the first disk to be copied for the corresponding backup disk of target master disk, will be corresponding with target backup disk
Master disk is determined as the second disk to be copied.
In the present embodiment, the executing subject (such as server for storing data) of data processing method is in response to inspection
Disk replacement is measured, it can be using the disk being replaced as target master disk, by the corresponding backup disk of above-mentioned target master disk
It is determined as the first disk to be copied.Meanwhile it can will be backed up with above-mentioned target using the disk being replaced as target backup disk
The corresponding master disk of disk is determined as the second disk to be copied.
It should be noted that each disk can be stored using the data as described in above-mentioned Fig. 1 embodiment or Fig. 3 embodiment
Method carries out data storage.Specific storing process may refer to the associated description of Fig. 1 embodiment or Fig. 3 embodiment, herein no longer
It repeats.
Step 502, the data in the first disk to be copied and the second disk to be copied are replicated in the new disk replaced
In.
In the present embodiment, above-mentioned executing subject can be by the above-mentioned first disk to be copied and above-mentioned second disk to be copied
In data be replicated in the new disk replaced.
It is understood that since each disk can be simultaneously as master disk and some other or multiple disks
Backup disk thus for each disk, the data in the disk can be considered as and be made of two parts data.Wherein,
A part of data be the disk as master disk when the data that are stored；Another part data be the disk as backup disk when
The data stored.When some disk failures, by replicating the data in the corresponding backup disk of the failed disk,
The reconstruction of a part of data can be carried out to the failed disk.Meanwhile the failed disk may also can be as the standby of other disks
Part disk, at this point, there is also another part data (i.e. the Backup Datas of other disks) in failed disk.Therefore, Ke Yitong
When replicate data in other disks, the reconstruction of above-mentioned another part data can be carried out to the failed disk.As an example,
When the quantity of destination virtual node is two, when some disk (such as disk A), which breaks down, to carry out disk, disk A
In master data a (data stored using disk A as master disk) backed up by adjacent discs B, and in disk A backup have
The master data d (data stored using disk D as master disk) of another adjacent discs D.It therefore, can be directly by disk B
It is copied to the data in disk D in new disk A, does not need the data in reading disk C.
The method provided by the above embodiment of the application does not need to mention from whole disks when disk failures
Full dose data are taken, thus disk can be improved and rebuild speed, and can guarantee the performance of disk.Further, due to disk
Performance when reconstruction is not limited by disk size, thus, it is possible to realize the creation of the logical volume of larger capacity, increases logical volume
Capacity.
With continued reference to Fig. 6, as the realization to method shown in above-mentioned Fig. 5, this application provides a kind of data processing equipments
One embodiment.The Installation practice is corresponding with embodiment of the method shown in fig. 5, which specifically can be applied to various
In electronic equipment.
As shown in fig. 6, character recognition device 600 described in the present embodiment comprises determining that unit 601, it is configured to respond to
In detecting disk replacement, using the disk being replaced as target master disk and target backup disk, by the target master
The corresponding backup disk of disk is determined as the first disk to be copied, and master disk corresponding with the target backup disk is determined as
Second disk to be copied；Copied cells 602, being configured to will be in the described first disk to be copied and second disk to be copied
Data be replicated in the new disk replaced
It is understood that all units recorded in the device 600 and each step phase in the method with reference to Fig. 5 description
It is corresponding.As a result, above with respect to the operation of method description, the beneficial effect of feature and generation be equally applicable to device 600 and its
In include unit, details are not described herein.
Below with reference to Fig. 7, it illustrates the computer systems 700 for the electronic equipment for being suitable for being used to realize the embodiment of the present application
Structural schematic diagram.Electronic equipment shown in Fig. 7 is only an example, function to the embodiment of the present application and should not use model
Shroud carrys out any restrictions.
As shown in fig. 7, computer system 700 includes central processing unit (CPU) 701, it can be read-only according to being stored in
Program in memory (ROM) 702 or be loaded into the program in random access storage device (RAM) 703 from storage section 708 and
Execute various movements appropriate and processing.In RAM 703, also it is stored with system 700 and operates required various programs and data.
CPU 701, ROM 702 and RAM 703 are connected with each other by bus 704.Input/output (I/O) interface 705 is also connected to always
Line 704.
I/O interface 705 is connected to lower component: the importation 706 including keyboard, mouse etc.；Including such as liquid crystal
Show the output par, c 707 of device (LCD) etc. and loudspeaker etc.；Storage section 708 including disk etc.；And including such as LAN
The communications portion 709 of the network interface card of card, modem etc..Communications portion 709 is executed via the network of such as internet
Communication process.Driver 710 is also connected to I/O interface 705 as needed.Detachable media 711, such as disk, CD, magneto-optic
Disk, semiconductor memory etc. are mounted on as needed on driver 710, in order to from the computer program root read thereon
According to needing to be mounted into storage section 708.
Particularly, in accordance with an embodiment of the present disclosure, it may be implemented as computer above with reference to the process of flow chart description
Software program.For example, embodiment of the disclosure includes a kind of computer program product comprising be carried on computer-readable medium
On computer program, which includes the program code for method shown in execution flow chart.In such reality
It applies in example, which can be downloaded and installed from network by communications portion 709, and/or from detachable media
711 are mounted.When the computer program is executed by central processing unit (CPU) 701, limited in execution the present processes
Above-mentioned function.It should be noted that computer-readable medium described herein can be computer-readable signal media or
Computer readable storage medium either the two any combination.Computer readable storage medium for example can be --- but
Be not limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor system, device or device, or any above combination.
The more specific example of computer readable storage medium can include but is not limited to: have one or more conducting wires electrical connection,
Portable computer diskette, disk, random access storage device (RAM), read-only memory (ROM), erasable type may be programmed read-only deposit
Reservoir (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic memory
Part or above-mentioned any appropriate combination.In this application, computer readable storage medium, which can be, any include or stores
The tangible medium of program, the program can be commanded execution system, device or device use or in connection.And
In the application, computer-readable signal media may include in a base band or the data as the propagation of carrier wave a part are believed
Number, wherein carrying computer-readable program code.The data-signal of this propagation can take various forms, including but not
It is limited to electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be computer
Any computer-readable medium other than readable storage medium storing program for executing, the computer-readable medium can send, propagate or transmit use
In by the use of instruction execution system, device or device or program in connection.Include on computer-readable medium
Program code can transmit with any suitable medium, including but not limited to: wireless, electric wire, optical cable, RF etc., Huo Zheshang
Any appropriate combination stated.
Flow chart and block diagram in attached drawing are illustrated according to the system of the various embodiments of the application, method and computer journey
The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation
A part of one module, program segment or code of table, a part of the module, program segment or code include one or more use
The executable instruction of the logic function as defined in realizing.It should also be noted that in some implementations as replacements, being marked in box
The function of note can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are actually
It can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it to infuse
Meaning, the combination of each box in block diagram and or flow chart and the box in block diagram and or flow chart can be with holding
The dedicated hardware based system of functions or operations as defined in row is realized, or can use specialized hardware and computer instruction
Combination realize.
Being described in unit involved in the embodiment of the present application can be realized by way of software, can also be by hard
The mode of part is realized.Described unit also can be set in the processor, for example, can be described as: a kind of processor packet
Include map unit, the first determination unit and storage unit.Wherein, the title of these units is not constituted to this under certain conditions
The restriction of unit itself.
As on the other hand, present invention also provides a kind of computer-readable medium, which be can be
Included in device described in above-described embodiment；It is also possible to individualism, and without in the supplying device.Above-mentioned calculating
Machine readable medium carries one or more program, when said one or multiple programs are executed by the device, so that should
Device: according to consistency hash algorithm, the dummy node of respectively each disk distribution preset quantity, and it is each virtual by what is distributed
Node and data to be stored are respectively mapped in Hash ring；Mapping position based on the data in the Hash ring, determining should
At least two destination virtual nodes in Hash ring；The data are stored into disk corresponding to each destination virtual node respectively
In.
Above description is only the preferred embodiment of the application and the explanation to institute's application technology principle.Those skilled in the art
Member is it should be appreciated that invention scope involved in the application, however it is not limited to technology made of the specific combination of above-mentioned technical characteristic
Scheme, while should also cover in the case where not departing from foregoing invention design, it is carried out by above-mentioned technical characteristic or its equivalent feature
Any combination and the other technical solutions formed.Such as features described above has similar function with (but being not limited to) disclosed herein
Can technical characteristic replaced mutually and the technical solution that is formed.
Claims (16)
1. a kind of date storage method characterized by comprising
According to consistency hash algorithm, respectively each disk distributes one or more dummy nodes, and each virtual by what is distributed
Node and data to be stored are respectively mapped in Hash ring, wherein in the Hash ring, the dummy node of the same disk
It is adjacent；
Mapping position in the Hash ring based on the data determines at least two destination virtual sections in the Hash ring
Point, wherein the corresponding disk of each destination virtual node is different；
The data are stored respectively into the corresponding disk of each destination virtual node.
2. date storage method according to claim 1, which is characterized in that described based on the data in the Hash ring
In mapping position, determine at least two destination virtual nodes in the Hash ring, comprising:
Using mapping position of the data in the Hash ring as target position, from the target position, according to default
Search direction, first dummy node nearest with target position is searched in the Hash ring；
Using mapping position of first dummy node in the Hash ring as the initial position searched, according to the lookup
At least one second dummy node is searched in the Hash ring in direction, wherein first dummy node and each second is virtually
Node respectively corresponds different disks；
First dummy node and second dummy node are determined as destination virtual node.
3. date storage method according to claim 2, which is characterized in that by the number for the dummy node that each disk distributes
Amount is equal to preset quantity；
The mapping position using first dummy node in the Hash ring is as the initial position searched, according to described
Search direction searches at least one second dummy node in the Hash ring, comprising:
Based on the preset quantity, target interval number is determined；
Using mapping position of first dummy node in the Hash ring as the initial position searched, according to the lookup
Direction, and according to the target interval number, dummy node successively is searched in the Hash ring, the dummy node found is true
It is set to the second dummy node, wherein the number for searching the second dummy node is equal to preset backup quantity of data.
4. date storage method according to claim 3, which is characterized in that store the data to institute respectively described
After stating in the corresponding disk of each destination virtual node, the method also includes:
It is using the corresponding disk of first dummy node as the master disk for storing the data, each second dummy node is corresponding
Disk be identified as backup disk corresponding with the master disk.
5. date storage method according to claim 4, which is characterized in that the method also includes:
In response to receiving the inquiry request to target data, according to the consistency hash algorithm, the target data is determined
Target mapping position in the Hash ring；
Based on the target mapping position, the dummy node to be checked in the Hash ring is determined；
The target data is inquired from the corresponding disk of the dummy node to be checked.
6. date storage method according to claim 5, which is characterized in that the dummy node to be checked include first to
Inquire dummy node and at least one second dummy node to be checked；And
It is described to inquire the target data from the corresponding disk of the dummy node to be checked, comprising:
Using the corresponding disk of the described first dummy node to be checked as the master disk for storing the target data, from the storage
The target data is inquired in the master disk of the target data；
Do not inquire the target data in response to determination, will the corresponding disk of the described second dummy node to be checked as storing
The backup disk of the target data inquires the target data from the backup disk of the storage target data.
7. a kind of data storage device characterized by comprising
Map unit is configured to according to consistency hash algorithm, and respectively each disk distributes one or more dummy nodes, and
Each dummy node distributed and data to be stored are respectively mapped in Hash ring, wherein same in the Hash ring
The dummy node of a disk is adjacent；
First determination unit is configured to the mapping position based on the data in the Hash ring, determines the Hash ring
In at least two destination virtual nodes, wherein the corresponding disk of each destination virtual node is different；
Storage unit is configured to respectively store the data into the corresponding disk of each destination virtual node.
8. data storage device according to claim 7, which is characterized in that first determination unit, comprising:
First searching module is configured to the mapping position using the data in the Hash ring as target position, from institute
It states target position to rise, according to preset search direction, the first virtual section nearest with target position is searched in the Hash ring
Point；
Second searching module is configured to the mapping position using first dummy node in the Hash ring as lookup
Initial position searches at least one second dummy node according to the search direction in the Hash ring, wherein described
One dummy node and each second dummy node respectively correspond different disks；
Determining module is configured to first dummy node and second dummy node being determined as destination virtual node.
9. data storage device according to claim 8, which is characterized in that by the number for the dummy node that each disk distributes
Amount is equal to preset quantity；And
Second searching module, is further configured to:
Based on the preset quantity, target interval number is determined；
Using mapping position of first dummy node in the Hash ring as the initial position searched, according to the lookup
Direction, and according to the target interval number, dummy node successively is searched in the Hash ring, the dummy node found is true
It is set to the second dummy node, wherein the number for searching the second dummy node is equal to preset backup quantity of data.
10. data storage device according to claim 9, which is characterized in that described device further include:
Second determination unit is configured to using the corresponding disk of first dummy node as the main magnetic for storing the data
The corresponding disk of each second dummy node is identified as backup disk corresponding with the master disk by disk.
11. data storage device according to claim 10, which is characterized in that described device further include:
Third determination unit is configured in response to receive the inquiry request to target data, according to the consistency Hash
Algorithm determines target mapping position of the target data in the Hash ring；
4th determination unit is configured to determine the virtual section to be checked in the Hash ring based on the target mapping position
Point；
Query unit is configured to inquire the target data from the corresponding disk of the dummy node to be checked.
12. data storage device according to claim 11, which is characterized in that the dummy node to be checked includes first
Dummy node to be checked and at least one second dummy node to be checked；And
The query unit, comprising:
First enquiry module is configured to using the corresponding disk of the described first dummy node to be checked as storing the number of targets
According to master disk, inquire the target data from the master disk of the storage target data；
Second enquiry module is configured in response to determination and does not inquire the target data, to be checked virtual by described second
The corresponding disk of node is as the backup disk for storing the target data, from the backup disk of the storage target data
The middle inquiry target data.
13. a kind of data processing method, which is characterized in that the described method includes:
In response to detecting that disk is replaced, using the disk being replaced as target master disk and target backup disk, by institute
It states the corresponding backup disk of target master disk and is determined as the first disk to be copied, it will main magnetic corresponding with the target backup disk
Disk is determined as the second disk to be copied, wherein each disk carries out data using the method as described in one of claim 1-6 and deposits
Storage；
Data in described first disk to be copied and second disk to be copied are replicated in the new disk replaced.
14. a kind of data processing equipment, which is characterized in that described device includes:
Determination unit is configured in response to detect that disk is replaced, using the disk being replaced as target master disk and
The corresponding backup disk of the target master disk is determined as the first disk to be copied by target backup disk, will be with the target
The corresponding master disk of backup disk is determined as the second disk to be copied, wherein each disk is used as described in one of claim 1-6
Method carry out data storage；
Copied cells are configured to the data in the described first disk to be copied and second disk to be copied being replicated in institute
In the new disk of replacement.
15. a kind of electronic equipment characterized by comprising
One or more processors；
Storage device is stored thereon with one or more programs,
When one or more of programs are executed by one or more of processors, so that one or more of processors are real
The now method as described in any in claim 1-6,13.
16. a kind of computer-readable medium, is stored thereon with computer program, which is characterized in that the program is executed by processor
Method of the Shi Shixian as described in any in claim 1-6,13.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910245119.8A CN110096227B (en) | 2019-03-28 | 2019-03-28 | Data storage method, data processing device, electronic equipment and computer readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910245119.8A CN110096227B (en) | 2019-03-28 | 2019-03-28 | Data storage method, data processing device, electronic equipment and computer readable medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110096227A true CN110096227A (en) | 2019-08-06 |
| CN110096227B CN110096227B (en) | 2023-04-18 |
Family
ID=67444100
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910245119.8A Active CN110096227B (en) | 2019-03-28 | 2019-03-28 | Data storage method, data processing device, electronic equipment and computer readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110096227B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110633053A (en) * | 2019-09-16 | 2019-12-31 | 北京马赫谷科技有限公司 | Storage capacity balancing method, object storage method and device |
| CN111522883A (en) * | 2020-04-28 | 2020-08-11 | 杭州海康威视系统技术有限公司 | Backup method, device and equipment of object data and storage medium |
| CN111756828A (en) * | 2020-06-19 | 2020-10-09 | 广东浪潮大数据研究有限公司 | Data storage method, device and equipment |
| CN111930316A (en) * | 2020-09-09 | 2020-11-13 | 上海七牛信息技术有限公司 | Cache read-write system and method for content distribution network |
| CN112230861A (en) * | 2020-10-26 | 2021-01-15 | 金钱猫科技股份有限公司 | Data storage method and terminal based on consistent hash algorithm |
| CN112306688A (en) * | 2020-10-30 | 2021-02-02 | 天地伟业技术有限公司 | Innovative hash consistency algorithm suitable for cloud storage |
| CN113112193A (en) * | 2020-01-13 | 2021-07-13 | 北京京东振世信息技术有限公司 | Method, apparatus, server and medium for determining package location |
| CN113672524A (en) * | 2021-08-20 | 2021-11-19 | 上海哔哩哔哩科技有限公司 | Data processing method and system based on multi-level cache |
| CN115878046A (en) * | 2023-01-09 | 2023-03-31 | 苏州浪潮智能科技有限公司 | Data processing method, system, device, storage medium and electronic equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102843403A (en) * | 2011-06-23 | 2012-12-26 | 盛大计算机（上海）有限公司 | File processing method based on distributed file system, system, and client |
| CN104636286A (en) * | 2015-02-06 | 2015-05-20 | 华为技术有限公司 | Data access method and equipment |
| US20160342588A1 (en) * | 2015-05-19 | 2016-11-24 | Morgan Stanley | Topology aware distributed storage system |
| CN106909557A (en) * | 2015-12-23 | 2017-06-30 | 中国电信股份有限公司 | The storage method and device of main memory cluster, the read method and device of main memory cluster |
| WO2017122922A1 (en) * | 2016-01-11 | 2017-07-20 | 충북대학교 산학협력단 | Load balancing system using data replication and data migration in distributed in-memory environment |
| CN108737375A (en) * | 2018-04-13 | 2018-11-02 | 中山大学 | A kind of block chain common recognition method and system |
-
2019
- 2019-03-28 CN CN201910245119.8A patent/CN110096227B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102843403A (en) * | 2011-06-23 | 2012-12-26 | 盛大计算机（上海）有限公司 | File processing method based on distributed file system, system, and client |
| CN104636286A (en) * | 2015-02-06 | 2015-05-20 | 华为技术有限公司 | Data access method and equipment |
| US20160342588A1 (en) * | 2015-05-19 | 2016-11-24 | Morgan Stanley | Topology aware distributed storage system |
| CN106909557A (en) * | 2015-12-23 | 2017-06-30 | 中国电信股份有限公司 | The storage method and device of main memory cluster, the read method and device of main memory cluster |
| WO2017122922A1 (en) * | 2016-01-11 | 2017-07-20 | 충북대학교 산학협력단 | Load balancing system using data replication and data migration in distributed in-memory environment |
| CN108737375A (en) * | 2018-04-13 | 2018-11-02 | 中山大学 | A kind of block chain common recognition method and system |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110633053A (en) * | 2019-09-16 | 2019-12-31 | 北京马赫谷科技有限公司 | Storage capacity balancing method, object storage method and device |
| CN113112193A (en) * | 2020-01-13 | 2021-07-13 | 北京京东振世信息技术有限公司 | Method, apparatus, server and medium for determining package location |
| CN111522883A (en) * | 2020-04-28 | 2020-08-11 | 杭州海康威视系统技术有限公司 | Backup method, device and equipment of object data and storage medium |
| CN111522883B (en) * | 2020-04-28 | 2023-04-28 | 杭州海康威视系统技术有限公司 | Backup method, device, equipment and storage medium of object data |
| CN111756828A (en) * | 2020-06-19 | 2020-10-09 | 广东浪潮大数据研究有限公司 | Data storage method, device and equipment |
| CN111930316A (en) * | 2020-09-09 | 2020-11-13 | 上海七牛信息技术有限公司 | Cache read-write system and method for content distribution network |
| CN112230861A (en) * | 2020-10-26 | 2021-01-15 | 金钱猫科技股份有限公司 | Data storage method and terminal based on consistent hash algorithm |
| CN112306688A (en) * | 2020-10-30 | 2021-02-02 | 天地伟业技术有限公司 | Innovative hash consistency algorithm suitable for cloud storage |
| CN113672524A (en) * | 2021-08-20 | 2021-11-19 | 上海哔哩哔哩科技有限公司 | Data processing method and system based on multi-level cache |
| WO2023020085A1 (en) * | 2021-08-20 | 2023-02-23 | 上海哔哩哔哩科技有限公司 | Data processing method and system based on multi-level cache |
| CN115878046A (en) * | 2023-01-09 | 2023-03-31 | 苏州浪潮智能科技有限公司 | Data processing method, system, device, storage medium and electronic equipment |
| CN115878046B (en) * | 2023-01-09 | 2023-05-12 | 苏州浪潮智能科技有限公司 | Data processing method, system, device, storage medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110096227B (en) | 2023-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110096227A (en) | Date storage method, data processing method, device, electronic equipment and computer-readable medium | |
| US10977124B2 (en) | Distributed storage system, data storage method, and software program | |
| CN103354923B (en) | A kind of data re-establishing method, device and system | |
| US8799601B1 (en) | Techniques for managing deduplication based on recently written extents | |
| US7979641B2 (en) | Cache arrangement for improving raid I/O operations | |
| JP5539683B2 (en) | Scalable secondary storage system and method | |
| US20170075741A1 (en) | Prioritizing Data Reconstruction in Distributed Storage Systems | |
| US9773012B2 (en) | Updating map structures in an object storage system | |
| US9514008B2 (en) | System and method for distributed processing of file volume | |
| CN105027069A (en) | Deduplication of volume regions | |
| US20130124559A1 (en) | Method and system for inserting data records into files | |
| US20200043524A1 (en) | RAID Storage System with Logical Data Group Priority | |
| CN105468473A (en) | Data migration method and data migration apparatus | |
| CN103534688B (en) | Data reconstruction method, memory device and storage system | |
| JP2001142751A (en) | Method and device for data base management and storage medium with recorded database managing program | |
| JP6805816B2 (en) | Information processing equipment, information processing system, information processing method and program | |
| US10031682B1 (en) | Methods for improved data store migrations and devices thereof | |
| KR20160081851A (en) | Computing system with distributed compute-enabled storage group and operating method thereof | |
| CN110427364A (en) | A kind of data processing method, device, electronic equipment and storage medium | |
| CN109726264A (en) | Method, apparatus, equipment and the medium updated for index information | |
| CN110187834B (en) | Data processing method and device for duplicate copies and electronic equipment | |
| CN103514222B (en) | Storage method, management method, memory management unit and the system of virtual machine image | |
| CN103348653B (en) | The method and apparatus of dilatation and the method and apparatus of visit data | |
| CN104067241B (en) | Synchronizing local and remote storage devices | |
| CN117193672A (en) | Data processing method and device of storage device, storage medium and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |