CN107728940A - The method, apparatus and storage medium of a kind of management copy in data-storage system - Google Patents
The method, apparatus and storage medium of a kind of management copy in data-storage system Download PDFInfo
- Publication number
- CN107728940A CN107728940A CN201710889792.6A CN201710889792A CN107728940A CN 107728940 A CN107728940 A CN 107728940A CN 201710889792 A CN201710889792 A CN 201710889792A CN 107728940 A CN107728940 A CN 107728940A
- Authority
- CN
- China
- Prior art keywords
- storage
- data
- data block
- file
- cluster
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0643—Management of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses the method, apparatus and storage medium of the management copy in a kind of data-storage system, and position Las of the file to be stored Fa in Chord rings is obtained by preset algorithm, and searches the storage cluster of the Fa storages；Fa described in piecemeal, and obtain the storage location of the file Fa after memory partitioning；Data storage block, and create the metadata of the data block；Focused on designed based in the cloud storage model of storage cluster equity by the metadata and routing iinformation that access high frequency on high performance storage cluster management node, to realize the access performance for preferably improving system.
Description
Technical field
The present embodiments relate to a kind of side of the management copy in computer technology, more particularly to data-storage system
Method, device and storage medium.
Background technology
To ensure the integrality of cloud storage data management, the data in general cloud storage system on server node can all protect
More parts are deposited, so that the permanent loss of the data will not be caused when ensuring that the server node for storing the data breaks down, but
It is that copy redundancy is realized by replicating, and replicating will also pay a high price, the increase of copy data can take greatly
Memory space resource in amount system, while in order to ensure that the uniformity of multiple copies can also consume substantial amounts of calculate and network money
Source.
The content of the invention
The embodiment of the present invention provides the method, apparatus and storage medium of the management copy in a kind of data-storage system, such as
While where improving the reliabilty and availability of system, also reduce the consumption for the various resources that copy redundancy is brought as far as possible,
It is the important goal of replica management research.
In a first aspect, the embodiments of the invention provide a kind of method of the management copy in data-storage system, the side
Method includes：
Position Las of the file to be stored Fa in Chord rings is obtained by preset algorithm, and searches depositing for the Fa storages
Store up cluster；
Fa described in piecemeal, and obtain the storage location of the file Fa after memory partitioning；
Data storage block, and create the metadata of the data block.
Alternatively, the storage cluster for searching the Fa storages, including：
According to the La on the Chord rings that the management node by storage cluster is formed, first management section is found clockwise
Point Mi, the Mi are the management node that the Fa is stored in the storage cluster.
Alternatively, Fa described in the piecemeal, including：
Fa orders are divided into each data block Bi for 64MB fixed sizes, and data block is numbered.
Alternatively, the storage location of the file Fa after the acquisition memory partitioning, including：
The request to the Mi data storages sent is received, the Mi stores the data block server in cluster according to place
The loading condition of node, return to the information for the data block server node for storing the Fa.
Alternatively, the data storage block, including：
Blocked file B1~BN is deposited into corresponding data block server node.
Alternatively, the metadata for creating the data block, including：
After the Fa is stored, the management node Mi for storing cluster preserves resource identification File ID, the filename of the Fa
The version number of mapping and the Fa between title, file size, supporter, data block Bi and storage location.
Alternatively, the data storage block, and after creating the metadata of the data block, in addition to：
Store redundant copy；
According to Replica Placement Strategy, selection storage cluster simultaneously stores the redundant copy；
Return to storage result.
Second aspect, a kind of device of the management copy in data-storage system, described device include：
Searching modul, for obtaining position Las of the file to be stored Fa in Chord rings by preset algorithm, and search institute
State the storage cluster of Fa storages；
Acquisition module, for Fa described in piecemeal, and obtain the storage location of the file Fa after memory partitioning；
Creation module, for data storage block, and create the metadata of the data block.
Alternatively, the searching modul, is used for：
According to the La on the Chord rings that the management node by storage cluster is formed, first management section is found clockwise
Point Mi, the Mi are the management node that the Fa is stored in the storage cluster；
The acquisition module, is used for：
Fa orders are divided into each data block Bi for 64MB fixed sizes, and data block is numbered；Connect
The request to the Mi data storages sent is received, the Mi stores the load of the data block server node in cluster according to place
Situation, return to the information for the data block server node for storing the Fa；
The creation module, is used for：
Blocked file B1~BN is deposited into corresponding data block server node, after the Fa is stored, stores cluster
Management node Mi preserves resource identification File ID, file name, file size, supporter, data block Bi and the storage of the Fa
The version number of mapping and the Fa between position；
The data storage block, and after creating the metadata of the data block, in addition to：
Store redundant copy；
According to Replica Placement Strategy, selection storage cluster simultaneously stores the redundant copy；
Return to storage result.
The third aspect, a kind of computer-readable recording medium, it is characterised in that be stored thereon with computer program, the meter
Calculation machine program realizes the management copy in above-mentioned data-storage system method when being executed by processor.
In the embodiment of the present invention, position Las of the file to be stored Fa in Chord rings is obtained by preset algorithm, and search
The storage cluster of the Fa storages；Fa described in piecemeal, and obtain the storage location of the file Fa after memory partitioning；Data storage block,
And create the metadata of the data block；In the designed cloud storage model reciprocity based on storage cluster, by the way that high frequency is visited
The metadata and routing iinformation asked are focused on high performance storage cluster management node, to realize the access for preferably improving system
Performance.
Brief description of the drawings
Fig. 1 is the structural representation of P2P in the embodiment of the present invention one a kind of；
Fig. 2 is the schematic flow sheet of the management copy in a kind of data-storage system in the embodiment of the present invention two；
Fig. 3 is the schematic flow sheet of the management copy in a kind of data-storage system in the embodiment of the present invention three；
Fig. 4 is the method schematic diagram of the management copy in a kind of data-storage system in the embodiment of the present invention three；
Fig. 5 is the structural representation of the device of the management copy in a kind of data-storage system in the embodiment of the present invention four
Figure；
Fig. 6 is a kind of structural representation of computer equipment of five kinds of the embodiment of the present invention.
Embodiment
The present invention is described in further detail with reference to the accompanying drawings and examples.It is understood that this place is retouched
The specific embodiment stated is used only for explaining the present invention, rather than limitation of the invention.It also should be noted that in order to just
Part related to the present invention rather than entire infrastructure are illustrate only in description, accompanying drawing.
Embodiment one
As shown in figure 1, the peering structure in P2P network topologies, according to the elementary cell characteristic of equity, is broadly divided into all
The complete peering structure of node and the peering structure based on storage cluster for unit.
It is all services for being used to store in cloud storage based on the complete equity in complete reciprocity cloud storage model
Device node forms a loop configuration, is all equality between each node, without hierarchical relationship.Clothes are stored in the loop configuration
It is engaged on device node, has both been responsible for storage file data and the metadata of these file datas, while store the route of correlation again
Data, its system architecture are distributed hashtable (DHT) networks, and it uses uniformity hash algorithm, by all compositions
The server apparatus of cloud storage network is mapped on a hash space by hash function, and the hash space joins end to end abstract
For ring.In based on complete reciprocity cloud storage model, each storage server for forming cloud storage system is to be mapped
For Chord rings, (Chord is most simple, most exactly circular P2P models, is the one of DHT (Distributed Hash Table)
It is kind classical to realize) on a node, each storage server is a self-government node for being capable of small range route, this
The structure of sample make it that the node that is mapped on Chord rings is excessive, is unfavorable for managing and route.Moreover, the one of cloud storage system
As in access process, user will be far longer than the visit for file content for the visiting frequency of file metadata and route data
Ask frequency.Each node safeguards the local data block of low frequency access and provides the routing function of high frequency access, node tasks simultaneously
Complexity, both are put together to the performance for being unfavorable for raising system.
It is the optimization on complete peering structure basis based on the peering structure that storage cluster is unit.In this configuration, will
It is responsible for the file metadata of high frequency access and safeguards function for routing iinformation data, concentrates depositing by higher performance
The management node for storing up cluster provides；And the data block server of common performance will play the part of the copy data for safeguarding that rather low-frequency accesses.
Two kinds of big types are divided into based on the server in storage cluster peering structure model：It is high performance storage cluster management server and
The data block storage server of common performance.In each storage cluster, the major function of its management node is user profile
The metadata such as the mapping between certification, the catalogue for safeguarding storage cluster and file and data block server, and data block server
It is to be used to store user file data.According in cloud storage system the characteristics of data access, by for the data of route, file
The hot spot datas such as metadata are concentrated, and transfer to higher performance server process, are advantageous to preferably improve the performance of system, are stored cluster
Between use structured P 2 P network topological structure, formed a DHT network.
In addition, for data block server, according to the degree of reliability, it is divided into believable data block server node again
(Reliable Chunk Server Node, abbreviation RCSN, RN) and incredible data block server node (Unreliable
Chunk Server Node, abbreviation UCSN, UN).Believable data block server node (Reliable Chunk Server
Node, abbreviation RCSN) server of high reliability that is mainly provided by cloud storage service business forms.RCSN is responsible for user in system
The storage of data, each RCSN by storage cluster management node management.Because RCSN reliabilities are high, the data being stored in RCSN
Relative reliability, availability are just higher, so being cloud storage system on RCSN the management copy in user data storage system
One of important guarantee of reliabilty and availability in system.Incredible data block server node (Unreliable Chunk
Server Node, abbreviation UCSN) be mainly user machine form.Because the general reliability of user's machine can not be guaranteed,
So assume that the data being stored in UCSN do not ensure its reliabilty and availability in cloud storage model.
Embodiment two
With reference to figure 2, the present embodiment is applicable to the situation of the management copy in data-storage system, and this method can be by this hair
The device of management copy in the data-storage system that bright embodiment provides performs, and the device can use software and/or hardware
Mode realize.This method includes：
Step 210, position L as of the file to be stored Fa in Chord rings is obtained by preset algorithm, and searches the Fa
The storage cluster of storage；
Wherein, the storage cluster for searching the Fa storages, including：
According to the La on the Chord rings that the management node by storage cluster is formed, first management section is found clockwise
Point Mi, the Mi are the management node that the Fa is stored in the storage cluster.
Step 220, Fa described in piecemeal, and obtain the storage location of the file Fa after memory partitioning；
Wherein, Fa described in the piecemeal, including：
Fa orders are divided into each data block Bi for 64MB fixed sizes, and data block is numbered.
The storage location of file Fa after the acquisition memory partitioning, including：
The request to the Mi data storages sent is received, the Mi stores the data block server in cluster according to place
The loading condition of node, return to the information for the data block server node for storing the Fa.
Step 230, data storage block, and create the metadata of the data block.
Wherein, the data storage block, including：
Blocked file B1~BN is deposited into corresponding data block server node；
The metadata for creating the data block, including：
After the Fa is stored, the management node Mi for storing cluster preserves resource identification File ID, the filename of the Fa
The version number of mapping and the Fa between title, file size, supporter, data block Bi and storage location.
Exemplary, in the cloud storage system based on storage cluster equity, user carries out the general place of file storage operations
Reason process is as follows：
(1) file for assuming to be stored in system is Fa；
(2) system gives file Fa distribution system Uique Resource Identities, then by SHA-1 algorithms acquisition Fa in Chord rings
In position La；
(3) the storage cluster that locating file Fa should be placed.According to file Fa La, formed in the management node by storage cluster
On Chord rings, first management node Mi is found clockwise.Mi is the management node that file Fa should deposit storage cluster；
(4) file Fa piecemeals.File Fa orders are divided into each data block Bi for 64MB fixed sizes, and to data
Block is numbered.Assuming that file Fa is divided into N blocks, then it is B1~BN the data block numbering that segmentation is got off；
(5) specific storage location is obtained.The request of data storage is sent to Mi, Mi stores the data block in cluster according to place
The loading condition of server node, return to the data block server node that file is finally deposited；
(6) data storage block.Blocked file B1~BN is deposited into the corresponding data block server of distribution by subscription client
Node；
(7) metadata of user data is created.After user file stores successfully, storing the management node Mi of cluster needs to protect
Deposit reflecting between file Fa resource identification File ID, file name, file size, supporter, data block Bi and storage location
Penetrate and the range of information such as the version number of file；
(8) other two parts of redundant copies are stored.According to Replica Placement Strategy, selection storage cluster simultaneously stores the superfluous of user file
Remaining copy；
(9) storage result is returned.
Embodiment three
With reference to figure 3, the present embodiment is on the basis of embodiment two, in data storage block, and creates the member of the data block
After data, in addition to storage redundant copy, it is specific as follows：
Step 310, position L as of the file to be stored Fa in Chord rings is obtained by preset algorithm, and searches the Fa
The storage cluster of storage；
Step 320, Fa described in piecemeal, and obtain the storage location of the file Fa after memory partitioning；
Step 330, data storage block, and create the metadata of the data block；
Step 340, redundant copy is stored；According to Replica Placement Strategy, selection storage cluster simultaneously stores the redundant copy；Return
Restore storage result.
Exemplary, alternatively, the storage redundant copy, including：
In the Chord rings of preset algorithm formation, by the storage cluster selection of redundant copy storage to be directly preceding after storage cluster N
I-1 and immediate successor storage cluster N i+1；
By management node M i-1 and the M i+1 in storage cluster N i-1 and N i+1 according to each data block in respective storage cluster
The loading condition distribution data block server of server；
The data block for storing the user file F a stored in cluster N i is copied in N i-1 and N i+1 storage clusters and distributed
Data block server in.
Wherein, it is described directly before after storage cluster N i-1 be：So that clockwise for positive direction, each node N is arranged in it
Before or after node, wherein the node come above is referred to as the preceding after after the direct of referred to as node before first of node N
It is preceding after N i-1；
The immediate successor stores cluster N i+1：The node come behind is referred to as the follow-up of node N, and first follow-up
Referred to as node N immediate successor.
Alternatively, the user file F a stored in the cluster N i by storage data block copies to N i-1 and N i+1 and deposited
After in the data block server distributed in storage cluster, in addition to：
File name, file creation time, file File ID and version are preserved on the M i-1 and the M i+1
Number metadata.
It is exemplary, in Replica Placement Strategy, the selection of redundant copy storage storage cluster be wherein important content it
One.The advantage being stored in different storage clusters is, if some storage cluster breaks down, still has in other storage clusters
Multiple copies can use, and can recover copy amount by this multiple copies, it is ensured that reliability, the availability of system.With three parts
Exemplified by copy storage, the choice of first part of copy storage cluster position has hereinbefore been elaborated.Other two parts of redundant copies need
The different storage cluster of reselection two.The position of decision-making redundant copy storage, mainly selection stores cluster in the model, finally
Specific data block server node in deposit storage cluster is determined by the management node in the storage cluster, so emphasis is exactly such as
The storage cluster of what selection storage.
The characteristics of storage cluster selection of redundant copy storage should make full use of Chord algorithms.Formed in Chord algorithms
In Chord rings, so that clockwise for positive direction, each node N has the node before or after being arranged in it, wherein before coming
The node in face be referred to as node N it is preceding after, before first after referred to as node it is direct before after and coming node behind and be referred to as saving
Point N's is follow-up, first follow-up immediate successor for being referred to as node N.Due in Chord algorithms each node can store before it after
Node and descendant node positional information, then in order to quickly and easily find redundant copy, redundant copy can be stored in copy
Deposit node it is direct before after with immediate successor.The storage cluster selection that redundant copy is deposited in the present invention is directly preceding after storage
Cluster and immediate successor storage cluster.
Assuming that L a are calculated by SHA-1 algorithms using resource identification File ID in file F a, pass through Chord algorithms
Find out file should deposit in N i storage cluster, then file will be stored in N i storage cluster and its directly before after Ni-1 store cluster and
On immediate successor storage cluster Ni+1,.It is former copy referred to here as N i copy is stored in, claims the pair for being stored in N i-1 and N i+1
This is redundant copy, is mutually redundant between them.
As shown in figure 4, the process of storage cluster storage redundant copy is as storage copy process, it is first by storage cluster N i-1
With management node M i-1 and the M i+1 in N i+1 number is distributed according to the loading condition of each data block server in respective storage cluster
According to block server.Then the user file F a stored in storage cluster N i data block is copied to N i-1 and N i+1 storage clusters
In the data block server of middle distribution.Finally, File name, file creation time, file are preserved on M i-1 and M i+1
Metadata related to version number etc. File ID.After above-mentioned Placement Strategy, each user file can deposit three parts of pairs
This, is stored in the storage cluster positioned by Chord algorithms respectively, and the storage cluster it is direct before after storing cluster with immediate successor.
Therefore the user data of each storage cluster storage includes three types, they be it is directly preceding after redundant copy, immediate successor
Redundant copy and the former copy that be stored in the storage cluster is drawn by Chord algorithms.
Example IV
With reference to figure 5, the device is adapted for carrying out the management in a kind of data-storage system that the embodiment of the present invention is supplied to
The method of copy.The device can specifically include：
Searching modul 510, for obtaining position L as of the file to be stored Fa in Chord rings by preset algorithm, and look into
Look for the storage cluster of the Fa storages；
Wherein, the searching modul 510, is used for：
According to the La on the Chord rings that the management node by storage cluster is formed, first management section is found clockwise
Point Mi, the M i are the management node that the Fa is stored in the storage cluster.
Acquisition module 520, for Fa described in piecemeal, and obtain the storage location of the file Fa after memory partitioning；
Wherein, the acquisition module 520, is used for：
Fa orders are divided into each data block Bi for 64MB fixed sizes, and data block is numbered；Connect
The request to the Mi data storages sent is received, the Mi stores the load of the data block server node in cluster according to place
Situation, return to the information for the data block server node for storing the Fa；
Creation module 530, for data storage block, and create the metadata of the data block.
Wherein, the creation module 530, is used for：
Blocked file B1~BN is deposited into corresponding data block server node, after the Fa is stored, stores cluster
Management node Mi preserves resource identification File ID, file name, file size, supporter, data block Bi and the storage of the Fa
The version number of mapping and the Fa between position；
The data storage block, and after creating the metadata of the data block, in addition to：
Store redundant copy；
According to Replica Placement Strategy, selection storage cluster simultaneously stores the redundant copy；
Return to storage result.
Embodiment five
Fig. 6 shows the block diagram suitable for being used for the exemplary computer device 12 for realizing embodiment of the present invention.Fig. 6 is shown
Computer equipment 12 be only an example, any restrictions should not be brought to the function and use range of the embodiment of the present invention.
As shown in fig. 6, computer equipment 12 is showed in the form of universal computing device.The component of computer equipment 12 can be with
Including but not limited to：One or more processor or processing unit 16, system storage 28, connect different system component
The bus 18 of (including system storage 28 and processing unit 16).
Bus 18 represents the one or more in a few class bus structures, including memory bus or Memory Controller,
Peripheral bus, graphics acceleration port, processor or the local bus using any bus structures in a variety of bus structures.Lift
For example, these architectures include but is not limited to industry standard architecture (ISA) bus, MCA (MAC)
Bus, enhanced isa bus, VESA's (VESA) local bus and periphery component interconnection (PCI) bus.
Computer equipment 12 typically comprises various computing systems computer-readable recording medium.These media can be it is any can be by
The usable medium that computer equipment 12 accesses, including volatibility and non-volatile media, moveable and immovable medium.
System storage 28 can include the computer system readable media of form of volatile memory, such as arbitrary access
Memory (RAM) 30 and/or cache memory 32.Computer equipment 12 may further include it is other it is removable/can not
Mobile, volatile/non-volatile computer system storage medium.Only as an example, storage system 34 can be used for read-write not
Movably, non-volatile magnetic media (Fig. 6 is not shown, is commonly referred to as " hard disk drive ").Although not shown in Fig. 6, can with
There is provided for the disc driver to may move non-volatile magnetic disk (such as " floppy disk ") read-write, and to removable non-volatile
The CD drive of CD (such as CD-ROM, DVD-ROM or other optical mediums) read-write.In these cases, each driving
Device can be connected by one or more data media interfaces with bus 18.Memory 28 can include at least one program and produce
Product, the program product have one group of (for example, at least one) program module, and these program modules are configured to perform of the invention each
The function of embodiment.
Program/utility 40 with one group of (at least one) program module 42, such as memory 28 can be stored in
In, such program module 42 includes --- but being not limited to --- operating system, one or more application program, other programs
Module and routine data, the realization of network environment may be included in each or certain combination in these examples.Program mould
Block 42 generally performs function and/or method in embodiment described in the invention.
Computer equipment 12 can also be with one or more external equipments 14 (such as keyboard, sensing equipment, display 24
Deng) communication, the equipment communication interacted with the computer equipment 12 can be also enabled a user to one or more, and/or with making
Obtain any equipment that the computer equipment 12 can be communicated with one or more of the other computing device (such as network interface card, modulatedemodulate
Adjust device etc.) communication.This communication can be carried out by input/output (I/O) interface 22.Also, computer equipment 12 may be used also
To pass through network adapter 20 and one or more network (such as LAN (LAN), wide area network (WAN) and/or public network
Network, such as internet) communication.As illustrated, network adapter 20 is led to by bus 18 and other modules of computer equipment 12
Letter.It should be understood that although not shown in Fig. 6, computer equipment 12 can be combined and use other hardware and/or software module, bag
Include but be not limited to：Microcode, device driver, redundant processing unit, external disk drive array, RAID system, magnetic tape drive
Device and data backup storage system etc..
Processing unit 16 is stored in program in system storage 28 by operation, so as to perform various function application and
Data processing, such as the method for realizing a kind of data distribution that the embodiment of the present invention is provided：
That is, the processing unit is realized when performing described program：File to be stored Fa is obtained by preset algorithm to exist
Position L a in Chord rings, and search the storage cluster of the Fa storages；Fa described in piecemeal, and obtain the file after memory partitioning
Fa storage location；Data storage block, and create the metadata of the data block.
Embodiment six
The embodiment of the present invention seven provides a kind of computer-readable recording medium, is stored thereon with computer program, the journey
A kind of resource allocation methods provided such as all inventive embodiments of the application are provided when sequence is executed by processor：
That is, the program is realized when being executed by processor：The resource allocation request that Target cell is sent is received, and extracts institute
State request resource type and request resource size that resource allocation request includes；According to the physical area of the Target cell
Mark, cell portion number, System Frame Number and slot index, calculate randomization factor corresponding to the Target cell；According to
The randomization factor, the cell portion number, the request resource type and system bandwidth, determine the Target cell
The starting resource block position of scheduling of resource；According to the starting resource block position and the request resource size, resource is determined
Index set, and search target resource in the resource index set and distribute to the Target cell.
Any combination of one or more computer-readable media can be used.Computer-readable medium can be calculated
Machine readable signal medium or computer-readable recording medium.Computer-readable recording medium for example can be --- but it is unlimited
In system, device or the device of --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor, or it is any more than combination.Calculate
The more specifically example (non exhaustive list) of machine readable storage medium storing program for executing includes：Electrical connection with one or more wires, just
Take formula computer disk, hard disk, random access memory (RAM), read-only storage (ROM), erasable type and may be programmed read-only storage
Device (EPROM or flash memory), optical fiber, portable compact disc read-only storage (CD-ROM), light storage device, magnetic memory device,
Or above-mentioned any appropriate combination.In this document, computer-readable recording medium can any include or store journey
The tangible medium of sequence, the program can be commanded the either device use or in connection of execution system, device.
Computer-readable signal media can include in a base band or as carrier wave a part propagation data-signal,
Wherein carry computer-readable program code.The data-signal of this propagation can take various forms, including --- but
It is not limited to --- electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be
Any computer-readable medium beyond computer-readable recording medium, the computer-readable medium can send, propagate or
Transmit for by instruction execution system, device either device use or program in connection.
The program code included on computer-readable medium can be transmitted with any appropriate medium, including --- but it is unlimited
In --- wireless, electric wire, optical cable, RF etc., or above-mentioned any appropriate combination.
It can be write with one or more programming languages or its combination for performing the computer that operates of the present invention
Program code, described program design language include object oriented program language-such as Java, Smalltalk, C++,
Also include conventional procedural programming language-such as " C " language or similar programming language.Program code can be with
Fully perform, partly perform on the user computer on the user computer, the software kit independent as one performs, portion
Divide and partly perform or performed completely on remote computer or server on the remote computer on the user computer.
Be related in the situation of remote computer, remote computer can pass through the network of any kind --- including LAN (LAN) or
Wide area network (WAN)-be connected to subscriber computer, or, it may be connected to outer computer (such as carried using Internet service
Pass through Internet connection for business).
Pay attention to, above are only presently preferred embodiments of the present invention and institute's application technology principle.It will be appreciated by those skilled in the art that
The invention is not restricted to specific embodiment described here, can carry out for a person skilled in the art various obvious changes,
Readjust and substitute without departing from protection scope of the present invention.Therefore, although being carried out by above example to the present invention
It is described in further detail, but the present invention is not limited only to above example, without departing from the inventive concept, also
Other more equivalent embodiments can be included, and the scope of the present invention is determined by scope of the appended claims.
Claims (10)
1. the method for the management copy in a kind of data-storage system, it is characterised in that methods described includes：
Position L as of the file to be stored Fa in Chord rings is obtained by preset algorithm, and searches the storage of the Fa storages
Cluster；
Fa described in piecemeal, and obtain the storage location of the file Fa after memory partitioning；
Data storage block, and create the metadata of the data block.
2. according to the method for claim 1, it is characterised in that the storage cluster for searching the Fa storages, including：
According to the La on the Chord rings that the management node by storage cluster is formed, first management node Mi is found clockwise,
The M i are the management node that the Fa is stored in the storage cluster.
3. according to the method for claim 1, it is characterised in that Fa described in the piecemeal, including：
Fa orders are divided into each data block Bi for 64MB fixed sizes, and data block is numbered.
4. according to the method for claim 2, it is characterised in that the storage position of the file Fa after the acquisition memory partitioning
Put, including：
The request to the Mi data storages sent is received, the Mi stores the data block server node in cluster according to place
Loading condition, return to the information of the data block server node for storing the Fa.
5. according to the method for claim 3, it is characterised in that the data storage block, including：
Blocked file B1~BN is deposited into corresponding data block server node.
6. according to the method for claim 1, it is characterised in that the metadata for creating the data block, including：
After the Fa is stored, the management node Mi for storing cluster preserves resource identification File ID, file name, the text of the Fa
The version number of mapping and the Fa between part size, supporter, data block Bi and storage location.
7. according to the method described in claim 1 to 6 any one, it is characterised in that the data storage block, and described in establishment
After the metadata of data block, in addition to：
Store redundant copy；
According to Replica Placement Strategy, selection storage cluster simultaneously stores the redundant copy；
Return to storage result.
8. the device of the management copy in a kind of data-storage system, it is characterised in that described device includes：
Searching modul, for obtaining position L as of the file to be stored Fa in Chord rings by preset algorithm, and described in lookup
The storage cluster of Fa storages；
Acquisition module, for Fa described in piecemeal, and obtain the storage location of the file Fa after memory partitioning；
Creation module, for data storage block, and create the metadata of the data block.
9. device according to claim 8, it is characterised in that the searching modul, be used for：
According to the La on the Chord rings that the management node by storage cluster is formed, first management node Mi is found clockwise,
The Mi is the management node that the Fa is stored in the storage cluster；
The acquisition module, is used for：
Fa orders are divided into each data block Bi for 64MB fixed sizes, and data block is numbered；Receive to
The request for the data storage that the Mi is sent, the Mi store the load feelings of the data block server node in cluster according to place
Condition, return to the information for the data block server node for storing the Fa；
The creation module, is used for：
Blocked file B1~BN is deposited into corresponding data block server node, after the Fa is stored, stores the management of cluster
Node M i preserves resource identification File ID, file name, file size, supporter, data block Bi and the storage location of the Fa
Between mapping and the Fa version number；
The data storage block, and after creating the metadata of the data block, in addition to：
Store redundant copy；
According to Replica Placement Strategy, selection storage cluster simultaneously stores the redundant copy；
Return to storage result.
10. a kind of computer-readable recording medium, it is characterised in that be stored thereon with computer program, the computer program quilt
The method that the management copy in the data-storage system any one of claim 1-7 is realized during computing device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710889792.6A CN107728940A (en) | 2017-09-27 | 2017-09-27 | The method, apparatus and storage medium of a kind of management copy in data-storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710889792.6A CN107728940A (en) | 2017-09-27 | 2017-09-27 | The method, apparatus and storage medium of a kind of management copy in data-storage system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107728940A true CN107728940A (en) | 2018-02-23 |
Family
ID=61208223
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710889792.6A Pending CN107728940A (en) | 2017-09-27 | 2017-09-27 | The method, apparatus and storage medium of a kind of management copy in data-storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107728940A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111382123A (en) * | 2018-12-28 | 2020-07-07 | 广州市百果园信息技术有限公司 | File storage method, device, equipment and storage medium |
| CN113821362A (en) * | 2021-11-25 | 2021-12-21 | 云和恩墨(北京)信息技术有限公司 | Data copying method and device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103810061A (en) * | 2014-01-28 | 2014-05-21 | 河南科技大学 | High-availability cloud storage method |
| CN104050015A (en) * | 2014-06-27 | 2014-09-17 | 国家计算机网络与信息安全管理中心 | Mirror image storage and distribution system for virtual machines |
-
2017
- 2017-09-27 CN CN201710889792.6A patent/CN107728940A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103810061A (en) * | 2014-01-28 | 2014-05-21 | 河南科技大学 | High-availability cloud storage method |
| CN104050015A (en) * | 2014-06-27 | 2014-09-17 | 国家计算机网络与信息安全管理中心 | Mirror image storage and distribution system for virtual machines |
Non-Patent Citations (1)
| Title |
|---|
| 王军正: "基于对等结构云存储的副本管理研究", 《中国优秀硕士学位论文全文数据库》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111382123A (en) * | 2018-12-28 | 2020-07-07 | 广州市百果园信息技术有限公司 | File storage method, device, equipment and storage medium |
| CN111382123B (en) * | 2018-12-28 | 2023-06-16 | 广州市百果园信息技术有限公司 | File storage method, device, equipment and storage medium |
| CN113821362A (en) * | 2021-11-25 | 2021-12-21 | 云和恩墨(北京)信息技术有限公司 | Data copying method and device |
| CN113821362B (en) * | 2021-11-25 | 2022-03-22 | 云和恩墨(北京)信息技术有限公司 | Data copying method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103793425B (en) | Data processing method and device for distributed system | |
| Zhu et al. | Efficient, proximity-aware load balancing for DHT-based P2P systems | |
| CN107111653A (en) | The query optimization that Installed System Memory suitable for parallel database system is loaded | |
| CN104050015A (en) | Mirror image storage and distribution system for virtual machines | |
| CN106161610A (en) | A kind of method and system of distributed storage | |
| Chatzimilioudis et al. | Distributed in-memory processing of all k nearest neighbor queries | |
| CN106034160A (en) | Distributed computing system and method | |
| US9501509B2 (en) | Throwaway spatial index structure for dynamic point data | |
| CN104811392A (en) | Method and system for processing resource access request in network | |
| CN103399894A (en) | Distributed transaction processing method on basis of shared storage pool | |
| CN111030983B (en) | Data processing method and device based on distributed distribution and related equipment | |
| CN110502519A (en) | A kind of method, apparatus of data aggregate, equipment and storage medium | |
| US20230229580A1 (en) | Dynamic index management for computing storage resources | |
| CN109800336A (en) | Table implementation method, device, equipment and medium based on key-value pair storage system | |
| WO2022087837A1 (en) | Blockchain system having efficient world state data structures | |
| WO2022087834A1 (en) | Blockchain system having efficient world state data structures | |
| CN108431815A (en) | The duplicate removal complex data of distributed data in processor grid | |
| Zhang et al. | Enhanced adaptive cloudlet placement approach for mobile application on spark | |
| CN107728940A (en) | The method, apparatus and storage medium of a kind of management copy in data-storage system | |
| US11748004B2 (en) | Data replication using active and passive data storage modes | |
| WO2024021470A1 (en) | Cross-region data scheduling method and apparatus, device, and storage medium | |
| JP2012008854A (en) | Storage virtualization device | |
| CN102821122A (en) | Method and apparatus for node distribution and computer program product | |
| US20060209717A1 (en) | Distributed storing of network position information for nodes | |
| CN107547657A (en) | A kind of method, apparatus and storage medium numbered based on one point data in cloud storage system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180223 |