CN102594852A - Data access method, node and system - Google Patents
Data access method, node and system Download PDFInfo
- Publication number
- CN102594852A CN102594852A CN2011100050990A CN201110005099A CN102594852A CN 102594852 A CN102594852 A CN 102594852A CN 2011100050990 A CN2011100050990 A CN 2011100050990A CN 201110005099 A CN201110005099 A CN 201110005099A CN 102594852 A CN102594852 A CN 102594852A
- Authority
- CN
- China
- Prior art keywords
- data
- node
- blocks
- memory node
- memory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data access method, a data access device and a data access system. The method comprises the following steps that: an access request containing a block number identifier is received; a pre-established local block unit searches the corresponding relation between the block number of the local block unit and storage node resource information providing storage services according to the block number to obtain a storage node position corresponding to the block number identifier and a block data identifier; and the local block unit sends the data access request to the storage node according to the storage node position corresponding to the block number identifier and the block data identifier, wherein the storage node resource information comprises the storage node position information and the block number identifier of a storage node. By the method, the device and the system, the problem that a metadata server becomes a bottleneck of the system because the current distributed file system is required to address through the metadata server at every access is solved, and the access efficiency is low is overcome.
Description
Technical field
The present invention relates to data service technology in the communications field, particularly, relate to data access method, node and system.
Background technology
The most frequently used occupation mode of data-storage system is a file system; For example, Fig. 1 file system passes through pathname, as/the home/myname/novel/novell.txt access file; Pathname is in the NameSpace; The path is in the tree-shaped space, and the logical relation that tree-shaped spatial characterization is certain is relatively more friendly to the user.Database Systems are occupation modes of another kind of storage system, and it does not rely on file system path and embodies logical relation, come characterization logic with " table " or other structures, like Database Systems among Fig. 1 with " table " characterization logic.
Like Fig. 1, all need the bottom of the data-storage system (file system and Database Systems) of random access all to be implemented on the block device.So-called block device; Be can be through specifying a block number; Visit an appointed area in the equipment; The numbering of piece can be carried out randow addressing in a very big addressing space, but block number itself does not have the logic implication, and the user need have the system of user logic by the file system on upper strata or Database Systems etc.
The visit of file system has better logical construction, but block access can be supported various file system and other storage systems, and like Database Systems, flexibility is higher.To the file read-write flow process of Fig. 1 as shown in Figures 2 and 3; Read-write is initiated by client; Deliver to file system layer, file system client provides operation-interfaces such as File Open, location, read-write to application, and successful File Open operation can be returned the descriptor of a file; This descriptor is sequence number or pointer; Correspondence unique sign that is used for saying so is wanted the sign of operation file, the file access operation of application programs, and file system layer will go some pieces of block device visit to read information or write data one or more times.Storing the metadata that management and constituent act are used in some piece; Some piece is being stored user data; The different files system realizes adopting different modes to deposit and read these pieces, and a plurality of block operations that file system is converted into are sent to the block device layer; The final disk that arrives is operated, and has omitted optimized Measures such as buffer memory here.Use the client of block device service to obtain a storage area, instruct through small computer system interface (SCSI) or advanced technology attachment interface (ATA) block operations and deposit, read, upgrade and delete memory block with forms such as disks.General user can set up file system and use on disk.
Along with the development of cloud computing, the challenge of storage system is capacity and cost, has both needed jumbo memory space, need reduce the cost of storage system again.In the face of this demand, need to use a large amount of common servers to make up jumbo storage system, the block access of Fig. 1-Fig. 3 is inappropriate for distributed access.
Above-mentioned Fig. 1-Fig. 3 is for passing through file system or data system access block equipment; A kind of parallel network file system (pNFS) of following Fig. 4-6 for providing to the low-cost demand of the big capacity of cloud computing; Be that the user visits the pNFS meta data server with the mode of file system interface and obtains metadata; And then from the equipment (document storage server) of distributed store with piece interface or file system interface visit data, improve the extensibility and the performance of system with the resource of utilizing distribution.Other have also all used similar framework like LustreFS distributed file system.
In Fig. 4, the correspondence table of the file system layer storage file descriptor of meta data server and Data Position information (block number), wherein; In the distributed system of Fig. 4; Be not the notion of lay special stress on piece, Data Position information need have file storage nodal information and port numbers, and the user removes to visit the corresponding file memory node according to Data Position information; The file storage node finds file or data block according to filename or piece ID, and Fig. 5 and Fig. 6 are the corresponding read and write access control flow chart of system shown in Figure 4.
The file system different operation of distributed file system client employing and Fig. 1-Fig. 3 among Fig. 5 and Fig. 6; It can filesystem-aware distributed; And have the access mode of meta data server at least, about document creation, operation such as open, close, all be the operation of carrying out through meta data server; The function that it is born basically can this machine of analogy file system file system layer, be the operation information that transmits through network between the two.Like Fig. 5; The client uses with the file system interface mode and sends the File Open access request to meta data server; The file system of meta data server finds corresponding Data Position information according to the filec descriptor in the access request; And return the client and use, the client uses can read the corresponding file memory node according to Data Position information.Like Fig. 6; The client uses with the file system interface mode and sends the establishment file request to meta data server; The file system of meta data server is created the corresponding file descriptor; When the client uses the write operation visit, distribute the memory space of appointment to return the client according to write operation visit file descriptor and use, comprise designated store node and data description sign during allocation space.The client uses according to the assignment of allocation space and finds corresponding memory node, and the file storage node is carried out write operation.
Metadata server node is when allocation space or return data positional information, and the memory node of meeting designated user data and the description of data identify, and wherein data description identifies the normally address or the data number of a data position.Concrete user data read-write operation occurs on the memory node, and the data class of the piecemeal that these are concrete is similar to the piece of unit file system, but possibly be some files or the piece on some concentrated piece memory device on the memory node in different realizations.
In realizing process of the present invention, the inventor finds that the each visit of existing distributed file system needs one or many to carry out addressing through meta data server, access stored node then, and meta data server becomes the bottleneck of system.
Summary of the invention
First purpose of the present invention is to propose a kind of data access method, to reduce the visit expense of prior art through the meta data server addressing.
Second purpose of the present invention is to propose a kind of client node, to reduce the visit expense of prior art through the meta data server addressing.
The 3rd purpose of the present invention is to propose a kind of storage Control Node, to reduce the visit expense of prior art through the meta data server addressing.
The 4th purpose of the present invention is to propose a kind of data-storage system, to realize improving access efficiency.
For realizing above-mentioned first purpose, according to an aspect of the present invention, a kind of data access method is provided, comprising: receive the access request that comprises the block number sign;
Pre-established local module unit is searched the block number and the corresponding relation that the memory node resource information of stores service is provided of local module unit according to the block number sign, obtain corresponding memory node position of block number sign and blocks of data sign;
Memory node position and blocks of data sign that local module unit is corresponding according to the block number sign are sent the blocks of data access request to memory node;
Wherein, the memory node resource information comprises the blocks of data sign that the memory node of stores service positional information and memory node are provided.
Preferably, receiving the access request that comprises the block number sign can also comprise before: send the blocks of data that comprises the memory capacity size to Control Node and open request; Receive Control Node and open the client node positional information of request, the memory node resource information that the memory allocated amount of capacity is corresponding according to the storage resource status information and/or the transmission blocks of data of memory node; Set up local block device unit according to the memory node resource information.
Preferably; Reception can also comprise before comprising the access request of block number sign: receive the access request that directly comprises the block number sign and/or receive filename/table data access request through file/Database Systems interface through the block device interface, resolution file name or the corresponding block number of table data identify.
Preferably.Local module unit is to provide the virtual local block device of distributed storage service to drive perhaps for the virtual machine of application service is provided.
For realizing above-mentioned second purpose, according to another aspect of the present invention, a kind of client node is provided, comprising: one or more local module unit, wherein, local module unit comprises: interface module is used to receive the access request that comprises the block number sign; Memory module, the corresponding relation that is used to store the block number of local module unit and the memory node resource information of stores service is provided; The memory node resource information comprises the blocks of data sign that the memory node of stores service information, memory node are provided; Processing module is used for searching memory cell according to the block number sign, obtains corresponding memory node position of block number sign and blocks of data sign, sends the blocks of data access request through interface module to memory node.
For realizing above-mentioned the 3rd purpose, according to another aspect of the present invention, a kind of storage Control Node is provided, comprising: interface module is used to receive blocks of data and opens/discharge request;
Control module is used for opening the client node positional information of request according to the storage resource status information of one or more memory node and/or the blocks of data of reception, distributes corresponding memory node resource information; Perhaps discharge request and discharge corresponding memory node resource information according to blocks of data;
Memory module is used to store the memory node resource information that one or more memory node has distributed;
Wherein, the memory node resource information comprises the blocks of data sign of memory node information and memory node.
For realizing above-mentioned the 4th purpose, according to another aspect of the present invention, a kind of data-storage system is provided, comprise the client node of second purpose and the storage Control Node of the 3rd purpose.
The data access method of various embodiments of the present invention, device and system; Owing to the more local module unit of bottom is provided, by the corresponding relation of local module unit memory block numbering, therefore when carrying out data access with pre-assigned memory node resource information; Need not carry out addressing through meta data server of the prior art etc.; Therefore nonexistence ability bottleneck can significantly reduce the visit expense, has improved access efficiency.
The present invention also some embodiment can ownly directly use the file system interface of block device interface or establishment oneself to visit local module unit according to application choice in use, therefore has the better system expandability.Local module unit among the present invention can be applied to virtual machine, and the user can only visit the storage resources of distribution, and stronger for many tenants isolating power of public service, fail safe is higher.
Other features and advantages of the present invention will be set forth in specification subsequently, and, partly from specification, become obvious, perhaps understand through embodiment of the present invention.The object of the invention can be realized through the structure that in the specification of being write, claims and accompanying drawing, is particularly pointed out and obtained with other advantages.
Through accompanying drawing and embodiment, technical scheme of the present invention is done further detailed description below.
Description of drawings
Accompanying drawing is used to provide further understanding of the present invention, and constitutes the part of specification, is used to explain the present invention with embodiments of the invention, is not construed as limiting the invention.In the accompanying drawings:
Fig. 1 is the logic relation picture according to block device of the prior art and file system, Database Systems;
Fig. 2 is the read access operational flowchart to Fig. 1;
Fig. 3 is the number of write access operations flow chart to Fig. 1;
Fig. 4 is the logic relation picture according to parallel network file system of the prior art (pNFS);
Fig. 5 is the read access operational flowchart to Fig. 4;
Fig. 6 is the number of write access operations flow chart to Fig. 4;
Fig. 7 is data access method embodiment one flow chart according to the present invention;
Fig. 8 is data access method embodiment two flow charts according to the present invention;
Fig. 9 is the parsing sketch map of the data access method according to the present invention, and data storage system architecture sketch map of the present invention;
Figure 10 opens/the release flow sketch map for the data resource application to Fig. 9;
Figure 11 is access process embodiment sketch map in the data access method of the present invention;
Figure 12 is the read access operational flowchart to Figure 11;
Figure 13 is the number of write access operations flow chart to Figure 11;
Figure 14 is memory control device embodiment sketch map according to the present invention;
Figure 15 is client node embodiment sketch map according to the present invention.
Embodiment
Below in conjunction with accompanying drawing the preferred embodiments of the present invention are described, should be appreciated that preferred embodiment described herein only is used for explanation and explains the present invention, and be not used in qualification the present invention.
Method embodiment
Fig. 7 is data access method embodiment one flow chart according to the present invention, and as shown in Figure 7, present embodiment comprises:
Step S102: receive the access request that comprises the block number sign;
Step S104: pre-established local module unit is searched the block number and the corresponding relation that the memory node resource information of stores service is provided of local module unit according to the block number sign, obtains corresponding memory node position of block number sign and blocks of data sign; Wherein, the memory node resource information comprises the blocks of data sign that the memory node of stores service positional information and this memory node are provided;
Local module unit in the present embodiment can pass through to a Control Node application blocks of data; Thereby obtain to provide the memory node resource information of server; And set up a local module unit according to the blocks of data of application in advance; As providing the virtual local block device driving of distributed storage service to offer the application program use, application program can be carried out follow-up data access according to this unit;
Step S106: memory node position and blocks of data sign that local module unit is corresponding according to the block number sign, send the blocks of data access request to this memory node.
Because local module unit stores one or more memory node resource information; Then can carry out data access according to memory node position in each storage resources information and blocks of data sign; Memory node also can divide the visit from different client nodes according to the blocks of data tag slot, specifically can be referring to follow-up Fig. 9 embodiment.Present embodiment is owing to provide the more local module unit of bottom; By the corresponding relation of local module unit memory block numbering with pre-assigned memory node resource information; Therefore when carrying out data access, need not carry out addressing, so nonexistence can bottleneck through meta data server of the prior art etc.; Can significantly reduce the visit expense, improve access efficiency.
Fig. 8 is data access method embodiment two flow charts according to the present invention, and as shown in Figure 8, present embodiment comprises:
Step S202: as shown in Figure 9, client node sends the blocks of data that comprises the memory capacity size to Control Node and opens/discharge request;
Step S204: storage Control Node (also claiming management node or Control Node) is monitored the resource user mode of each memory node; Allocation space according to the client node application; And the resource state information of each memory node; Distribute to the resource of one or more memory node of client node, comprise memory node positional information (address/port number), blocks of data sign etc.;
Management node can also be further according to the position of the client node of this request of transmission, distributes nearer memory node resource information to client node;
Management node also can combine the resource status of client node position and memory node, distributes one or more memory node resource to give client node;
Step S206: client node obtains the memory node resource information;
Step S208: be created as a local block device according to the memory node resource information and drive;
Step S210: client node can directly be visited local block device driving through the block device interface accessing instruction of standard, and is mapped to the visit to memory node; Can also oneself create file system or Database Systems; And filename or table data are resolved through file system interface or Database Systems interface; Obtain local block device and drive corresponding block number sign, further drive and be mapped to visit certain memory node by local block device.
Present embodiment combines the advantage of centralized access and distributed access; Can have access to blocks of data and the visit capacity is big; Client node drives through allocation space and the local block device of corresponding increase that Control Node obtains; The user can directly use the local file system of block device or establishment oneself in use according to application choice, therefore has the better system expandability.Present embodiment adopts the local block device that increases to drive, and do not use in the meta data server and search, so nonexistence can bottleneck.Present embodiment can also be applied to virtual machine, and the user can only visit the storage resources of distribution, and stronger for many tenants isolating power of public service, fail safe is higher.
Fig. 9 is the parsing sketch map of the data access method according to the present invention, and data storage system architecture sketch map of the present invention.
Data-storage system as shown in Figure 9 comprises storage Control Node and client node; Control Node is a centralized management node in logic; Externally provide one can be used to apply for, the interface of release block data; And can carry out the overall situation, that concentrate, management control reliably, and the application interface with client node is provided, control the storage resource status of each memory node; Memory node (also can be called service node); Be mainly used in the distribution of carrying out concrete blocks of data and stores service is provided; After Control Node was distributed to client node with the storage resources of certain memory node, memory node can directly offer the user with the storage resources on this node without Control Node and use.
Like Fig. 9, physics is disposed, and Control Node is not necessarily a single or node independently, can be a cluster, even the multiplexing part memory node, is positioned at memory node and carries out the function that storage administration is controlled.And memory node provides the physical server that has disk space of block device service corresponding to each.Control Node receives the operation of also process user application, release block equipment, adjustment block device size; The memory node storaging user data, carry out data block increase, delete, obtain, operation such as modification.
Be elaborated through Figure 10-13 pair of data distribution, access process below.
Figure 10 opens/the release flow sketch map for the data resource application to Fig. 9, and shown in figure 10, data storage resource application/release flow occurs between client node and the Control Node, comprising:
Step 1: client node can apply for specifying the block device of size, opens request as sending the blocks of data that comprises the designated store capacity to Control Node;
Step 2: Control Node receives the resource operating positions such as free space and effective status that can check each memory node before the request of opening, can active monitoring or each memory node report;
Step 3: Control Node is owing to stored the resource operating position of each node; According to the storage resource status of memory node and the position of client node, the information such as memory capacity space size of request; Allocation space on memory node; The memory space that distributes is offered client node use, in order to reach high availability, Control Node can distribute the resource of a plurality of memory nodes to give same client as redundancy.Occupied for fear of follow-up space, Control Node can be sent memory node in advance after distribution, makes it reserve corresponding space size;
Like the space of client node request 2G, management node distributes according to each memory node resource operating position:
Memory node 1, port numbers 5, allocation block Data Identification lv0 (the blocks of data 1-10000 on the corresponding stored node 1);
Memory node 2, port numbers 4, allocation block Data Identification lv1 (the data 2000-12000 on the corresponding stored node 2); ...
The concrete memory node that divides timing can according to circumstances distribute a plurality of redundancies is given same client node.
Step 4: the memory node resource information after Control Node will be distributed sends to client node; And the memory node resource information of this client node has been distributed in the Control Node storage, for example memory node sign, positional information, space size, blocks of data area identification or the like;
Step 5: client node is according to the result who returns; The sign of storage area that comprises the concrete service of memory node address, appointment; The area identification lv0 that the blocks of data that for example distributes in the step 3 is corresponding, lv1 etc., according to these information, the memory node of service can be distinguished the visit from different client nodes; Client node is set up a local block device driving with the resource information of these memory nodes subsequently and is offered the application program use, and so far application process is accomplished.Each memory node of Control Node management confirms that the space that can store is big or small.
Present embodiment only is the example explanation with the application process, and particularly, before client node was initiatively initiated release flow, these storage resources were operable all the time for the user.Release flow is similar with application, no longer illustrates.
The framework that the local block device that Figure 10 is built into uses is shown in figure 11.After storage resources has been opened in Fig. 9 and Figure 10 application; The use flow process of follow-up resource; Only occur in client node and provide between the memory node (service node) of service like read and write etc.; The position of must be at every turn searching the block device place with prior art in the meta data server is different, and the present embodiment nonexistence can bottleneck.
Described like Fig. 8 embodiment; Among Figure 11; Client node can be used as a local block device and drives after setting up local module unit; Offer application program and use, and can directly carry out the block number visit or carry out file access, therefore have the better system expandability through creating file system, Database Systems etc. to the block device interface of this this locality block device through standard.Wherein can the block device interface of standard between the local module unit of Figure 11 and the upper strata.Local block device receives can rank to wait after the block access instruction and handles to improve access efficiency, specifically can be referring to follow-up Figure 12, and the block access instruction after the processing is mapped as corresponding storage node accesses by local module unit, and sends to the memory node of correspondence.
The local block device that present embodiment adopt to increase drives and shines upon visit, can be applied to virtual machine technique, the block device service that system shown in figure 11 can provide for the virtual machine of Da Yun elasticity calculation services can permanent storage, have high availability.
The mode user of distributed file system can see other resources of the overall situation in the prior art, and there are hidden danger in fail safe and reliability.Storage access services provided by the invention, the user can only visit the storage resources of distribution, can't see other storage resource status of the overall situation, and stronger for many tenants isolating power of public service, fail safe and reliability are higher.
Under the prior art virtual machine operating position; Each client virtual machine all has only a disk, and this disk is actual to come from " virtual machine image " file, is moving client operating system; After virtual machine was destroyed by instance, all existed the data on the disk just not exist.After the present invention provides data storage service; Virtual machine can have the local block device of several mappings to use, and local block device provided by the invention and virtual machine image are independently, and it doesn't matter with the operating system of client actual motion; Even virtual machine also can exist after destroying; And can regain this block device through management node and used by user's other virtual machine, even a local block device can use by a plurality of virtual machines simultaneously, greatly improved service quality.The local block device of service is installed on the main frame (Host) or server that virtual machine service is provided; The virtual machine that this block device is directly offered user's appointment uses; Virtual machine itself just can use this " this locality " disk without any need for changing, not needing installing drive.The local block device that various embodiments of the present invention are distributed to the user can allow the fully autonomous customization of user, has better flexibility.
If local module unit is not created file system or Database Systems for client node among Figure 11; Directly carry out the blocks of data visit; Then identify according to the block number of local module unit storage and the corresponding relation of memory node resource information, be mapped as visit the blocks of data of certain memory node according to block number, and the backward reference result; Process is fairly simple, and not drawing at this illustrates.
In order to compare the process of carrying out file access through use local block device of the present invention among Figure 12, Figure 13 below with centralized visit the of the prior art with distributed file system.
Figure 12 is the read access operational flowchart to Figure 11, and Figure 13 is the number of write access operations flow chart to Figure 11.
In Figure 12 and Figure 13 embodiment, the file system that client node is created has the corresponding block number of filec descriptor.When carrying out data access through file system; The application of client node is through the filename (descriptor) of file system; Obtain corresponding block number; And then carry out blocks of data to local block device and visit, local block device drives the blocks of data visit that this blocks of data is mapped as the corresponding stored node, thus the return data result.
Block number that filec descriptor is corresponding under the explained later and local block device drive the implication of the blocks of data sign of mapping, and the block number that filec descriptor is corresponding is the such numbering of for example 0x0500001 on the local block device; The blocks of data sign is wherein certain the block device corresponding identification on the memory node, like lv0, lv1, lv2.
For example; When the user visits the 0x0500001 block number on the own local block device through filec descriptor; Drive the block number of storage and the corresponding relation of memory node resource information according to local block device; The actual access stored node that is mapped to, the for example 0x50001 on the lv1 (blocks of data area identification) on the main frame host07 (address).
When carrying out write operation, local block device drives can issue the memory node that all distribute to the user according to block number, carries out the concurrent write operation.When read operation, can only issue the corresponding memory node of block number.
In the prior art that compares; Figure 12 and Figure 13 client are applied in the local block device of visit when driving; Flow process arrive local block device drive before with Fig. 2 and Fig. 3 in non-distributed applied scene similar, therefore compatible with conventional use, but the subsequent access flow process is distinguished to some extent: local block device drives the memory node resource information that obtains when applying for; The local block device of visit is mapped to the memory node that visit provides service; Like Figure 12 and Figure 13, local block device can carry out following processing after driving and receiving the blocks of data visit, comprising:
1. cache blocks device data visit before arriving preset visit quantity or time threshold;
2. visit sort (the blocks of data sign as according to visit sorts, and is convenient to improve the access efficiency of the identical memory node of visit) for the block device of buffer memory;
3. according to the characteristics of user capture, judge whether to read not request msg of certain customers in advance; As, know that according to user capture history carrying out continuity visits, and then can read in advance; User capture is if noncontinuity, can be read data in advance;
4. packing compression blocks accessing operation instruction, the information that obtains during according to application sends to and corresponds to stores service node (read operation is selected, and write operation also is dealt into a plurality of service nodes).
The above embodiment of the present invention can distributedly provide memory node with respect to centralized file of the prior art, and local block device drives according to resource bid in advance and finds corresponding memory node; If memory node is merely one, then file system and the local block device with client node is similar to the function that has centralized file system now, and at this moment, the local block device of client node is equivalent to a transponder, is forwarded to memory node.
Above-mentioned each method embodiment has following advantage:
1. compare with the storage system of distributed file system class in the prior art; The access to netwoks process of a metadata server node that the file system layer of prior art distributed file system is many; The present invention but can provide the more local block device access interface of bottom; The user can ownly directly use local block device to drive or create the local file system of oneself according to application choice in use, when flexibility is provided, has also reduced the expense of file system layer.
2. through the mapping of client node, client node on the physical store resource of one or more memory nodes, need not to ask Control Node with the blocks of data access map in the access process; The dispatching management of Control Node only play a role when breaking down in Resources allocation, and therefore, nonexistence can bottleneck.
3. compare with centralized block device, the distributed system framework has the better system expandability, because idle memory space in can the focus utilization cluster has the cost performance of better resource utilization and Geng Gao.
4. because the user only visits the resource of distributing to the user, other when using storage resources
Storage resources invisible concerning the user, stronger for many tenants' isolating power.
If Control Node distributes a plurality of memory node resource informations to give same client node as redundancy; The a plurality of memory nodes that obtain through mapping the storage data be consistent; When sudden failures (fault) took place several in the memory node, the information that remains effective memory node remains complete can provide service.After Control Node monitors the memory node inefficacy; Can be according to the resource operating position of each memory node of having stored before; The effective node of notice residue; The information reproduction of identical storage data to newly-increased memory node, and is pointed to new memory node with the mapping on the client node and reconfigures, reach high availability.
For example, the mapping relations of the user of each client node of Control Node storage and memory node can be as shown in table 1 below:
The mapping relations table of table 1 user and memory node (service node)
Particularly, the processing procedure after certain memory node lost efficacy can comprise:
Step 1: remove the malfunction and failure memory node, add new memory node, Control Node is distributed onesize subregion on newly-increased memory node; And the Control Node notice has effective node of identical data with failure node, recovers so that carry out follow-up data;
Step 2: effective node of service still, receive the notice of Control Node after, the blocks of data that requires is carried out complete snapshot, and utilizes snapshot data restore data in newly-increased node;
Step 3: because in execution in step 2 processes, data possibly upgraded, therefore; The data recovered that requires in the effective node in the step 2 is carried out increment snapshot (Updating Information in execution in step 2 processes); And with snapshot data restore data in newly-increased node, if increment snapshot size is bigger, greater than preset threshold value; Repeating step 3, otherwise get into step 4;
Step 4: if the size of last increment snapshot is less than threshold value; But still be not 0; Then can lock, the fast last increment snapshot data of quick-recovery makes new memory node in full accord with the memory node data of carrying out snapshot; Between lockup period, can utilize the local block device of client node to drive data cached change;
Step 5: remove memory node locking in the step 4, recover normal service.
Present embodiment is under the situation of using a plurality of memory nodes; When the part memory node breaks down; Can utilize the residue node to continue the service that provides continuous, and data are copied on other memory nodes automatically, to keep the availability of service; The dispatching management of Control Node only play a role when breaking down in Resources allocation, and nonexistence can bottleneck.
Figure 14 is client node embodiment sketch map according to the present invention.Shown in figure 14, the present embodiment client node comprises one or more local module unit, and wherein, local module unit comprises:
Interface module is used to receive the access request that comprises the block number sign;
Memory module, the corresponding relation that is used to store the block number of local module unit and the memory node resource information of stores service is provided; The memory node resource information comprises the blocks of data sign that the memory node of stores service information, memory node are provided;
Processing module is used for searching memory module according to the block number sign, obtains corresponding memory node position of block number sign and blocks of data sign, sends this blocks of data access request to memory node
Wherein, Communication module is the outer module of the local module unit of client node; Be used for opening request to Control Node application blocks of data through network; And receiving the memory node resource information that Control Node is distributed, local module unit is to drive according to the local block device that this memory node resource information is set up, when the interface module of local module unit receives data access request; Through after the processing module processing memory node blocks of data access request of correspondence being sent communication module, through communication module backward reference result.
Wherein the interface module of local module unit can comprise:
Directly the visit submodule is used for receiving the access request that directly comprises the block number sign through the block device interface; And/or
System's visit submodule is used for receiving filename/table data access request through file/Database Systems interface, and the block number sign that resolution file name/table data are corresponding is sent the access request that comprises the block number sign to processing module.
Processing module can also comprise:
The block access processing sub is used for blocks of data visit carrying out preliminary treatment, and to improve access efficiency, preliminary treatment comprises following combination in any: when the blocks of data visit is not more than preset visit quantity or time threshold, and the cache blocks data access; To blocks of data visit carrying out buffer memory and ordering; According to the blocks of data accessing characteristic, determine whether to read in advance not ask the blocks of data of visiting; Packing compression blocks data access.
The concrete access process of the client of present embodiment can no longer be repeated in this description same or similar content at this referring to said method embodiment related description.
Figure 15 is for storing Control Node embodiment sketch map according to the present invention.Like Figure 15, the present embodiment concrete function can be referring to the Control Node function among the said method embodiment, and like figure, present embodiment comprises:
Interface module, request is opened/discharged to the blocks of data that is used to receive client node;
Control module is used for opening the client node positional information of request according to the storage resource status information of one or more memory node and/or the blocks of data of reception, distributes corresponding memory node resource information; Perhaps discharge request and discharge corresponding memory node resource information according to blocks of data;
Memory module is used to store the memory node resource information that one or more memory node has distributed;
Wherein, the memory node resource information comprises memory node positional information, and the blocks of data of memory node sign.
This node apparatus can also comprise:
Monitoring modular after being used to monitor memory node and losing efficacy, is searched in the memory module effective memory node that identical storage data are arranged with the memory node that lost efficacy, and the storage data of effective memory node are copied to newly-established memory node.
Preferably, memory module is also stored the corresponding relation of client node and the memory node resource information of having distributed, and is as shown in table 1; Interface module further receives the request of obtaining that comprises the client node sign; Control module further according to client node sign from memory module search to the memory node resource information, and return interface module.
The storage Control Node of present embodiment can be between client node and one or more memory node, perhaps is positioned to be compounded on the memory node.Concrete control procedure can no longer be repeated in this description same or similar content at this referring to said method embodiment related description.
Can implement technology described herein through various means.For instance, these technology may be implemented in hardware, firmware, software or its combination.For the hardware embodiment, Figure 15 control module and native 14 processing modules may be implemented in one or more application-specific integrated circuit (ASIC)s (ASIC), digital signal processor (DSP), programmable logic device (PLD), field programmable gate array (FPGA), processor, controller, microcontroller, microprocessor, electronic installation, other through design with the electronic unit or its combination of carrying out function described herein in.
For firmware and/or software implementation scheme, the module of available execution function described herein (for example, process, step, flow process etc.) is implemented said technology.Firmware and/or software code can be stored in the memory and by processor and carry out.Memory may be implemented in the processor or processor outside.
The present invention can have multiple multi-form embodiment; Be that example combines accompanying drawing that technical scheme of the present invention is illustrated with Fig. 7-Figure 15 above; This does not also mean that the applied instantiation of the present invention can only be confined in the specific flow process or example structure; Those of ordinary skill in the art should understand; The specific embodiments that preceding text provided is some examples in the multiple its preferred usage, and is any through the memory node resource is mapped as local block device, and will be that the execution mode of corresponding stored node visit all should be within technical scheme of the present invention scope required for protection to the access map of local block device.
One of ordinary skill in the art will appreciate that: all or part of step that realizes said method embodiment can be accomplished through the relevant hardware of program command; Aforesaid program can be stored in the computer read/write memory medium; This program the step that comprises said method embodiment when carrying out; And aforesaid storage medium comprises: various media that can be program code stored such as ROM, RAM, magnetic disc or CD.
What should explain at last is: the above is merely the preferred embodiments of the present invention; Be not limited to the present invention; Although the present invention has been carried out detailed explanation with reference to previous embodiment; For a person skilled in the art, it still can be made amendment to the technical scheme that aforementioned each embodiment put down in writing, and perhaps part technical characterictic wherein is equal to replacement.All within spirit of the present invention and principle, any modification of being done, be equal to replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (15)
1. a data access method is characterized in that, comprising:
Reception comprises the access request of block number sign;
Pre-established local module unit is searched the block number of said local module unit according to said block number sign and the corresponding relation of the memory node resource information of stores service is provided, and obtains corresponding memory node position of said block number sign and blocks of data sign;
Memory node position and blocks of data sign that said local module unit is corresponding according to said block number sign are sent the blocks of data access request to said memory node;
Wherein, said memory node resource information comprises the blocks of data sign that the memory node of stores service positional information and said memory node are provided.
2. data access method according to claim 1 is characterized in that, receives the access request that comprises the block number sign and also comprises before:
Send the blocks of data that comprises the memory capacity size to Control Node and open request;
Receive said Control Node according to the storage resource status information of memory node and/or send the client node positional information that said blocks of data is opened request, distribute the corresponding memory node resource information of said memory capacity size;
Set up said local module unit according to said memory node resource information.
3. data access method according to claim 1 is characterized in that, receives the access request that comprises the block number sign and comprises before:
Receive the access request that comprises the block number sign through the block device interface; And/or
Receive filename/table data access request through file/Database Systems interface; Said file/Database Systems interface is resolved the corresponding block number sign of said filename/table data, sends the access request that comprises said block number sign to said local module unit.
4. data access method according to claim 1 is characterized in that, before said memory node sends the blocks of data access request, also comprises:
The blocks of data access request is carried out preliminary treatment, and said preliminary treatment comprises following combination in any:
When said blocks of data visit is not more than preset visit quantity or time threshold, the said blocks of data visit of buffer memory; Visit is carried out buffer memory and is sorted to blocks of data; According to the blocks of data accessing characteristic, determine whether to read in advance not ask the blocks of data of visiting; The said blocks of data visit of packing compression.
5. according to each described data access method of claim 1-4, it is characterized in that said local module unit is that the virtual local block device driving of distributed storage service is provided or is the virtual machine that application service is provided.
6. data access method according to claim 5 is characterized in that, also comprises:
After said virtual machine is destroyed, send the request of obtaining of the memory node resource information that comprises the client node sign to Control Node;
Receive the memory node resource information that said Control Node is returned according to said client node sign.
7. a client node is characterized in that, comprising:
One or more local module unit, wherein, said local module unit comprises:
Interface module is used to receive the access request that comprises the block number sign;
Memory module, the corresponding relation that is used to store the block number of local module unit and the memory node resource information of stores service is provided; Said memory node resource information comprises the blocks of data sign that the memory node of stores service information and said memory node are provided;
Processing module is used for searching said memory cell according to said block number sign, obtains corresponding memory node position of said block number sign and blocks of data sign, sends the blocks of data access request to said memory node.
8. client node according to claim 7 is characterized in that, said interface module comprises:
Directly the visit submodule is used for receiving the access request that comprises the block number sign through the block device interface; And/or
System's visit submodule is used for receiving filename/table data access request through file/Database Systems interface, resolves the corresponding block number sign of said filename/table data, sends the access request that comprises said block number sign to said processing module.
9. according to claim 7 or 8 described client nodes, it is characterized in that said processing module also comprises:
The block access processing sub is used for preliminary treatment is carried out in said blocks of data visit, and said preliminary treatment comprises following combination in any: when said blocks of data visit is not more than preset visit quantity or time threshold, and the said blocks of data visit of buffer memory; Said blocks of data visit to buffer memory is sorted; According to the blocks of data accessing characteristic, determine whether to read in advance not ask the blocks of data of visiting; The said blocks of data visit of packing compression.
10. according to claim 7 or 8 described client nodes, it is characterized in that said local module unit is the virtual machine that the virtual local block device driving of distributed storage service is provided or application service is provided.
11. a storage Control Node is characterized in that, comprising:
Interface module is used to receive blocks of data and opens/discharge request;
Control module is used for opening the client node positional information of request according to the storage resource status information of one or more memory node and/or the said blocks of data of reception, distributes the corresponding memory node resource information of the said blocks of data request of opening; Perhaps discharge request and discharge corresponding memory node resource information according to said blocks of data;
Memory module is used to store the memory node resource information that one or more memory node has distributed;
Wherein, said memory node resource information comprises the blocks of data sign of memory node information and said memory node.
12. storage Control Node according to claim 11 is characterized in that, also comprises:
Monitoring modular after being used to monitor memory node and losing efficacy, is searched in the said memory module effective memory node that identical storage data are arranged with the memory node of said inefficacy, and the storage data of said effective memory node are copied to newly-established memory node.
13. storage Control Node according to claim 11 is characterized in that, said memory module is also stored the corresponding relation of client node and the memory node resource information of having distributed;
Said interface module further receives the request of obtaining that comprises the client node sign;
Said control module further according to said client node sign from said memory module search to the memory node resource information, and return said interface module.
14. according to each described storage Control Node of claim 11-13, it is characterized in that, between client node and one or more memory node, perhaps be positioned on the said memory node.
15. a data-storage system is characterized in that, comprises each described storage Control Node of each described client node of claim 7-10 and claim 11-14.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110005099.0A CN102594852B (en) | 2011-01-04 | 2011-01-04 | Data access method, node and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110005099.0A CN102594852B (en) | 2011-01-04 | 2011-01-04 | Data access method, node and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102594852A true CN102594852A (en) | 2012-07-18 |
| CN102594852B CN102594852B (en) | 2016-03-30 |
Family
ID=46483053
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110005099.0A Active CN102594852B (en) | 2011-01-04 | 2011-01-04 | Data access method, node and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102594852B (en) |
Cited By (43)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103024053A (en) * | 2012-12-18 | 2013-04-03 | 华为技术有限公司 | Cloud storage method, resource scheduling system and cloud storage node and system |
| CN103366573A (en) * | 2013-07-10 | 2013-10-23 | 中兴智能交通（无锡）有限公司 | Vehicle running information tracking method and system based on cloud computing |
| WO2014101240A1 (en) * | 2012-12-31 | 2014-07-03 | 华为技术有限公司 | Data storage method and apparatus, data operation method and system, and access server |
| CN104184812A (en) * | 2014-08-20 | 2014-12-03 | 四川九成信息技术有限公司 | Multi-point data transmission method based on private cloud |
| CN104239517A (en) * | 2014-09-17 | 2014-12-24 | 杭州华三通信技术有限公司 | Storage object access method and device for distributed storage system |
| CN104360915A (en) * | 2014-10-31 | 2015-02-18 | 北京思特奇信息技术股份有限公司 | Data recovery method and device on basis of distributed storage |
| CN104461736A (en) * | 2014-12-08 | 2015-03-25 | 畅捷通信息技术股份有限公司 | Resource allocation and search method and system and cloud server |
| CN104484236A (en) * | 2014-11-28 | 2015-04-01 | 曙光云计算技术有限公司 | HA (high availability) access adaptation method |
| CN104517067A (en) * | 2014-12-30 | 2015-04-15 | 华为技术有限公司 | Method, device and system for data access |
| CN104754021A (en) * | 2013-12-31 | 2015-07-01 | 伊姆西公司 | Device and method for promoting access of data in distributed storage system |
| CN105187555A (en) * | 2015-09-29 | 2015-12-23 | 北京奇艺世纪科技有限公司 | Method, device and system for message processing |
| CN105446794A (en) * | 2014-09-30 | 2016-03-30 | 北京金山云网络技术有限公司 | Disc operation method, apparatus and system based on virtual machine |
| WO2016065613A1 (en) * | 2014-10-31 | 2016-05-06 | 华为技术有限公司 | File access method, distributed storage system and network device |
| CN105704218A (en) * | 2016-01-19 | 2016-06-22 | 国家电网公司 | Data storage and management method in cloud computing platform |
| CN106302607A (en) * | 2015-06-05 | 2017-01-04 | 腾讯科技（深圳）有限公司 | It is applied to block storage system and the method for cloud computing |
| CN106503058A (en) * | 2016-09-27 | 2017-03-15 | 华为技术有限公司 | A kind of data load method, terminal and computing cluster |
| CN106850827A (en) * | 2017-02-24 | 2017-06-13 | 深圳市中博睿存信息技术有限公司 | A kind of metadata of distributed type file system treating method and apparatus |
| CN106993022A (en) * | 2016-12-30 | 2017-07-28 | 中国银联股份有限公司 | For the mark management method of cluster, identification server and corresponding system |
| CN107092437A (en) * | 2016-02-17 | 2017-08-25 | 杭州海康威视数字技术股份有限公司 | Data write-in, read method and device, cloud storage system |
| CN107346341A (en) * | 2017-07-11 | 2017-11-14 | 王焱华 | A kind of management method of big data |
| CN107562376A (en) * | 2017-08-23 | 2018-01-09 | 北京小米移动软件有限公司 | Data method for deleting and device |
| CN108156187A (en) * | 2016-12-02 | 2018-06-12 | 航天星图科技（北京）有限公司 | A kind of cloud service system |
| CN108197159A (en) * | 2017-12-11 | 2018-06-22 | 厦门集微科技有限公司 | Digital independent, wiring method and device based on distributed file system |
| CN109120709A (en) * | 2018-09-03 | 2019-01-01 | 杭州云创共享网络科技有限公司 | A kind of caching method, device, equipment and medium |
| CN109495568A (en) * | 2018-11-19 | 2019-03-19 | 北京金山云网络技术有限公司 | A kind of data capture method, data acquisition facility, server and storage medium |
| CN109831540A (en) * | 2019-04-12 | 2019-05-31 | 成都四方伟业软件股份有限公司 | Distributed storage method, device, electronic equipment and storage medium |
| CN109977092A (en) * | 2019-03-26 | 2019-07-05 | 武汉深之度科技有限公司 | A kind of method and calculating equipment of duplication file |
| CN110365719A (en) * | 2018-03-26 | 2019-10-22 | 华为技术有限公司 | A kind of method and relevant device of data processing |
| CN110413694A (en) * | 2019-08-01 | 2019-11-05 | 重庆紫光华山智安科技有限公司 | Metadata management method and relevant apparatus |
| CN110889121A (en) * | 2019-10-12 | 2020-03-17 | 中国平安财产保险股份有限公司 | Method, server and storage medium for preventing data leakage |
| WO2020183311A1 (en) * | 2019-03-08 | 2020-09-17 | International Business Machines Corporation | Secure storage query and donation |
| CN111881470A (en) * | 2020-07-20 | 2020-11-03 | 易通星云（北京）科技发展有限公司 | Data access method and device and computer readable storage medium |
| CN112015696A (en) * | 2020-08-21 | 2020-12-01 | 北京奇艺世纪科技有限公司 | Data access method, data relationship setting method, data access device, data relationship setting device and storage medium |
| CN112597173A (en) * | 2021-03-08 | 2021-04-02 | 成都虚谷伟业科技有限公司 | Distributed database cluster system peer-to-peer processing system and processing method |
| CN113419671A (en) * | 2021-05-20 | 2021-09-21 | 佛山市禅城区政务服务数据管理局 | Personal data space access control method and system |
| WO2021189306A1 (en) * | 2020-03-25 | 2021-09-30 | Beijing Didi Infinity Technology And Development Co., Ltd. | Write operation in object storage system using enhanced meta structure |
| US11176054B2 (en) | 2019-03-08 | 2021-11-16 | International Business Machines Corporation | Host virtual address space for secure interface control storage |
| US11182192B2 (en) | 2019-03-08 | 2021-11-23 | International Business Machines Corporation | Controlling access to secure storage of a virtual machine |
| US11283800B2 (en) | 2019-03-08 | 2022-03-22 | International Business Machines Corporation | Secure interface control secure storage hardware tagging |
| CN114647559A (en) * | 2022-03-21 | 2022-06-21 | 北京百度网讯科技有限公司 | Storage usage amount statistical method and device, electronic equipment and storage medium |
| US11455398B2 (en) | 2019-03-08 | 2022-09-27 | International Business Machines Corporation | Testing storage protection hardware in a secure virtual machine environment |
| CN115712660A (en) * | 2022-01-29 | 2023-02-24 | 杭州宇信数字科技有限公司 | Data storage method, device, server and storage medium |
| WO2024139946A1 (en) * | 2022-12-28 | 2024-07-04 | International Business Machines Corporation | Increasing resource utilization in cloud computing clusters |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1858748A (en) * | 2006-05-22 | 2006-11-08 | 上海怀瑾计算机科技有限公司 | Method for realizing directly access USB master facility end file from slave facility end at USB |
| US20070276838A1 (en) * | 2006-05-23 | 2007-11-29 | Samy Khalil Abushanab | Distributed storage |
| CN101094129A (en) * | 2006-06-20 | 2007-12-26 | 腾讯科技（深圳）有限公司 | Method for accessing domain name, and client terminal |
| US20090177860A1 (en) * | 2005-02-23 | 2009-07-09 | Yaolong Zhu | Data management method for network storage system and the network storage system built thereof |
| CN101510219A (en) * | 2009-03-31 | 2009-08-19 | 成都市华为赛门铁克科技有限公司 | File data accessing method, apparatus and system |
| CN101751233A (en) * | 2009-12-31 | 2010-06-23 | 成都索贝数码科技股份有限公司 | Method and system for expanding capacity of memory device |
-
2011
- 2011-01-04 CN CN201110005099.0A patent/CN102594852B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090177860A1 (en) * | 2005-02-23 | 2009-07-09 | Yaolong Zhu | Data management method for network storage system and the network storage system built thereof |
| CN1858748A (en) * | 2006-05-22 | 2006-11-08 | 上海怀瑾计算机科技有限公司 | Method for realizing directly access USB master facility end file from slave facility end at USB |
| US20070276838A1 (en) * | 2006-05-23 | 2007-11-29 | Samy Khalil Abushanab | Distributed storage |
| CN101094129A (en) * | 2006-06-20 | 2007-12-26 | 腾讯科技（深圳）有限公司 | Method for accessing domain name, and client terminal |
| CN101510219A (en) * | 2009-03-31 | 2009-08-19 | 成都市华为赛门铁克科技有限公司 | File data accessing method, apparatus and system |
| CN101751233A (en) * | 2009-12-31 | 2010-06-23 | 成都索贝数码科技股份有限公司 | Method and system for expanding capacity of memory device |
Cited By (73)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103024053A (en) * | 2012-12-18 | 2013-04-03 | 华为技术有限公司 | Cloud storage method, resource scheduling system and cloud storage node and system |
| CN103024053B (en) * | 2012-12-18 | 2015-11-25 | 华为技术有限公司 | Cloud storage means, resource scheduling system, cloud memory node and system |
| WO2014101240A1 (en) * | 2012-12-31 | 2014-07-03 | 华为技术有限公司 | Data storage method and apparatus, data operation method and system, and access server |
| CN103366573A (en) * | 2013-07-10 | 2013-10-23 | 中兴智能交通（无锡）有限公司 | Vehicle running information tracking method and system based on cloud computing |
| CN104754021B (en) * | 2013-12-31 | 2018-04-13 | 伊姆西公司 | Apparatus and method for promoting the access to the data in distributed memory system |
| CN104754021A (en) * | 2013-12-31 | 2015-07-01 | 伊姆西公司 | Device and method for promoting access of data in distributed storage system |
| CN104184812A (en) * | 2014-08-20 | 2014-12-03 | 四川九成信息技术有限公司 | Multi-point data transmission method based on private cloud |
| CN104184812B (en) * | 2014-08-20 | 2017-08-11 | 霍尔果斯智融未来信息科技有限公司 | A kind of multipoint data transmission method based on private clound |
| CN104239517A (en) * | 2014-09-17 | 2014-12-24 | 杭州华三通信技术有限公司 | Storage object access method and device for distributed storage system |
| CN104239517B (en) * | 2014-09-17 | 2018-05-15 | 新华三技术有限公司 | The storage object access method and device of a kind of distributed memory system |
| CN105446794A (en) * | 2014-09-30 | 2016-03-30 | 北京金山云网络技术有限公司 | Disc operation method, apparatus and system based on virtual machine |
| CN104360915A (en) * | 2014-10-31 | 2015-02-18 | 北京思特奇信息技术股份有限公司 | Data recovery method and device on basis of distributed storage |
| WO2016065613A1 (en) * | 2014-10-31 | 2016-05-06 | 华为技术有限公司 | File access method, distributed storage system and network device |
| CN104360915B (en) * | 2014-10-31 | 2017-08-01 | 北京思特奇信息技术股份有限公司 | A kind of data reconstruction method and device based on distributed storage |
| CN104484236A (en) * | 2014-11-28 | 2015-04-01 | 曙光云计算技术有限公司 | HA (high availability) access adaptation method |
| CN104484236B (en) * | 2014-11-28 | 2017-11-03 | 曙光云计算集团有限公司 | A kind of HA accesses adaptive method |
| CN104461736A (en) * | 2014-12-08 | 2015-03-25 | 畅捷通信息技术股份有限公司 | Resource allocation and search method and system and cloud server |
| CN104517067A (en) * | 2014-12-30 | 2015-04-15 | 华为技术有限公司 | Method, device and system for data access |
| CN104517067B (en) * | 2014-12-30 | 2017-12-15 | 华为技术有限公司 | Access the method, apparatus and system of data |
| CN106302607B (en) * | 2015-06-05 | 2019-08-16 | 腾讯科技（深圳）有限公司 | Block storage system and method applied to cloud computing |
| CN106302607A (en) * | 2015-06-05 | 2017-01-04 | 腾讯科技（深圳）有限公司 | It is applied to block storage system and the method for cloud computing |
| CN105187555A (en) * | 2015-09-29 | 2015-12-23 | 北京奇艺世纪科技有限公司 | Method, device and system for message processing |
| CN105704218B (en) * | 2016-01-19 | 2019-04-05 | 国家电网公司 | Data storage and management method in cloud computing platform |
| CN105704218A (en) * | 2016-01-19 | 2016-06-22 | 国家电网公司 | Data storage and management method in cloud computing platform |
| CN107092437A (en) * | 2016-02-17 | 2017-08-25 | 杭州海康威视数字技术股份有限公司 | Data write-in, read method and device, cloud storage system |
| US11586673B2 (en) | 2016-02-17 | 2023-02-21 | Hangzhou Hikvision Digital Technology Co., Ltd. | Data writing and reading method and apparatus, and cloud storage system |
| WO2018058998A1 (en) * | 2016-09-27 | 2018-04-05 | 华为技术有限公司 | Data loading method, terminal and computing cluster |
| CN106503058A (en) * | 2016-09-27 | 2017-03-15 | 华为技术有限公司 | A kind of data load method, terminal and computing cluster |
| CN106503058B (en) * | 2016-09-27 | 2019-01-18 | 华为技术有限公司 | A kind of data load method, terminal and computing cluster |
| CN108156187A (en) * | 2016-12-02 | 2018-06-12 | 航天星图科技（北京）有限公司 | A kind of cloud service system |
| CN108156187B (en) * | 2016-12-02 | 2019-04-09 | 中科星图股份有限公司 | A kind of cloud service system |
| CN106993022B (en) * | 2016-12-30 | 2020-03-31 | 中国银联股份有限公司 | Identification management method for cluster, identification server and corresponding system |
| CN106993022A (en) * | 2016-12-30 | 2017-07-28 | 中国银联股份有限公司 | For the mark management method of cluster, identification server and corresponding system |
| CN106850827A (en) * | 2017-02-24 | 2017-06-13 | 深圳市中博睿存信息技术有限公司 | A kind of metadata of distributed type file system treating method and apparatus |
| CN107346341A (en) * | 2017-07-11 | 2017-11-14 | 王焱华 | A kind of management method of big data |
| CN107562376A (en) * | 2017-08-23 | 2018-01-09 | 北京小米移动软件有限公司 | Data method for deleting and device |
| CN108197159A (en) * | 2017-12-11 | 2018-06-22 | 厦门集微科技有限公司 | Digital independent, wiring method and device based on distributed file system |
| CN108197159B (en) * | 2017-12-11 | 2020-07-10 | 厦门集微科技有限公司 | Data reading and writing method and device based on distributed file system |
| US11477067B2 (en) | 2018-03-26 | 2022-10-18 | Huawei Technologies Co., Ltd. | Packet transmission method and apparatus |
| CN110365719A (en) * | 2018-03-26 | 2019-10-22 | 华为技术有限公司 | A kind of method and relevant device of data processing |
| CN110365719B (en) * | 2018-03-26 | 2021-10-01 | 华为技术有限公司 | Data processing method and related equipment |
| CN109120709A (en) * | 2018-09-03 | 2019-01-01 | 杭州云创共享网络科技有限公司 | A kind of caching method, device, equipment and medium |
| CN109495568A (en) * | 2018-11-19 | 2019-03-19 | 北京金山云网络技术有限公司 | A kind of data capture method, data acquisition facility, server and storage medium |
| CN109495568B (en) * | 2018-11-19 | 2021-06-18 | 北京金山云网络技术有限公司 | Data acquisition method, data acquisition device, server and storage medium |
| CN113544642A (en) * | 2019-03-08 | 2021-10-22 | 国际商业机器公司 | Secure storage query and donation |
| GB2596024A (en) * | 2019-03-08 | 2021-12-15 | Ibm | Secure storage query and donation |
| JP7531509B2 (en) | 2019-03-08 | 2024-08-09 | インターナショナル・ビジネス・マシーンズ・コーポレーション | Method, system, and program for querying and providing secure storage |
| US11669462B2 (en) | 2019-03-08 | 2023-06-06 | International Business Machines Corporation | Host virtual address space for secure interface control storage |
| GB2596024B (en) * | 2019-03-08 | 2022-04-27 | Ibm | Secure storage query and donation |
| WO2020183311A1 (en) * | 2019-03-08 | 2020-09-17 | International Business Machines Corporation | Secure storage query and donation |
| US11068310B2 (en) | 2019-03-08 | 2021-07-20 | International Business Machines Corporation | Secure storage query and donation |
| US11635991B2 (en) | 2019-03-08 | 2023-04-25 | International Business Machines Corporation | Secure storage query and donation |
| JP2022522499A (en) * | 2019-03-08 | 2022-04-19 | インターナショナル・ビジネス・マシーンズ・コーポレーション | How to query and deliver secure storage, systems, programs |
| US11283800B2 (en) | 2019-03-08 | 2022-03-22 | International Business Machines Corporation | Secure interface control secure storage hardware tagging |
| US11455398B2 (en) | 2019-03-08 | 2022-09-27 | International Business Machines Corporation | Testing storage protection hardware in a secure virtual machine environment |
| US11176054B2 (en) | 2019-03-08 | 2021-11-16 | International Business Machines Corporation | Host virtual address space for secure interface control storage |
| US11182192B2 (en) | 2019-03-08 | 2021-11-23 | International Business Machines Corporation | Controlling access to secure storage of a virtual machine |
| CN109977092A (en) * | 2019-03-26 | 2019-07-05 | 武汉深之度科技有限公司 | A kind of method and calculating equipment of duplication file |
| CN109977092B (en) * | 2019-03-26 | 2021-06-08 | 武汉深之度科技有限公司 | Method and computing device for copying files |
| CN109831540A (en) * | 2019-04-12 | 2019-05-31 | 成都四方伟业软件股份有限公司 | Distributed storage method, device, electronic equipment and storage medium |
| CN110413694A (en) * | 2019-08-01 | 2019-11-05 | 重庆紫光华山智安科技有限公司 | Metadata management method and relevant apparatus |
| CN110889121A (en) * | 2019-10-12 | 2020-03-17 | 中国平安财产保险股份有限公司 | Method, server and storage medium for preventing data leakage |
| WO2021189306A1 (en) * | 2020-03-25 | 2021-09-30 | Beijing Didi Infinity Technology And Development Co., Ltd. | Write operation in object storage system using enhanced meta structure |
| CN111881470A (en) * | 2020-07-20 | 2020-11-03 | 易通星云（北京）科技发展有限公司 | Data access method and device and computer readable storage medium |
| CN111881470B (en) * | 2020-07-20 | 2024-07-12 | 易通星云（北京）科技发展有限公司 | Data access method and device and computer readable storage medium |
| CN112015696A (en) * | 2020-08-21 | 2020-12-01 | 北京奇艺世纪科技有限公司 | Data access method, data relationship setting method, data access device, data relationship setting device and storage medium |
| CN112597173A (en) * | 2021-03-08 | 2021-04-02 | 成都虚谷伟业科技有限公司 | Distributed database cluster system peer-to-peer processing system and processing method |
| CN113419671B (en) * | 2021-05-20 | 2022-02-18 | 佛山市禅城区政务服务数据管理局 | Personal data space access control method |
| CN113419671A (en) * | 2021-05-20 | 2021-09-21 | 佛山市禅城区政务服务数据管理局 | Personal data space access control method and system |
| CN115712660A (en) * | 2022-01-29 | 2023-02-24 | 杭州宇信数字科技有限公司 | Data storage method, device, server and storage medium |
| CN115712660B (en) * | 2022-01-29 | 2023-05-30 | 杭州宇信数字科技有限公司 | Data storage method, device, server and storage medium |
| CN114647559A (en) * | 2022-03-21 | 2022-06-21 | 北京百度网讯科技有限公司 | Storage usage amount statistical method and device, electronic equipment and storage medium |
| WO2024139946A1 (en) * | 2022-12-28 | 2024-07-04 | International Business Machines Corporation | Increasing resource utilization in cloud computing clusters |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102594852B (en) | 2016-03-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102594852A (en) | Data access method, node and system | |
| US11487468B2 (en) | Healing failed erasure-coded write attempts in a distributed data storage system configured with fewer storage nodes than data plus parity fragments | |
| US11314687B2 (en) | Container data mover for migrating data between distributed data storage systems integrated with application orchestrators | |
| US20210263658A1 (en) | Data system with flush views | |
| US10521135B2 (en) | Data system with data flush mechanism | |
| US9521198B1 (en) | Systems and methods for implementing an enterprise-class converged compute-network-storage appliance | |
| KR101677418B1 (en) | Prioritizing data reconstruction in distributed storage systems | |
| US11182096B1 (en) | Data storage system with configurable durability | |
| US20180024964A1 (en) | Disaggregated compute resources and storage resources in a storage system | |
| US10516645B1 (en) | Address resolution broadcasting in a networked device | |
| JP5004975B2 (en) | Data storage system | |
| CN102158540A (en) | System and method for realizing distributed database | |
| WO2019231646A1 (en) | Garbage collection implementing erasure coding | |
| CN103237046A (en) | Distributed file system supporting mixed cloud storage application and realization method thereof | |
| US11262918B1 (en) | Data storage system with uneven drive wear reduction | |
| US11681443B1 (en) | Durable data storage with snapshot storage space optimization | |
| US10230544B1 (en) | Efficient data forwarding in a networked device | |
| US8701113B2 (en) | Switch-aware parallel file system | |
| CN104023058A (en) | Method and system for managing and allocating storage resources | |
| CN102982182A (en) | Data storage planning method and device | |
| US11868334B2 (en) | Integrating change tracking of storage objects of a distributed object storage database into a distributed storage system | |
| CN109871365A (en) | A kind of distributed file system | |
| US20200401567A1 (en) | Object Storage System with Versioned Meta Objects | |
| US8868970B2 (en) | Object based storage system and method of operating thereof | |
| CN112398668B (en) | IaaS cluster-based cloud platform and node switching method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |