CN101706802A - Method, device and sever for writing, modifying and restoring data - Google Patents
Method, device and sever for writing, modifying and restoring data Download PDFInfo
- Publication number
- CN101706802A CN101706802A CN200910224819A CN200910224819A CN101706802A CN 101706802 A CN101706802 A CN 101706802A CN 200910224819 A CN200910224819 A CN 200910224819A CN 200910224819 A CN200910224819 A CN 200910224819A CN 101706802 A CN101706802 A CN 101706802A
- Authority
- CN
- China
- Prior art keywords
- data
- object server
- server
- servers
- revised
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The embodiment of the invention discloses a method, a device and a sever for writing, modifying and restoring data. The scheme provided by the embodiment of the invention starts from three basic operations of writing, modifying and restoring the data on object servers respectively, and ensures the consistency when a plurality of copies of same object data are simultaneously stored on different object servers through a series of methods, thereby greatly reducing the probability of inconsistent data between the copies, fundamentally preventing the appearance of a single copy, and greatly improving the reliability of a distributed file system.
Description
Technical field
The present invention relates to the distributed system technical field, relate in particular to method, device and server that a kind of data write, revise and recover.
Background technology
Along with computer network, the particularly fast development of Internet, huge variation has taken place in traditional infosystem notion, and these variations show storage, transmission, the issue of information and the revolutionary change that obtain manner took place highlightedly.Meanwhile, based on network distributed information system has obtained using widely in every field, is just bringing into play the effect that becomes increasingly conspicuous in entire society's life.Wherein, distributed file system begins to be widely used in the storage solution.
Distributed file system and traditional disk array have very big difference at storage mode.What disk array adopted is the mode of centralized stores, and all data all concentrate on in an array, realizes the backup and the recovery of data by controller and RAID.Distributed file system is then different, in distributed file system, in order to guarantee the reliability of data, data object need be carried out redundancy.A kind of typical method is that an object tools is being become a plurality of copies, then these copies is stored on the different object storage servers.
The inventor is by discovering, though employed method can guarantee the reliability of data in the existing distributed file system, also brings a problem, i.e. data consistency problem between the copy.Because a variety of causes, in distributed system, may cause the data of copy inconsistent, imperfectly in time do not found but occur when for example on copy, writing data interrupting causing certain copy in certain object server to write; Perhaps, when needs are made amendment to certain data, when the copy on a plurality of object servers is made amendment, break down on certain object server and cause the data modification failure on this object server but in time do not found; Perhaps, in time do not found, thereby cause the copy of storing it in time not being recovered or the like but the object server of having stored certain copy breaks down to run well.Because each distributed file system is made up of a plurality of memory nodes,, make that the scale of distributed file system is increasing along with increasing of distributed file system memory node number.The probability that memory node breaks down in the so huge distributed file system is also increasing, thereby causes the data consistency of same target copy on different memory nodes also more and more to be difficult to guarantee.
Summary of the invention
In view of this, the method, device and the server that provide a kind of data to write, revise and recover is provided the purpose of the embodiment of the invention, to guarantee the data consistency of same object copy on different memory nodes.
For achieving the above object, the embodiment of the invention provides following technical scheme:
A kind of method for writing data is used for comprising the distributed file system of at least two object servers, and this method comprises:
When receiving the write operation requests that client sends, layout information is sent to described client, comprise data to be written in the write operation requests in the described layout information with the object server information that is written into;
Indicate described client described data to be written to be write first object server that writes down in the described layout information according to described layout information, after described client is finished write operation, indicate described first object server described data to be written to be write second object server that writes down in the described layout information;
Indicate described client to send and write the results verification request, and receive the result that writes that described first object server and described second object server return to described first object server and described second object server.
A kind of data write server, comprising:
Transmitting element is used for when receiving the write operation requests that client sends layout information being returned to described client, comprises data to be written in the write operation requests in the described layout information with the object server information that is written into;
First indicating member, be used for indicating described client that data to be written are write first object server that described layout information writes down according to described layout information, after described client is finished write operation, indicate described first object server described data to be written to be write second object server that writes down in the described layout information.
Second indicating member is used to indicate described client to send to described first object server and described second object server and writes the results verification request, and receives the result that writes that described first object server and described second object server return.
A kind of data modification method is used for comprising the distributed file system of at least two object servers, and this method comprises:
Whether at least two object servers that detection waits to revise the data place are normal condition;
When described at least two object servers are normal condition, the data to be revised on described at least two object servers are locked, and guarantee to lock successfully;
Data to be revised on described at least two object servers are made amendment;
Releasing is to described at least two lock-out states of waiting to revise data.
A kind of data modifying apparatus is used for comprising the distributed file system of at least two object servers, and this device comprises:
Detecting unit is used to detect whether at least two object servers waiting to revise the data place are normal condition;
Lock cell is used for when described object server is normal condition, the data to be revised on described at least two object servers is locked, and guarantee to lock successfully;
Revise the unit, be used for the data to be revised on described at least two object servers are made amendment;
Separate lock unit, be used to remove the lock-out state of waiting to revise data to described at least two.
A kind of data reconstruction method is used for comprising the distributed file system of at least two object servers, and this method comprises:
Detect the operation conditions of described at least two object servers by the heartbeat inspection;
In the time of after in finding described at least two object servers of fault, having the fault object server, determine object data to be recovered according to layout information;
Send recovery message to the destination object server, make the destination object server recover the described data recovered for the treatment of, write down the sign of described destination object server in the described recovery message and treat data recovered;
Described destination object server according to described recovery message to described when treating that restore data is recovered successfully, according to described destination object server identification and the data that are resumed described layout information is upgraded.
A kind of data are recovered server, are used for comprising the distributed file system of at least two object servers, and this server comprises:
Detecting unit is used for the operation conditions by described at least two object servers of heartbeat inspection detection;
Determining unit is used for determining to treat data recovered according to layout information when there is the fault object server in described at least two object servers;
Transmitting element is used for sending recovery message to the destination object server, makes the destination object server recover the described data recovered for the treatment of, writes down the sign of described destination object server in the described recovery message and treats data recovered;
Updating block, be used for described destination object server according to described recovery message to described when treating that restore data is recovered successfully, according to described destination object server identification and the data that are resumed described layout information is upgraded.
As seen, the scheme that the embodiment of the invention the provided data from the object server respectively writes, revises and recovers three basic operations, consistance when guaranteeing that by a series of method a plurality of copies of same object data are stored on the different object servers simultaneously, greatly reduced the inconsistent possibility of data between copy, fundamentally prevent the situation that single copy occurs, improved the reliability of distributed file system greatly.
Description of drawings
In order to be illustrated more clearly in the technical scheme in the embodiment of the present application, the accompanying drawing of required use is done to introduce simply in will describing embodiment below, apparently, the accompanying drawing that describes below only is some embodiment that put down in writing among the application, for those of ordinary skills, under the prerequisite of not paying creative work, can also obtain other accompanying drawing according to these accompanying drawings.
Fig. 1 is the process flow diagram of the method that one embodiment of the invention provided;
Fig. 2 is the Signalling exchange figure of the method that one embodiment of the invention provided;
Fig. 3 is the structural representation of the server that one embodiment of the invention provided;
Fig. 4 is the process flow diagram of the other method that one embodiment of the invention provided;
Fig. 5 is the process flow diagram of a substep in the method that one embodiment of the invention provided;
Fig. 6 is the synoptic diagram of a scene in the method that one embodiment of the invention provided;
Fig. 7 is the structural representation of the device that one embodiment of the invention provided;
Fig. 8 is the structural representation of a unit in the device that another embodiment of the present invention provided;
Fig. 9 is the structural representation of a unit in the device that further embodiment of this invention provided;
Figure 10 is the structural representation of a unit in the device that another embodiment of the present invention provided;
Figure 11 is the process flow diagram of the other method that one embodiment of the invention provided;
Figure 12 is the structural representation of the server that another embodiment of the present invention provided.
Embodiment
The embodiment of the invention discloses the methods, devices and systems that a kind of data write, revise and recover, for making purpose of the present invention, technical scheme and advantage clearer, below with reference to the accompanying drawing embodiment that develops simultaneously, the present invention is described in further detail.
In distributed file system,, data object need be carried out redundancy backup in order to guarantee the reliability of data.A kind of typical method is to a plurality of copies of object tools, then these copies is stored on the different object servers.This method can guarantee the reliability of data, but also proposes a problem, i.e. data consistency problem between the copy.Have only the data between each copy to be consistent, the existence of the copy that these are redundant is only valuable.Because that copy data is write fashionable fault etc. is numerous, there is inconsistent risk in the data of a plurality of copies in the distributed system but in fact.The method that the embodiment of the invention is passed through is in order to realize in the distributed system that the consistance of data proposes in the different object servers.
The inventor is by discovering prior art, influence in the distributed system to same data in the different object servers inconsistent appear at mainly that data write, in the process such as data modification and data recovery, below, the method that the embodiment of the invention provided is described in detail at these several processes respectively.
Referring to Fig. 1, method for writing data that one embodiment of the invention provides is used for comprising the distributed file system of at least two object servers, and this method comprises:
S101, when receiving the write operation requests that client sends, layout information is returned to described client.
The related ablation process of the method that the embodiment of the invention provided is all finished according to layout information.And this layout information is to be formed by the write operation requests of meta data server according to client in an embodiment of the present invention.Be that the write operation that the embodiment of the invention provides all is to be realized by layout information control by meta data server.
In the method that the embodiment of the invention provided, layout information has consequence, be equivalent to the DATA DISTRIBUTION map in the distributed file system, write down the distribution situation of each data in the distributed file system, also comprise data to be written with the object server information that is written into certainly;
For example in a distributed file system three object servers are arranged, be numbered 1～3, so for a data object Data1 to be written, the information that may write down in the layout information is Data1～object server 1, object server 3.This just represents and this data object of Data1 need be written in object server 1 and the object server 3.
S102, indicate described client described data to be written to be write first object server that writes down in the described layout information according to described layout information, after described client is finished write operation, indicate described first object server described data to be written to be write second object server that writes down in the described layout information.S103, the described client of indication send to described first object server and described second object server and write the results verification request, and receive the result that writes that described first object server and described second object server return.
The ablation process of the embodiment of the invention writes for order, data to be written will be written in two object servers in the current distributed file system at least, after client is finished the work that writes in first object server, again data to be written are written in second object server by first object server.
Here, represent at first to write the object server of data, represent to write the server of data with second object server by first object server by client with first object server.
As seen, the method that the embodiment of the invention provided is applied to comprise at least in the distributed file system of two object servers, this method order in first object server and second object server writes data and writes data affirmation process afterwards and guarantees write operation success simultaneously on first object server and second object server, thereby fundamentally avoided the appearance of single copy data, guaranteed the consistance of the copy of the same data that first object server and second object server are preserved in the write operation process.
In the practical application, when described first object server finish on described second object server write work after, can also for example write work on the 3rd object server at other object servers, be equivalent to be written in parallel to.
Ablation process with above-mentioned Data1 is an example, is first object server with object server 1, and object server 2 and 3 is second object server and the 3rd object server, and then client writes Data1 at object server 1 earlier; Object server 1 writes Data1 respectively on object server 2 and object server 3.
In other embodiments, when first object server finish on second object server write work after, also can continue at other object servers by second object server, write work as continuing to carry out data on the 3rd object server.
Ablation process with above-mentioned Data1 is an example, when client writes Data1 at object server 1; Object server 1 writes Data1 on object server 2, object server 2 continues to write Data1. on object server 3 and realized the serial ablation process.
Need to prove, the method that the embodiment of the invention provided, no matter be data to have been write N object server (N is more than or equal to 3) by parallel mode or employing serial mode, described client all will send to described N object server according to indication write the results verification request, and receives the result that writes that described N object server returns.
In the practical application, the situation that first object server or second object server write failure may appear, in the method that the embodiment of the invention provided, when described first object server or described second object server return when writing data failure, meta data server indicates described first object server and second object server to re-execute described data write operation.
Alternatively, when having the N object server,, then re-execute N the write operation on the object server if in the N object server, write failure.
As seen, the method that the embodiment of the invention provided is applied to comprise at least in the distributed file system of two object servers, and this method writes data by order at least two object servers and writes data affirmation process afterwards and guarantees write operation success simultaneously at least two object servers.If unsuccessful situation is then all rewriteeing on the two-server at least, fundamentally avoided the appearance of isolated copy.
Fig. 2 is the application synoptic diagram of the present invention that one embodiment of the invention provided, and comprises client, meta data server, object server 1 and object server 2 among the figure.
Meta data server is the Core server in this example, and all write activities are all cooperated with different object servers according to layout information indication client by meta data server to be finished, and specifically comprises:
S201, client send data to meta data server and write request, and these data write record data object to be written in the request.
S202, meta data server return to client with layout information.
Comprise the information of these data to be written in the layout information, for example sign of object server or the like with the object server that is written into.For convenience, hypothesis data to be written need be written into object server 1 and object server 2 in the embodiment of the invention.
S203, client to first given object server write data, and is waited for the performance of write operation according to layout information.
S204, after client is finished write operation at first object server, first object server writes data into second object server, and waits for the performance of write operation;
S205, second object server return to first object server with the performance of write operation.
S206, first object server return to client with all results of write operation.
S207, client confirm to write the data success to first server.
S208, first object server confirm to write the data success to second object server.
S209, second object server return the result who writes success and return to first object server;
The result that S210, first object server will write success returns to client.
By the embodiment of the invention as can be seen, the method that the embodiment of the invention provided successfully guarantees the consistance of the copy of the same data of preserving on the different object servers in the write operation process by same object at the write operation of at least two different object servers.
Another embodiment of the present invention also provides a kind of data to write server, and referring to Fig. 3, this server comprises:
First transmitting element 301 is used for when receiving the write operation requests of client transmission layout information being returned to described client, comprises data to be written in the described layout information with the object server information that is written into;
First indicating member 302, be used for indicating described client that data to be written are write first object server that described layout information writes down according to described layout information, and indicate described first object server after described client is finished write operation, described data to be written are write second object server that writes down in the described layout information.
Second indicating member 303 is used to indicate described client to send to described first object server and described second object server and writes the results verification request, and receives the result that writes that described first object server and described second object server return.
Alternatively, in another embodiment of the present invention, server shown in Figure 3 also comprises
The 3rd indicating member 304 is used to indicate described first object server after described client is finished write operation, and described data to be written are write the N object server that writes down in the described layout information, and N is the natural number greater than 3;
Perhaps, be used to indicate described second object server after described first object server is finished write operation, described data to be written are write the N object server that writes down in the described layout information, N is the natural number greater than 3.
Accordingly, described second indicating member 303 also is used to indicate described client to send to described N object server and writes the results verification request, and receives the result that writes that described N object server returns.
Alternatively, among other embodiment of the present invention, server shown in Figure 3 can also comprise:
First control module is used for returning when writing data failure at described first object server or described second object server, indicates described first object server and second object server to re-execute described data write operation.
The server that the embodiment of the invention provided is controlled client and the object server corresponding with data to be written according to layout information, the mode of indicating each client and object server to write in order writes data on first object server and second object server, and confirm the execution result of write operation, on two different object servers, write the inconsistent situation of data in the copy of successfully avoiding same data correspondence on the different object servers at least by 2 copies that guarantee same target.
Referring to Fig. 4, one embodiment of the invention also provides a kind of data modification method, and this method is used for comprising the distributed file system of at least two object servers, comprising:
S401, detect whether at least two object servers waiting to revise the data place are normal condition;
S402, when described object server is normal condition, the data to be revised on described at least two object servers are locked, and guarantee to lock successfully;
In the embodiment of the invention, treating the modification data and lock, is in order to prevent in the same moment appearance that has a plurality of main bodys that the same object that is positioned on the different server is all made amendment and operated and lead to errors.With on two servers simultaneously the record an electrical form be example.When if the party A-subscriber operates the electrical form on this S1 server, the party B-subscriber also continues operation to this electrical form on the S2 server, will cause destroying the consistance of same electrical form on the different server so.So when the party A-subscriber will operate the electrical form on the S1 server, the electrical form on the S2 server is locked, make it for lock-out state, thereby avoid a plurality of users simultaneously same file to be operated.
Treat that to revise the data success that locks and will guarantee to lock necessary to the consistance that guarantees data.If do not lock, the situation that the same data in the same object server are revised by a plurality of clients simultaneously then may appear, and conflict appears, cause the data modification failure the most at last.
S403, the data to be revised on described at least two object servers are made amendment;
S404 removes described two lock-out states of waiting to revise data at least.
The amending method of the data trnascription that the embodiment of the invention provided, when definite at least two object servers are normal condition, data to be revised on described at least two object servers are locked, under lock-out state, the data to be revised at least two object servers are made amendment, 2 copies that guaranteed same target are revised successfully on two different object servers at least, fundamentally avoid the appearance of single copy data, guaranteed in the data modification process consistance of the copy of the same data of preserving at least two object servers.
Alternatively, referring to Fig. 5, in the above-described embodiments, the data to be revised on described at least two object servers are made amendment to be comprised:
S501, the data to be revised on described at least two object servers are copied to client;
S502, data described to be revised are made amendment in client;
S503, amended data are write described at least two object servers.
When the data write step of execution in step S503, can carry out according to the method for writing data that the embodiment of the invention provides previously.
After retouching operation is finished, need to remove lock-out state to data, make other terminal to operate to this data.
Alternatively, in the embodiment of the invention, the data to be revised on described at least two object servers are locked and can carry out synchronously, comprising:
Check whether treating on described at least two object servers revised data is locked state;
When the data to be revised on described at least two object servers are when not locked, the data to be revised on described at least two object servers are locked.
In other embodiments, the data to be revised on described at least two object servers are locked also can carry out in proper order, comprising:
After locking for data to be revised on first object server in described at least two object servers, again the data to be revised on first object server at least two object servers are locked.
When mode that employing locks in proper order is that data to be revised on the object server are when locking, the situation that deadlock may occur, referring to Fig. 6, promptly behind obj-1 on the client-1 lock object server-1, when preparing the obj-1 on the lock object server-2, the obj-1 on the object server-2 is locked by other clients at this moment.
If deadlock situation, the solution that one embodiment of the invention provided is:
After locking for data to be revised on first object server in described at least two object servers, detect treating on second object server in described at least two object servers and revise data and whether be in locked state;
If then the lock up-to-date on the data of waiting to revise on described second object server is carried out release, and the client that notice successfully locks at first locks to data described to be revised; Otherwise, the data to be revised on second object server in described at least two object servers are locked.
Compare by the priority of timestamp in the embodiment of the invention added lock on the identical file on the different object servers.For example the unlock method of deadlock situation is described below:
For example, an object 1 is arranged on the object server 1, also there is object 1. on the object server 2 and the object on the object server 11 locked (be designated as lock 1) back when preparing the object on the object server 21 locked as user A, find its locked (being designated as lock 2) by detecting by object server 2, obviously, if do not do corresponding processing, object 1 on object server 1 and the object server all can't be processed, so cause the appearance of deadlock like this, the embodiment of the invention is carried out release by the timestamp of same object data on the more different object servers. supposes, the timestamp of lock 1 is 10:01, the time lock of lock 2 is 10:02, illustrate that promptly user A locks to object 1 prior to user B. then according to the judgement of timestamp, up-to-date lock-the lock 2 of timestamp is released, and the client of the success that locks at first is retained. and notify the described client that successfully locks at first that the object on the object server 21 is locked and get final product.
The amending method of the data trnascription that the embodiment of the invention provided, make amendment by treating under the lock-out state of revising data, 2 copies that guaranteed same target are revised the inconsistent situation of data in the copy of successfully avoiding same data correspondence on the different object servers at least on two different object servers.
Referring to Fig. 7, one embodiment of the invention also provides a kind of data modifying apparatus, and this device is used for comprising the distributed file system of at least two object servers, comprising:
First detecting unit 701 is used to detect whether at least two object servers waiting to revise the data place are normal condition;
Revise unit 703, be used for the data to be revised on described at least two object servers are made amendment;
Alternatively, referring to Fig. 8, treating on each object server revised the locking of data and can carry out synchronously, particularly, described lock cell 702 comprises:
First checks subelement 801, is used to check whether treating on described at least two object servers revised data is locked state;
The first locking subelement 802 is used for being when not locked when the data to be revised on described at least two object servers, and the data to be revised on described at least two object servers are locked.
In another embodiment of the present invention, referring to Fig. 9, treating on each object server revised the locking of data and can carry out in proper order, particularly, described lock cell 702 comprises:
The second locking subelement 901 is used for locking to the data to be revised on first object server on described at least two object servers;
Second checks subelement 902, is used for detecting waiting whether revise data is in locked state on second object server of described at least two object servers;
The 3rd locking subelement 903, when being used for data to be revised on described second object server and being in locked state the lock up-to-date on the data of waiting to revise on described second object server is carried out release, and the client that notice successfully locks at first locks to data described to be revised;
The 4th locking subelement 904 locks to the data to be revised on second object server of described two object servers when being used for treating on described second object server and revising data and be not in locked state at least.
Referring to Figure 10, in one embodiment of the invention, described modification unit 703 comprises:
Revise subelement 1002, be used for data described to be revised being made amendment in client;
Write subelement 1003, be used for amended data are write described at least two object servers.
The modifier of the data trnascription that the embodiment of the invention provided, make amendment by treating under the lock-out state of revising data, 2 copies that guaranteed same target are revised the inconsistent situation of data in the copy of successfully avoiding same data correspondence on the different object servers at least on two different object servers.
One embodiment of the invention also provides a kind of data reconstruction method, is used for comprising the distributed file system of at least two object servers, and referring to Figure 11, this method comprises:
S1101, detect the operation conditions of described at least two object servers by the heartbeat inspection;
S1102, after finding the fault object server, determine object to be recovered according to layout information;
Write down the information of copy of the object of this fault object server stores in the layout information, because fault has taken place this object server, so the copy unloading that needs all objects that will store on this object server is to other object servers.
S1103, send to recover message, the described destination server sign of record and treat data recovered in the described recovery message to the destination object server;
In the embodiment of the invention copy will be called the destination object server by the object server of unloading.Destination server can be other object servers that do not break down in the current distributed file system, also can be an object server outside the current distributed file system.
S1104, described destination object server according to described recovery message to described when treating that restore data is recovered successfully, according to described destination object server identification and the data that are resumed described layout information is upgraded.
Described destination object server will treat that according to described recovery message restore data is written to this locality exactly to the described recovery operation essence of restore data for the treatment of.
In other embodiments of the invention, may carry out not success of restore data first on first destination server, at this moment, method shown in Figure 11 also comprises:
, send to described destination object server again and recover message when treating that restore data is recovered failure described according to described recovery message at described destination object server.
In the practical application, can control the number of times that sends recovering information again to described destination object server by default number of retries.
Effectively avoided operating situation by retry mechanism,, saved the processing time, improved treatment effeciency by automatic retry once the return results of failing.
The data reconstruction method that the embodiment of the invention provided, when being checked through the fault object server, in time in destination server, recover the copy of the object preserved in the fault object server again, simultaneously layout information is upgraded, thereby avoided occurring copy inconsistent situation on different object servers of same target because of the individual objects server failure.
Referring to Figure 12, another embodiment of the present invention also provides a kind of data to recover server, and this server is used for comprising the distributed file system of at least two object servers, comprising:
Second detecting unit 1201 is used for the operation conditions by described at least two object servers of heartbeat inspection detection;
Determining unit 1202 is used for determining object to be recovered according to the information of layout information and described fault object server after finding the fault object server;
Updating block 1204, be used for described destination object server according to described recovery message to described when treating that restore data is recovered successfully, according to described destination object server identification and the data that are resumed described layout information is upgraded.
Alternatively, if carry out restore data on first destination server during success, server shown in Figure 12 also comprises:
Second control module, be used for described destination object server according to described recovery message to described when treating that restore data is recovered failure, control described transmitting element and repeat to the destination object server and send the action that recovers message.
The data that the embodiment of the invention provided are recovered server, when being checked through the fault object server, in time in destination server, recover the copy of the object preserved in the fault object server again, simultaneously layout information is upgraded, thereby avoided occurring copy inconsistent situation on different object servers of same target because of the individual objects server failure.
The present invention can describe in the general context of the computer executable instructions of being carried out by computing machine, for example program module.Usually, program module comprises the routine carrying out particular task or realize particular abstract, program, object, assembly, data structure or the like.Also can in distributed computing environment, put into practice the present invention, in these distributed computing environment, by by communication network connected teleprocessing equipment execute the task.In distributed computing environment, program module can be arranged in the local and remote computer-readable storage medium that comprises memory device.
The above only is a preferred implementation of the present invention; should be pointed out that for those skilled in the art, under the prerequisite that does not break away from the principle of the invention; can also make some improvements and modifications, these improvements and modifications also should be considered as protection scope of the present invention.
Claims (18)
1. a method for writing data is characterized in that, is used for comprising the distributed file system of at least two object servers, and this method comprises:
When receiving the write operation requests that client sends, layout information is sent to described client, comprise data to be written in the write operation requests in the described layout information with the object server information that is written into;
Indicate described client described data to be written to be write first object server that writes down in the described layout information according to described layout information, after described client is finished write operation, indicate described first object server described data to be written to be write second object server that writes down in the described layout information;
Indicate described client to send and write the results verification request, and receive the result that writes that described first object server and described second object server return to described first object server and described second object server.
2. method according to claim 1 is characterized in that, also comprises:
After described client is finished write operation, indicate the described first object server write operation described data to be written to be write the 3rd object server that writes down in the described layout information;
Perhaps,
After described first object server is finished described write operation, indicate the described second object server write operation described data to be written to be write the 3rd object server that writes down in the described layout information;
Correspondingly, this method also comprises:
Indicate described client to send and write the results verification request, and receive the result that writes that described the 3rd object server returns to described the 3rd object server.
3. method according to claim 1 is characterized in that, also comprises:
When described first object server or described second object server return when writing data failure, indicate described first object server and second object server to re-execute write operation.
4. data write server, it is characterized in that, comprising:
First transmitting element is used for when receiving the write operation requests that client sends layout information being returned to described client, comprises data to be written in the write operation requests in the described layout information with the object server information that is written into;
First indicating member, be used for indicating described client that data to be written are write first object server that described layout information writes down according to described layout information, after described client is finished write operation, indicate described first object server described data to be written to be write second object server that writes down in the described layout information;
Second indicating member is used to indicate described client to send to described first object server and described second object server and writes the results verification request, and receives the result that writes that described first object server and described second object server return.
5. data according to claim 4 write server, it is characterized in that, also comprise:
The 3rd indicating member is used for after described client is finished write operation, indicates described first object server described data to be written to be write the 3rd object server that writes down in the described layout information;
Perhaps, be used for after described first object server is finished write operation, indicate described second object server described data to be written to be write the 3rd object server that writes down in the described layout information;
Accordingly, described second indicating member also is used to indicate described client to send to described the 3rd object server and writes the results verification request, and receives the result that writes that described the 3rd object server returns.
6. data according to claim 4 write server, it is characterized in that, also comprise:
First control module is used for returning when writing data failure at described first object server or described second object server, indicates described first object server and second object server to re-execute described data write operation.
7. a data modification method is characterized in that, is used for comprising the distributed file system of at least two object servers, and this method comprises:
Whether at least two object servers that detection waits to revise the data place are normal condition;
When described at least two object servers are normal condition, the data to be revised on described at least two object servers are locked, and guarantee to lock successfully;
Data to be revised on described at least two object servers are made amendment;
Releasing is to described at least two lock-out states of waiting to revise data.
8. method according to claim 7 is characterized in that, the data to be revised on described at least two object servers are locked to be comprised:
Check whether treating on described at least two object servers revised data is locked state;
When the data to be revised on described at least two object servers are when not locked, the data to be revised on described at least two object servers are locked.
9. method according to claim 7 is characterized in that, the data to be revised on described at least two object servers are locked to be comprised:
Lock for the data to be revised on first object server in described at least two object servers;
Detecting treating on second object server in described at least two object servers revises data and whether is in locked state;
If then the lock up-to-date on the data of waiting to revise on described second object server is carried out release, and the client that notice successfully locks at first locks to data described to be revised;
Otherwise, the data to be revised on second object server in described at least two object servers are locked.
10. according to any described method of claim 7～9, it is characterized in that the data to be revised on described at least two object servers are made amendment to be comprised:
Data to be revised on described at least two object servers are copied to client;
In client data described to be revised are made amendment;
Amended data are write described at least two object servers.
11. a data modifying apparatus is characterized in that, is used for comprising the distributed file system of at least two object servers, this device comprises:
First detecting unit is used to detect whether at least two object servers waiting to revise the data place are normal condition;
Lock cell is used for when described object server is normal condition, the data to be revised on described at least two object servers is locked, and guarantee to lock successfully;
Revise the unit, be used for the data to be revised on described at least two object servers are made amendment;
Separate lock unit, be used to remove the lock-out state of waiting to revise data to described at least two.
12. device according to claim 11 is characterized in that, described lock cell comprises:
First checks subelement, is used to check whether treating on described at least two object servers revised data is locked state;
The first locking subelement is used for being when not locked when the data to be revised on described at least two object servers, and the data to be revised on described at least two object servers are locked.
13. device according to claim 11 is characterized in that, described lock cell comprises:
The second locking subelement is used for locking to the data to be revised on first object server of described at least two object servers;
Second checks subelement, is used for detecting waiting whether revise data is in locked state on second object server of described at least two object servers;
The 3rd locking subelement, when being used for data to be revised on described second object server and being in locked state the lock up-to-date on the data of waiting to revise on described second object server is carried out release, and the client that notice successfully locks at first locks to data described to be revised;
The 4th locking subelement locks to the data to be revised on second object server of described two object servers when being used for treating on described second object server and revising data and be not in locked state at least.
14., it is characterized in that described modification unit comprises according to any described device of claim 11～13:
The replicon unit is used for the data to be revised on described at least two object servers are copied to client;
Revise subelement, be used for data described to be revised being made amendment in client;
Write subelement, be used for amended data are write described at least two object servers.
15. a data reconstruction method is characterized in that, is used for comprising the distributed file system of at least two object servers, this method comprises:
Detect the operation conditions of described at least two object servers by the heartbeat inspection;
When having the fault object server in described at least two object servers, determine to treat data recovered according to layout information;
Send recovery message to the destination object server, make the destination object server recover the described data recovered for the treatment of, write down the sign of described destination object server in the described recovery message and treat data recovered;
Described destination object server according to described recovery message to described when treating that restore data is recovered successfully, according to described destination object server identification and the data that are resumed described layout information is upgraded.
16. method according to claim 15 is characterized in that, also comprises:
, send to described destination object server again and recover message when treating that restore data is recovered failure described according to described recovery message at described destination object server.
17. data are recovered server, it is characterized in that be used for comprising the distributed file system of at least two object servers, this server comprises:
Second detecting unit is used for the operation conditions by described at least two object servers of heartbeat inspection detection;
Determining unit is used for determining to treat data recovered according to layout information when there is the fault object server in described at least two object servers;
Second transmitting element is used for sending recovery message to the destination object server, makes the destination object server recover the described data recovered for the treatment of, writes down the sign of described destination object server in the described recovery message and treats data recovered;
Updating block, be used for described destination object server according to described recovery message to described when treating that restore data is recovered successfully, according to described destination object server identification and the data that are resumed described layout information is upgraded.
18. server according to claim 17 is characterized in that, also comprises:
Second control module, be used for described destination object server according to described recovery message to described when treating that restore data is recovered failure, control described transmitting element and repeat to the destination object server and send the action that recovers message.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200910224819.5A CN101706802B (en) | 2009-11-24 | 2009-11-24 | Method, device and sever for writing, modifying and restoring data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200910224819.5A CN101706802B (en) | 2009-11-24 | 2009-11-24 | Method, device and sever for writing, modifying and restoring data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101706802A true CN101706802A (en) | 2010-05-12 |
| CN101706802B CN101706802B (en) | 2013-06-05 |
Family
ID=42377027
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN200910224819.5A Active CN101706802B (en) | 2009-11-24 | 2009-11-24 | Method, device and sever for writing, modifying and restoring data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101706802B (en) |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102088484A (en) * | 2010-12-20 | 2011-06-08 | 南京中兴新软件有限责任公司 | Write lock method of distributed file system and a system thereof |
| CN102122288A (en) * | 2010-12-21 | 2011-07-13 | 北京高森明晨信息科技有限公司 | Method and system for concurrency control |
| CN102368268A (en) * | 2011-10-25 | 2012-03-07 | 无锡城市云计算中心有限公司 | Method for achieving consistency of pieces of metadata |
| CN103064635A (en) * | 2012-12-19 | 2013-04-24 | 华为技术有限公司 | Distributed storage method and device |
| CN103095758A (en) * | 2011-11-01 | 2013-05-08 | 北大方正集团有限公司 | Distributed file system and file data processing method thereof |
| CN103392167A (en) * | 2012-12-28 | 2013-11-13 | 华为技术有限公司 | Caching method of distributed storage system, nodes and computer readable medium |
| CN103905512A (en) * | 2012-12-28 | 2014-07-02 | 中国移动通信集团公司 | Data processing method and equipment |
| WO2014177085A1 (en) * | 2013-09-17 | 2014-11-06 | 中兴通讯股份有限公司 | Distributed multicopy data storage method and device |
| CN104461768A (en) * | 2013-09-22 | 2015-03-25 | 华为技术有限公司 | Copy storing device and copy storing method |
| CN104536919A (en) * | 2014-12-11 | 2015-04-22 | 浪潮(北京)电子信息产业有限公司 | Method and device both for avoiding IO conflict in cloning system |
| CN106372030A (en) * | 2016-10-27 | 2017-02-01 | 郑州云海信息技术有限公司 | Method and device for data synchronization of dual controller storage system |
| CN107347090A (en) * | 2017-06-08 | 2017-11-14 | 广州市呼百应网络技术股份有限公司 | Method of data synchronization |
| CN107507001A (en) * | 2017-08-11 | 2017-12-22 | 杭州秘猿科技有限公司 | A kind of data revision method based on license chain |
| CN107885488A (en) * | 2016-09-29 | 2018-04-06 | 微软技术许可有限责任公司 | It is intended to and has intentional lock |
| CN108228698A (en) * | 2016-12-09 | 2018-06-29 | 谷歌有限责任公司 | For the high-throughput algorithm of the Multi version concurrency control with the global synchronization time |
| CN109428843A (en) * | 2017-08-30 | 2019-03-05 | 英特尔公司 | The technology that efficient result shifts is provided from the accelerator installation in depolymerization framework |
| WO2019073324A1 (en) * | 2017-10-12 | 2019-04-18 | International Business Machines Corporation | File system content based security |
| CN110059514A (en) * | 2019-04-18 | 2019-07-26 | 珠海美佳音科技有限公司 | Method for writing data, NFC label, NFC device and storage medium |
| CN110399383A (en) * | 2019-07-29 | 2019-11-01 | 中国工商银行股份有限公司 | Applied to the data processing method of server, device, calculate equipment, medium |
| WO2021078222A1 (en) * | 2019-10-25 | 2021-04-29 | 北京金山云网络技术有限公司 | Determination method and apparatus for master copy files of object in object storage system |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7552223B1 (en) * | 2002-09-16 | 2009-06-23 | Netapp, Inc. | Apparatus and method for data consistency in a proxy cache |
| CN100517335C (en) * | 2007-10-25 | 2009-07-22 | 中国科学院计算技术研究所 | Distributed file system file writing system and method |
| CN101187931A (en) * | 2007-12-12 | 2008-05-28 | 浙江大学 | Distribution type file system multi-file copy management method |
-
2009
- 2009-11-24 CN CN200910224819.5A patent/CN101706802B/en active Active
Cited By (40)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102088484B (en) * | 2010-12-20 | 2014-12-10 | 南京中兴新软件有限责任公司 | Write lock method of distributed file system and a system thereof |
| WO2012083797A1 (en) * | 2010-12-20 | 2012-06-28 | 中兴通讯股份有限公司 | Write lock method and system for distributed file system |
| CN102088484A (en) * | 2010-12-20 | 2011-06-08 | 南京中兴新软件有限责任公司 | Write lock method of distributed file system and a system thereof |
| CN102122288A (en) * | 2010-12-21 | 2011-07-13 | 北京高森明晨信息科技有限公司 | Method and system for concurrency control |
| CN102368268A (en) * | 2011-10-25 | 2012-03-07 | 无锡城市云计算中心有限公司 | Method for achieving consistency of pieces of metadata |
| CN102368268B (en) * | 2011-10-25 | 2013-06-12 | 无锡城市云计算中心有限公司 | Method for achieving consistency of pieces of metadata |
| CN103095758A (en) * | 2011-11-01 | 2013-05-08 | 北大方正集团有限公司 | Distributed file system and file data processing method thereof |
| CN103064635A (en) * | 2012-12-19 | 2013-04-24 | 华为技术有限公司 | Distributed storage method and device |
| CN103064635B (en) * | 2012-12-19 | 2016-08-24 | 华为技术有限公司 | Distributed storage method and distributed storage devices |
| CN103905512A (en) * | 2012-12-28 | 2014-07-02 | 中国移动通信集团公司 | Data processing method and equipment |
| WO2014101108A1 (en) * | 2012-12-28 | 2014-07-03 | 华为技术有限公司 | Caching method for distributed storage system, node and computer readable medium |
| CN103392167A (en) * | 2012-12-28 | 2013-11-13 | 华为技术有限公司 | Caching method of distributed storage system, nodes and computer readable medium |
| CN103905512B (en) * | 2012-12-28 | 2017-06-20 | 中国移动通信集团公司 | A kind of data processing method and equipment |
| US9424204B2 (en) | 2012-12-28 | 2016-08-23 | Huawei Technologies Co., Ltd. | Caching method for distributed storage system, a lock server node, and a lock client node |
| WO2014177085A1 (en) * | 2013-09-17 | 2014-11-06 | 中兴通讯股份有限公司 | Distributed multicopy data storage method and device |
| CN104468651A (en) * | 2013-09-17 | 2015-03-25 | 南京中兴新软件有限责任公司 | Distributed multi-copy storage method and device |
| CN104468651B (en) * | 2013-09-17 | 2019-09-10 | 南京中兴新软件有限责任公司 | Distributed more copy data storage methods and device |
| WO2015039569A1 (en) * | 2013-09-22 | 2015-03-26 | 华为技术有限公司 | Copy storage device and copy storage method |
| CN104461768A (en) * | 2013-09-22 | 2015-03-25 | 华为技术有限公司 | Copy storing device and copy storing method |
| CN104461768B (en) * | 2013-09-22 | 2018-08-14 | 华为技术有限公司 | copy storage device and copy storage method |
| CN104536919A (en) * | 2014-12-11 | 2015-04-22 | 浪潮(北京)电子信息产业有限公司 | Method and device both for avoiding IO conflict in cloning system |
| CN104536919B (en) * | 2014-12-11 | 2018-02-13 | 浪潮(北京)电子信息产业有限公司 | A kind of method and apparatus for avoiding IO to conflict in cloning system |
| CN107885488A (en) * | 2016-09-29 | 2018-04-06 | 微软技术许可有限责任公司 | It is intended to and has intentional lock |
| US11327905B2 (en) | 2016-09-29 | 2022-05-10 | Microsoft Technology Licensing, Llc | Intents and locks with intent |
| CN107885488B (en) * | 2016-09-29 | 2021-06-01 | 微软技术许可有限责任公司 | Lock with intention and intention |
| CN106372030A (en) * | 2016-10-27 | 2017-02-01 | 郑州云海信息技术有限公司 | Method and device for data synchronization of dual controller storage system |
| US11601501B2 (en) | 2016-12-09 | 2023-03-07 | Google Llc | High-throughput algorithm for multiversion concurrency control with globally synchronized time |
| CN108228698A (en) * | 2016-12-09 | 2018-06-29 | 谷歌有限责任公司 | For the high-throughput algorithm of the Multi version concurrency control with the global synchronization time |
| CN108228698B (en) * | 2016-12-09 | 2021-11-16 | 谷歌有限责任公司 | High throughput algorithm for multi-version concurrent control with global synchronization time |
| CN107347090A (en) * | 2017-06-08 | 2017-11-14 | 广州市呼百应网络技术股份有限公司 | Method of data synchronization |
| CN107507001A (en) * | 2017-08-11 | 2017-12-22 | 杭州秘猿科技有限公司 | A kind of data revision method based on license chain |
| CN109428843A (en) * | 2017-08-30 | 2019-03-05 | 英特尔公司 | The technology that efficient result shifts is provided from the accelerator installation in depolymerization framework |
| US10467424B2 (en) | 2017-10-12 | 2019-11-05 | International Business Machines Corporation | File system content based security |
| GB2580005A (en) * | 2017-10-12 | 2020-07-08 | Ibm | File system content based security |
| GB2580005B (en) * | 2017-10-12 | 2020-12-16 | Ibm | File system content based security |
| WO2019073324A1 (en) * | 2017-10-12 | 2019-04-18 | International Business Machines Corporation | File system content based security |
| CN110059514A (en) * | 2019-04-18 | 2019-07-26 | 珠海美佳音科技有限公司 | Method for writing data, NFC label, NFC device and storage medium |
| CN110059514B (en) * | 2019-04-18 | 2021-06-08 | 珠海美佳音科技有限公司 | Data writing method, NFC tag, NFC device and storage medium |
| CN110399383A (en) * | 2019-07-29 | 2019-11-01 | 中国工商银行股份有限公司 | Applied to the data processing method of server, device, calculate equipment, medium |
| WO2021078222A1 (en) * | 2019-10-25 | 2021-04-29 | 北京金山云网络技术有限公司 | Determination method and apparatus for master copy files of object in object storage system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101706802B (en) | 2013-06-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101706802B (en) | Method, device and sever for writing, modifying and restoring data | |
| US10114580B1 (en) | Data backup management on distributed storage systems | |
| EP3179359B1 (en) | Data sending method, data receiving method, and storage device | |
| CN106776130B (en) | Log recovery method, storage device and storage node | |
| EP3361383B1 (en) | Modifying membership of replication groups via journal operations | |
| CN110389858B (en) | Method and device for recovering faults of storage device | |
| US10860447B2 (en) | Database cluster architecture based on dual port solid state disk | |
| CN107241430A (en) | A kind of enterprise-level disaster tolerance system and disaster tolerant control method based on distributed storage | |
| US20140108345A1 (en) | Exchanging locations of an out of synchronization indicator and a change recording indicator via pointers | |
| CN105550229A (en) | Method and device for repairing data of distributed storage system | |
| US10929043B2 (en) | Space reservation for distributed storage systems | |
| KR100922584B1 (en) | Distributed object-sharing system and method thereof | |
| CN111309524A (en) | Distributed storage system fault recovery method, device, terminal and storage medium | |
| CN110121694B (en) | Log management method, server and database system | |
| RU2643642C2 (en) | Use of cache memory and another type of memory in distributed memory system | |
| Alagappan et al. | {Fault-Tolerance}, Fast and Slow: Exploiting Failure Asynchrony in Distributed Systems | |
| CN106605217A (en) | Application transparent continuous availability using synchronous replication across data stores in failover cluster | |
| CN110737716B (en) | Data writing method and device | |
| CN109165117B (en) | Data processing method and system | |
| US9952941B2 (en) | Elastic virtual multipath resource access using sequestered partitions | |
| CN105550230A (en) | Method and device for detecting failure of node of distributed storage system | |
| CN112540873B (en) | Disaster tolerance method and device, electronic equipment and disaster tolerance system | |
| CN110198331A (en) | A kind of method and system of synchrodata | |
| JP2009265973A (en) | Data synchronization system, failure recovery method, and program | |
| JP2004272318A (en) | System changeover system, processing method therefor, and processing program therefor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C56 | Change in the name or address of the patentee |
Owner name: HUAWEI DIGITAL TECHNOLOGY (CHENGDU) CO., LTD. Free format text: FORMER NAME: CHENGDU HUAWEI SYMANTEC TECHNOLOGIES CO., LTD. |
|
| CP01 | Change in the name or title of a patent holder |
Address after: 611731 Chengdu high tech Zone, Sichuan, West Park, Qingshui River Patentee after: HUAWEI DIGITAL TECHNOLOGIES (CHENG DU) Co.,Ltd. Address before: 611731 Chengdu high tech Zone, Sichuan, West Park, Qingshui River Patentee before: CHENGDU HUAWEI SYMANTEC TECHNOLOGIES Co.,Ltd. |
|
| TR01 | Transfer of patent right |
Effective date of registration: 20220831 Address after: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen Patentee after: HUAWEI TECHNOLOGIES Co.,Ltd. Address before: 611731 Qingshui River District, Chengdu hi tech Zone, Sichuan, China Patentee before: HUAWEI DIGITAL TECHNOLOGIES (CHENG DU) Co.,Ltd. |
|
| TR01 | Transfer of patent right |