CN105162879A - Method, device and system for realizing data consistency among plurality of machine rooms - Google Patents
Method, device and system for realizing data consistency among plurality of machine rooms Download PDFInfo
- Publication number
- CN105162879A CN105162879A CN201510617174.7A CN201510617174A CN105162879A CN 105162879 A CN105162879 A CN 105162879A CN 201510617174 A CN201510617174 A CN 201510617174A CN 105162879 A CN105162879 A CN 105162879A
- Authority
- CN
- China
- Prior art keywords
- machine room
- data
- write
- room
- correct
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method, device and system for realizing data consistency among a plurality of machine rooms. The method comprises the following steps: after a data writing request is received, determining one of the plurality of machine rooms as a data writing machine room; and writing data content to be written into the data writing machine room, and writing the data content to be written into a preset public message queue, wherein the data content in the public message queue is used for performing corresponding data update for other machine rooms. Through adoption of the method, device and system, the problem of difficulty in realizing consistency under the situation of frequent update of data among the plurality of machine rooms in the prior art can be solved.
Description
Technical field
The present invention relates to Internet technical field, be specifically related to a kind of method, the Apparatus and system that realize multimachine room data consistency.
Background technology
Day by day universal along with network service, often needs some conventional data (as business datum etc.) to be stored on the server of multiple machine room simultaneously.Such as, in order to provide identical service to the user of all parts of the country, respectively in Beijing, Guangzhou and Tibet is provided with three machine rooms, user can have access to required service by arbitrary machine room node, and generally user accesses nearest machine room node.At this moment, the data stored in the machine room that Beijing, Guangzhou are regional with three, Tibet are all identical.And for example, sometimes the situation of service disruption is caused to occur in order to prevent because the server in a machine room hangs, also can dispose multiple machine room storing identical data, so that can by other machine room for user provides reliability services after a machine room is hung simultaneously.
In above-mentioned situation, the data in multiple machine room need to be consistent, once the data in each machine room have occurred that inconsistent situation will affect the normal use of user.Because business datum often changes, once after data in certain machine room change, other machine rooms do not carry out the situation generation that synchronized update will cause data inconsistent in time, therefore, also do not have a kind of effective mechanism can guarantee that the data in multiple machine room are completely the same at present.
Summary of the invention
In view of the above problems, the present invention is proposed to provide a kind of method realizing multimachine room data consistency, the Apparatus and system that overcome the problems referred to above or solve the problem at least in part.
According to an aspect of the present invention, provide a kind of method realizing multimachine room data consistency, it comprises: after receiving data write request, determines that a machine room writes machine room as data from multiple machine room; By data content to be written write data write machine room, and, by the public message queue that data content to be written write is preset; Wherein, the data content in public message queue is used for carrying out corresponding Data Update for other each machine room.
According to another aspect of the present invention, provide a kind of device realizing multimachine room data consistency, it comprises: write request receiver module, is suitable for after receiving data write request, determines that a machine room writes machine room as data from multiple machine room; First Data write. module, is suitable for data content to be written write data write machine room; And the second Data write. module, is suitable for public message queue data content to be written write preset, wherein, the data content in public message queue is used for carrying out corresponding Data Update for other each machine room.
According to another aspect of the present invention, provide a kind of system realizing multimachine room data consistency, it comprises: multiple machine room, and the client be connected respectively with multiple machine room, and wherein, client comprises the above-mentioned device realizing multimachine room data consistency.
In the method realizing multimachine room data consistency of the present invention, Apparatus and system, when receiving data write request, not only data are write a machine room in multiple machine room, but also in public message queue data write preset further, carry out corresponding Data Update to enable other each machine rooms according to this public message queue.Owing to all can upgrade public message queue accordingly when there is Data Update at every turn, thus each machine room can both carry out synchronized update according to this public message queue, ensure that the data consistency in each machine room.
Above-mentioned explanation is only the general introduction of technical solution of the present invention, in order to technological means of the present invention can be better understood, and can be implemented according to the content of specification, and can become apparent, below especially exemplified by the specific embodiment of the present invention to allow above and other objects of the present invention, feature and advantage.
Accompanying drawing explanation
By reading hereafter detailed description of the preferred embodiment, various other advantage and benefit will become cheer and bright for those of ordinary skill in the art.Accompanying drawing only for illustrating the object of preferred implementation, and does not think limitation of the present invention.And in whole accompanying drawing, represent identical parts by identical reference symbol.In the accompanying drawings:
Fig. 1 shows the flow chart of the method realizing multimachine room data consistency according to an embodiment of the invention;
Fig. 2 shows the flow chart realizing the method for multimachine room data consistency according to the present invention's specific embodiment;
Fig. 3 shows the schematic diagram of an application scenarios of the embodiment shown in Fig. 2;
Fig. 4 shows the processing procedure schematic diagram of data read request in the embodiment shown in Fig. 2;
Fig. 5 shows the structure chart of the device realizing multimachine room data consistency according to an embodiment of the invention; And,
Fig. 6 shows the structure chart of the system realizing multimachine room data consistency according to an embodiment of the invention.
Embodiment
Below with reference to accompanying drawings exemplary embodiment of the present disclosure is described in more detail.Although show exemplary embodiment of the present disclosure in accompanying drawing, however should be appreciated that can realize the disclosure in a variety of manners and not should limit by the embodiment set forth here.On the contrary, provide these embodiments to be in order to more thoroughly the disclosure can be understood, and complete for the scope of the present disclosure can be conveyed to those skilled in the art.
Embodiments provide a kind of method, the Apparatus and system that realize multimachine room data consistency, the data that at least can solve in prior art between multiple machine room are difficult to when frequent updating realize conforming problem.
Fig. 1 shows the flow chart of the method realizing multimachine room data consistency according to an embodiment of the invention, and as shown in Figure 1, the method comprises the steps:
Step S110: after receiving data write request, determines that from multiple machine room a machine room writes machine room as data.
Alternatively, when determining a machine room as data write machine room from multiple machine room, data write machine room can be determined according to one or more in following rule: according to the type of service determination data write machine room of data to be written; According to preset order, each machine room is defined as data write machine room in turn; And, send access bag to each machine room, the machine room returning respond packet is defined as available machine room, the machine room not returning respond packet in Preset Time is defined as unavailable machine room in Preset Time, and from available machine room, determine data write machine room further.
Step S120: by established data write machine room in data content write step S110 to be written.
Step S130: public message queue data content to be written write preset, wherein, the data content in this public message queue is used for carrying out corresponding Data Update for other each machine room.
Alternatively, when public message queue occurs to upgrade, other each machine room obtains the data content in public message queue by data synchronizing process, and carries out corresponding Data Update according to the data content got; Herein, public message queue is called QBus queue, and data synchronizing process is called QBusHandle process.
Alternatively, in order to prevent the failure of a certain machine room synchronized update, or the situation that data are tampered occurs, the method can further include following steps: regularly compare to the data content in each machine room, when finding that data are inconsistent, determine the machine room that data are correct, and according to the data content in the correct machine room of data, Data Update is carried out to other each machine rooms; Wherein, implementation when determining the correct machine room of data comprises: the timestamp obtaining the data content in each machine room, is defined as the correct machine room of data by the machine room at data content place maximum for timestamp; And/or, obtain the incremental data of each machine room, calculate the md5 value that incremental data is corresponding, the machine room corresponding to incremental data correct for md5 value is defined as the correct machine room of data.
Realize in the method for multimachine room data consistency of the present invention, when receiving data write request, not only data are write a machine room in multiple machine room, but also in public message queue data write preset further, carry out corresponding Data Update to enable other each machine rooms according to this public message queue.Owing to all can upgrade public message queue accordingly when there is Data Update at every turn, thus each machine room can both carry out synchronized update according to this public message queue, ensure that the data consistency in each machine room.
Fig. 2 shows the flow chart realizing the method for multimachine room data consistency according to the present invention's specific embodiment.In the embodiment shown in Figure 2, the executive agent of each step is the client be connected respectively with multiple machine room, this client can receive and process the data access demand from different calling service sides, and this data access demand comprises data write demand and digital independent demand.During specific implementation, the client SDK (ClientSDK) that can be provided by this client performs corresponding operation.Fig. 3 shows the schematic diagram of an application scenarios of the present embodiment, as shown in Figure 3, client 31 can receive the data access request from first network equipment (web equipment) 32 and/or second network equipment 33, and client 31 is connected with the 3rd machine room 36 respectively with the first machine room 34, second machine room 35.In addition, those skilled in the art it is understood that can also comprise more multiple stage network equipment or more machine room in the application scenarios of the present embodiment.
As shown in Figure 2, the method comprises the steps:
Step S210: client 31 receives data write request.
In the application scenarios shown in Fig. 3, this data write request is from first network equipment (web equipment) 32 and/or second network equipment 33.In other application scenarios of the present embodiment, this data write request also from other calling service side, such as, can send this data write request by the equipment such as user terminal or halfpace; In addition, this data write request also may derive from computer lab management person, such as, sends this data write request by the keeper in multimachine room by management server.Particularly, this data write request comprises newly-increased request of data, Update Table request and deletes request of data etc.
Step S220: client 31, according to the data write request received, determines that from multiple machine room a machine room writes machine room as data.
Suppose that client 31 have received a data write request from first network equipment 32, the data content in three machine rooms will be revised according to this data write request.At this moment, first client 31 will select a machine room to write machine room as data from the first machine room 34, second machine room 35 and the 3rd machine room 36.During specific implementation, data write machine room can be determined according to one or more in following rule:
(1) according to the type of service determination data write machine room of data to be written.Such as, when disposing multiple machine room, each machine room lays particular emphasis on respectively and stores dissimilar business datum, therefore, just can determine the data write machine room of which machine room as this according to the type of service of these data to be written received.
(2) according to preset order, each machine room is defined as data write machine room in turn.Such as, data write machine room is determined according to the order of the first machine room 34-second machine room 35-the 3rd machine room 36, suppose the last time receive data write request after determined data write machine room be the second machine room 35, then this after receiving data write request determined data write machine room be then the 3rd machine room 36.
(3) access bag is sent to each machine room, the machine room returning respond packet in Preset Time is defined as available machine room, the machine room not returning respond packet in Preset Time is defined as unavailable machine room, and from available machine room, determines data write machine room further.
Three kinds of above-mentioned modes both can be used alone, and also can be combined.Such as, the third mode can combine with first two mode: when the data determined according to first kind of way or the second way write machine room in Preset Time, do not return respond packet time, the machine room then other being returned respond packet is finally defined as data write machine room, to ensure the availability of data write machine room.
Step S230: client 31 is by established data write machine room in data content write step S220 to be written.
Such as, in the present embodiment, suppose that established data write machine room is the first machine room 34 in step S220, then data content to be written is write the first machine room 34 by client 31.Wherein, when each machine room adopts SSDB storage mode storage service data, comprise in each machine room inside the SSDB memory stored for realizing data further, therefore, data content to be written writes in the SSDB memory 341 of the first machine room 34 inside by client 31.
Step S240: in the QBus queue that data content to be written write is preset by client 31.
As shown in Figure 3, client 31 is connected with QBus queue further, and those skilled in the art are it is understood that except QBus queue, and other public message queue also can be used for realizing the present invention, as long as can reach the object of orderly storing message.Wherein, QBus queue is a kind of Distributed Message Queue of first in first out, it can as the container preserving message in message transmitting procedure, thus provide asynchronous communication mechanism for program or assembly, make transmit leg and recipient need not be simultaneously online, also the other side need not be understood, just can message transfer mutually.During specific implementation, QBus queue both can take point-to-point (Point-toPoint) pattern, also can take to subscribe to (Subscribe) pattern.In the present embodiment, be described for subscribing mode, and, QBus queue is stored in a special server, namely on QBus server 30, in a practical situation, also can directly QBus queue be stored in client 31.Wherein, QBus server 30 can receive the message comprising data content to be written that equipment such as client 31 grade is sent, and carries out persistent storage by QBus queue to the data content comprised in this message.Because QBus server 30 often receives multiple messages simultaneously, therefore, QBus server 30 can create multiple QBus queue simultaneously, and each QBus queue comprises again multiple part (partition) for depositing different messages.For the ease of management, the type of message/message subject (topic) of each bar message of each QBus queue for storing is identical, and each QBus queue is distinguished by type of message/theme.Each QBus queue can provide subscription service to the recipient specified, such as, the first machine room 34, second machine room 35 in Fig. 3 and the 3rd machine room 36 can subscribe to the message content in QBus queue, when QBus queue occurs to upgrade, first machine room 34, second machine room 35 and the 3rd machine room 36 just can know this update status in time, to obtain message content carrying out synchronized update from QBus queue.When QBus queue is multiple, each machine room both can subscribe to whole QBus queues, and part QBus queue wherein also can be selected to subscribe to.
Step S250: other each machine room carries out corresponding Data Update according to the data content in QBus queue.
Because each machine room all have subscribed QBus queue, therefore, after QBus queue upgrades in step S240, each machine room all can receive updating message.Each machine room judges whether to upgrade according to the identification information comprised in this updating message.This identification information can be such as timestamp information, message digest information etc.First machine room 34 goes out it according to the identification information judgment in updating message and had received this message content (message content writes in the first machine room 34 by client 31 in step S230), therefore, this renewal of QBus queue ignored by first machine room 34, is left intact.Second machine room 35 and the 3rd machine room 36 go out it according to the identification information judgment in updating message and did not receive this message content, therefore, second machine room 35 and the 3rd machine room 36 read the concrete message content corresponding to this updating message respectively from QBus queue, and correspondingly upgrade.During concrete renewal, the data content to be written comprised in the concrete message content read writes in the SSDB memory 351 of the second machine room 35 inside by the second machine room 35, and the data content to be written comprised in the concrete message content read writes in the SSDB memory 361 of the 3rd machine room 36 inside by the 3rd machine room 36.
In addition, except taking subscribing mode, can also a data simultaneous module be set separately in each machine room, be used for detecting QBus queue whether to upgrade, and the message content upgraded in active obtaining QBus queue when detecting renewal, this data simultaneous module can pass through a special process, and such as QBusHandle process realizes.
Because client in actual conditions 31 repeatedly can upgrade QBus queue, and comprise multiple part in QBus queue, each part, for storing at least one message, therefore, also relates to the problem how to manage the message read and the message do not read in the present embodiment.For the ease of management, each part (partition) in QBus queue is made up of multiple fragment (segment), and each fragment is using the skew (offset) in the portion of Article 1 message as filename.Wherein, skew is message id, and can be used to the memory location of direct localization message, QBus server regularly can delete the message read in units of fragment.When judging whether message reads, can be judged by the message status of record.
The data in multimachine room can be made to be consistent by above-mentioned step S210 to step S250.Such as, but still may data be caused inconsistent due to cause specific, a certain machine room there occurs mistake at no point in the update process; In addition, although QBus queue is the queue of a first in first out, be not precluded within the reversed order problem that in particular cases there will be small probability, the data between multimachine room thus also may be caused inconsistent.In order to the situation that the data stopping to cause because of various situation are inconsistent, the method in the embodiment of the present invention can further include following step S260.Wherein, step S260 is an optional step, and this step to step S250, is a step triggered according to the time independent of above-mentioned step S210.
Step S260: regularly compare to the data content in each machine room, when finding that data are inconsistent, determines the machine room that data are correct, and carries out Data Update according to the data content in the correct machine room of data to other each machine rooms.
Wherein, step S260 both can be performed by client 31, also can be performed by the some machine rooms in the middle of three machine rooms.The frequency of regular comparison can be arranged as required flexibly, such as, when the Data Update of each machine room is frequent, arranges higher versus frequency, as every comparison in a hour once; When each machine room Data Update infrequently time, lower versus frequency is set, as the comparison of every other day or one week once.
Implementation when determining the correct machine room of data at least comprises the one in following two kinds of modes:
(1) obtain the timestamp of the data content in each machine room, the machine room at data content place maximum for timestamp is defined as the correct machine room of data.
At least one data record can be produced after each more new data of machine room, in this data record, at least comprise timestamp information, in order to the update time of this data record of unique identification.Therefore, when finding that each machine room data is inconsistent, obtain the timestamp of each machine room data record, the machine room at data content place maximum for timestamp is defined as the correct machine room of data, and cover the corresponding data content in other machine rooms with the data content in this machine room.
(2) obtain the incremental data of each machine room, calculate the md5 value that incremental data is corresponding, the machine room corresponding to incremental data correct for md5 value is defined as the correct machine room of data.
Wherein, incremental data refers to that last ratio is to the more new data produced afterwards.Such as, in comparison every day application scenarios once, after nine o'clock comparison in the morning on Monday completes, the data content produced after only needing comparison nine o'clock of the morning on Monday Tuesday, to the data content before nine o'clock of the morning on Monday then without the need to comparison again.The mode right by incremental raio can improve comparison efficiency, reduces workload, avoids repetition comparison.In actual applications, increment alignments can also be combined with full dose alignments, that is: perform an incremental raio pair every the first Preset Time, perform a full dose comparison every the second Preset Time, wherein, the first Preset Time is less than the second Preset Time.Such as, carry out an incremental raio pair every day, carry out weekly a full dose comparison, thus prevent the data of comparison to be tampered.In addition, when comparison, in order to improve comparison efficiency, can be realized by the mode of the md5 value corresponding to comparison pieces of data record.After each more new data of machine room, can produce at least one data record, the corresponding md5 value of every bar data record, this md5 value is calculated by certain algorithm, for verifying this data record.So, when comparison, can md5 value corresponding to comparison pieces of data record, if md5 value is identical, then think that data content is identical.Because md5 value is shorter, to be compared speed by md5 value.When md5 value is different, both be likely that data record is different, also be likely that data record is identical, just md5 value mistake in computation, therefore, the data content of the data record also wanting further comparison corresponding with this md5 value, the i.e. actual value of data, if actual value is identical, then determine the data consistent of each machine room; If actual value is different, then determine that the data of each machine room are inconsistent, and when finding that data are inconsistent, determine by modes such as data checks the machine room that data are correct.Such as, when md5 value is different, can for the different data record of md5 value, recalculate the md5 value corresponding to this data record, the machine room at data record place correct for md5 value is defined as the correct machine room of data, and according to the data in other each machine rooms of the Data Update stored in this machine room, finally realize the object of multimachine room data consistent.
Two kinds of above-mentioned modes both can be used alone, and also can be combined, the present invention is not construed as limiting this.In addition, those skilled in the art can also take other various modes to determine the machine room that data are correct flexibly.
Generally speaking, the synchronized update between each machine room data can be realized by step S210 to step S250, the final consistency can guaranteeing between each machine room data further by asynchronous verification by step S260.
In addition, above-mentioned step S210 to step S250 essentially describes the processing procedure of data write request, in the above-described embodiments, can further include the processing procedure of data read request.Fig. 4 shows the processing procedure schematic diagram of data read request.As shown in Figure 4, the processing procedure of this data read request comprises the steps:
Step S410: client 31 receives data read request.
Still be described for the application scenarios shown in Fig. 3, this data read request is from first network equipment 32 and/or second network equipment 33.In other application scenarios of the present embodiment, this data read request also from other calling service side, such as, can send this data read request by the equipment such as user terminal or halfpace; In addition, this data read request also may derive from computer lab management person, such as, sends this data read request by the keeper in multimachine room by management server.
Step S420: client 31, according to the data read request received, determines that from multiple machine room a machine room is as digital independent machine room.
Wherein, digital independent machine room can be determined according to one or more in following rule: according to the type of service determination digital independent machine room of data to be read; According to preset order, each machine room is defined as digital independent machine room in turn; And, send access bag to each machine room, the machine room returning respond packet is defined as available machine room, the machine room not returning respond packet in Preset Time is defined as unavailable machine room, and from available machine room, determine digital independent machine room further in Preset Time.Three kinds of above-mentioned modes both can be used alone, and also can be combined.Such as, the third mode can combine with first two mode: when the digital independent machine room determined according to first kind of way or the second way does not return respond packet in Preset Time, the machine room then other being returned respond packet is finally defined as digital independent machine room, to ensure the availability of digital independent machine room.
Step S430: read data content from digital independent machine room.
Such as, in the present embodiment, supposing that established data reads machine room is in the step s 420 the first machine room 34, then client 31 reads data content and is supplied to the network equipment sending data read request from the first machine room 34.Wherein, when each machine room adopts SSDB storage mode storage service data, comprise in each machine room inside the SSDB memory stored for realizing data further, therefore, client 31 directly reads data from the SSDB memory 341 of the first machine room 34 inside.
Fig. 5 shows the structure chart of the device realizing multimachine room data consistency according to an embodiment of the invention, and as shown in Figure 5, this device 500 comprises:
Write request receiver module 51, is suitable for after receiving data write request, determines that a machine room writes machine room as data from multiple machine room;
First Data write. module 52, is suitable for data content to be written write data write machine room; And,
Second Data write. module 53, is suitable for public message queue data content to be written write preset, and wherein, the data content in this public message queue is used for carrying out corresponding Data Update for other each machine room.
Alternatively, wherein, write request receiver module determines data write machine room according to one or more in following rule: according to the type of service determination data write machine room of data to be written; According to preset order, each machine room is defined as data write machine room in turn; And, data access bag is sent to each machine room, the machine room returning data respond packet in Preset Time is defined as available machine room, the machine room of return data respond packet non-in Preset Time is defined as unavailable machine room, and from available machine room, determine data write machine room further.
Alternatively, completely the same in order to guarantee the data in each machine room further, this device can further include: regularly comparing module, be suitable for regularly comparing to the data content in each machine room, when finding that data are inconsistent, determine the machine room that data are correct, and according to the data content in the correct machine room of data, Data Update is carried out to other each machine rooms.Wherein, the implementation during machine room that regular comparing module determination data are correct comprises: the timestamp obtaining the data content in each machine room, is defined as the correct machine room of data by the machine room at data content place maximum for timestamp; And/or, obtain the incremental data of each machine room, calculate the md5 value that described incremental data is corresponding, the machine room corresponding to incremental data correct for md5 value is defined as the correct machine room of data.
In addition, this device can further include: read requests receiver module, is suitable for after receiving data read request, determines that a machine room is as digital independent machine room, and read data content from multiple machine room from digital independent machine room; Wherein, read requests receiver module determines described digital independent machine room according to one or more in following rule: according to the type of service determination digital independent machine room of data to be read; According to preset order, each machine room is defined as digital independent machine room in turn; And, data access bag is sent to each machine room, the machine room returning data respond packet in Preset Time is defined as available machine room, the machine room of return data respond packet non-in Preset Time is defined as unavailable machine room, and from available machine room, determines digital independent machine room further.
Fig. 6 shows the structure chart of the system realizing multimachine room data consistency according to an embodiment of the invention, as shown in Figure 6, this system comprises: multiple machine room 61, and the client 62 to be connected respectively with multiple machine room 61, wherein, client 62 comprises the above-mentioned device realizing multimachine room data consistency.
Wherein, can further include data simultaneous module in each machine room, data simultaneous module is suitable for the data content obtained when public message queue occurs to upgrade in public message queue, and carries out corresponding Data Update according to the data content got; Wherein, public message queue includes but not limited to QBus queue, and data simultaneous module inside starts QBusHandle process, but is not limited to be QBusHandle process.
In the method realizing multimachine room data consistency of the present invention, Apparatus and system, when receiving data write request, not only data are write a machine room in multiple machine room, but also in public message queue data write preset further, carry out corresponding Data Update to enable other each machine rooms according to this public message queue.Owing to all can upgrade public message queue accordingly when there is Data Update at every turn, thus each machine room can both carry out synchronized update according to this public message queue, ensure that the data consistency in each machine room.
Intrinsic not relevant to any certain computer, virtual system or miscellaneous equipment with display at this algorithm provided.Various general-purpose system also can with use based on together with this teaching.According to description above, the structure constructed required by this type systematic is apparent.In addition, the present invention is not also for any certain programmed language.It should be understood that and various programming language can be utilized to realize content of the present invention described here, and the description done language-specific is above to disclose preferred forms of the present invention.
In specification provided herein, describe a large amount of detail.But can understand, embodiments of the invention can be put into practice when not having these details.In some instances, be not shown specifically known method, structure and technology, so that not fuzzy understanding of this description.
Similarly, be to be understood that, in order to simplify the disclosure and to help to understand in each inventive aspect one or more, in the description above to exemplary embodiment of the present invention, each feature of the present invention is grouped together in single embodiment, figure or the description to it sometimes.But, the method for the disclosure should be construed to the following intention of reflection: namely the present invention for required protection requires feature more more than the feature clearly recorded in each claim.Or rather, as claims below reflect, all features of disclosed single embodiment before inventive aspect is to be less than.Therefore, the claims following embodiment are incorporated to this embodiment thus clearly, and wherein each claim itself is as independent embodiment of the present invention.
Those skilled in the art are appreciated that and adaptively can change the module in the equipment in embodiment and they are arranged in one or more equipment different from this embodiment.Module in embodiment or unit or assembly can be combined into a module or unit or assembly, and multiple submodule or subelement or sub-component can be put them in addition.Except at least some in such feature and/or process or unit be mutually repel except, any combination can be adopted to combine all processes of all features disclosed in this specification (comprising adjoint claim, summary and accompanying drawing) and so disclosed any method or equipment or unit.Unless expressly stated otherwise, each feature disclosed in this specification (comprising adjoint claim, summary and accompanying drawing) can by providing identical, alternative features that is equivalent or similar object replaces.
In addition, those skilled in the art can understand, although embodiments more described herein to comprise in other embodiment some included feature instead of further feature, the combination of the feature of different embodiment means and to be within scope of the present invention and to form different embodiments.Such as, in the following claims, the one of any of embodiment required for protection can use with arbitrary compound mode.
All parts embodiment of the present invention with hardware implementing, or can realize with the software module run on one or more processor, or realizes with their combination.It will be understood by those of skill in the art that the some or all functions realizing the some or all parts in the system of multimachine room data consistency that microprocessor or digital signal processor (DSP) can be used in practice to realize according to the embodiment of the present invention.The present invention can also be embodied as part or all equipment for performing method as described herein or device program (such as, computer program and computer program).Realizing program of the present invention and can store on a computer-readable medium like this, or the form of one or more signal can be had.Such signal can be downloaded from internet website and obtain, or provides on carrier signal, or provides with any other form.
The present invention will be described instead of limit the invention to it should be noted above-described embodiment, and those skilled in the art can design alternative embodiment when not departing from the scope of claims.In the claims, any reference symbol between bracket should be configured to limitations on claims.Word " comprises " not to be got rid of existence and does not arrange element in the claims or step.Word "a" or "an" before being positioned at element is not got rid of and be there is multiple such element.The present invention can by means of including the hardware of some different elements and realizing by means of the computer of suitably programming.In the unit claim listing some devices, several in these devices can be carry out imbody by same hardware branch.Word first, second and third-class use do not represent any order.Can be title by these word explanations.
The invention discloses:
A1, a kind of method realizing multimachine room data consistency, it comprises:
After receiving data write request, from multiple machine room, determine that a machine room writes machine room as data;
Data content to be written is write described data write machine room, and, by the public message queue that described data content to be written write is preset;
Wherein, the data content in described public message queue is used for carrying out corresponding Data Update for other each machine room.
A2, method according to A1, wherein, when described public message queue occurs to upgrade, other each machine room obtains the data content in described public message queue by data synchronizing process, and carries out corresponding Data Update according to the data content got.
A3, method according to A1 or A2, wherein, when determining a machine room as data write machine room from multiple machine room, determine described data write machine room according to one or more in following rule:
Described data write machine room is determined according to the type of service of data to be written;
According to preset order, each machine room is defined as data write machine room in turn; And,
Send access bag to each machine room, the machine room returning respond packet is defined as available machine room, the machine room not returning respond packet in Preset Time is defined as unavailable machine room in Preset Time, and from available machine room, determine described data write machine room further.
A4, according to the arbitrary described method of A1-A3, wherein, to comprise further:
Regularly the data content in each machine room being compared, when finding that data are inconsistent, determining the machine room that data are correct, and according to the data content in the correct machine room of data, Data Update is carried out to other each machine rooms;
Wherein, implementation when determining the correct machine room of data comprises:
Obtain the timestamp of the data content in each machine room, the machine room at data content place maximum for timestamp is defined as the correct machine room of data; And/or,
Obtain the incremental data of each machine room, calculate the md5 value that described incremental data is corresponding, the machine room corresponding to incremental data correct for md5 value is defined as the correct machine room of data.
A5, according to the arbitrary described method of A1-A4, wherein, to comprise further:
After receiving data read request, from multiple machine room, determine that a machine room is as digital independent machine room, and read data content from described digital independent machine room;
Wherein, described digital independent machine room is determined according to one or more in following rule:
Described digital independent machine room is determined according to the type of service of data to be read;
According to preset order, each machine room is defined as digital independent machine room in turn; And,
Send access bag to each machine room, the machine room returning respond packet is defined as available machine room, the machine room not returning respond packet in Preset Time is defined as unavailable machine room, and from available machine room, determine described digital independent machine room further in Preset Time.
B6, a kind of device realizing multimachine room data consistency, it comprises:
Write request receiver module, is suitable for after receiving data write request, determines that a machine room writes machine room as data from multiple machine room;
First Data write. module, is suitable for data content to be written being write described data write machine room; And,
Second Data write. module, is suitable for the public message queue described data content to be written write preset, and wherein, the data content in described public message queue is used for carrying out corresponding Data Update for other each machine room.
B7, device according to B6, wherein, said write request receiving module determines described data write machine room according to one or more in following rule:
Described data write machine room is determined according to the type of service of data to be written;
According to preset order, each machine room is defined as data write machine room in turn; And,
Data access bag is sent to each machine room, the machine room returning data respond packet in Preset Time is defined as available machine room, the machine room of return data respond packet non-in Preset Time is defined as unavailable machine room, and from available machine room, determines described data write machine room further.
B8, device according to B6 or B7, wherein, comprise further:
Regular comparing module, is suitable for regularly comparing to the data content in each machine room, when finding that data are inconsistent, determines the machine room that data are correct, and carries out Data Update according to the data content in the correct machine room of data to other each machine rooms;
Wherein, the implementation during machine room that described regular comparing module determination data are correct comprises:
Obtain the timestamp of the data content in each machine room, the machine room at data content place maximum for timestamp is defined as the correct machine room of data; And/or,
Obtain the incremental data of each machine room, calculate the md5 value that described incremental data is corresponding, the machine room corresponding to incremental data correct for md5 value is defined as the correct machine room of data.
B9, according to the arbitrary described device of B6-B8, wherein, to comprise further:
Read requests receiver module, is suitable for after receiving data read request, determines that a machine room is as digital independent machine room, and read data content from multiple machine room from described digital independent machine room;
Wherein, described read requests receiver module determines described digital independent machine room according to one or more in following rule:
Described digital independent machine room is determined according to the type of service of data to be read;
According to preset order, each machine room is defined as digital independent machine room in turn; And,
Data access bag is sent to each machine room, the machine room returning data respond packet in Preset Time is defined as available machine room, the machine room of return data respond packet non-in Preset Time is defined as unavailable machine room, and from available machine room, determines described digital independent machine room further.
C10, a kind of system realizing multimachine room data consistency, it comprises: multiple machine room, and the client be connected respectively with described multiple machine room, wherein, described client comprise as arbitrary in B6-B9 as described in the device realizing multimachine room data consistency.
C11, system according to C10, wherein, each machine room comprises data simultaneous module further, is suitable for the data content obtained when renewal occurs in described public message queue in described public message queue, and carries out corresponding Data Update according to the data content got.
Claims (10)
1. realize a method for multimachine room data consistency, it comprises:
After receiving data write request, from multiple machine room, determine that a machine room writes machine room as data;
Data content to be written is write described data write machine room, and, by the public message queue that described data content to be written write is preset;
Wherein, the data content in described public message queue is used for carrying out corresponding Data Update for other each machine room.
2. method according to claim 1, wherein, when described public message queue occurs to upgrade, other each machine room obtains the data content in described public message queue by data synchronizing process, and carries out corresponding Data Update according to the data content got.
3. method according to claim 1 and 2, wherein, when determining a machine room as data write machine room from multiple machine room, determine described data write machine room according to one or more in following rule:
Described data write machine room is determined according to the type of service of data to be written;
According to preset order, each machine room is defined as data write machine room in turn; And,
Send access bag to each machine room, the machine room returning respond packet is defined as available machine room, the machine room not returning respond packet in Preset Time is defined as unavailable machine room in Preset Time, and from available machine room, determine described data write machine room further.
4., according to the arbitrary described method of claim 1-3, wherein, comprise further:
Regularly the data content in each machine room being compared, when finding that data are inconsistent, determining the machine room that data are correct, and according to the data content in the correct machine room of data, Data Update is carried out to other each machine rooms;
Wherein, implementation when determining the correct machine room of data comprises:
Obtain the timestamp of the data content in each machine room, the machine room at data content place maximum for timestamp is defined as the correct machine room of data; And/or,
Obtain the incremental data of each machine room, calculate the md5 value that described incremental data is corresponding, the machine room corresponding to incremental data correct for md5 value is defined as the correct machine room of data.
5., according to the arbitrary described method of claim 1-4, wherein, comprise further:
After receiving data read request, from multiple machine room, determine that a machine room is as digital independent machine room, and read data content from described digital independent machine room;
Wherein, described digital independent machine room is determined according to one or more in following rule:
Described digital independent machine room is determined according to the type of service of data to be read;
According to preset order, each machine room is defined as digital independent machine room in turn; And,
Send access bag to each machine room, the machine room returning respond packet is defined as available machine room, the machine room not returning respond packet in Preset Time is defined as unavailable machine room, and from available machine room, determine described digital independent machine room further in Preset Time.
6. realize a device for multimachine room data consistency, it comprises:
Write request receiver module, is suitable for after receiving data write request, determines that a machine room writes machine room as data from multiple machine room;
First Data write. module, is suitable for data content to be written being write described data write machine room; And,
Second Data write. module, is suitable for the public message queue described data content to be written write preset, and wherein, the data content in described public message queue is used for carrying out corresponding Data Update for other each machine room.
7. device according to claim 6, wherein, said write request receiving module determines described data write machine room according to one or more in following rule:
Described data write machine room is determined according to the type of service of data to be written;
According to preset order, each machine room is defined as data write machine room in turn; And,
Data access bag is sent to each machine room, the machine room returning data respond packet in Preset Time is defined as available machine room, the machine room of return data respond packet non-in Preset Time is defined as unavailable machine room, and from available machine room, determines described data write machine room further.
8. the device according to claim 6 or 7, wherein, comprises further:
Regular comparing module, is suitable for regularly comparing to the data content in each machine room, when finding that data are inconsistent, determines the machine room that data are correct, and carries out Data Update according to the data content in the correct machine room of data to other each machine rooms;
Wherein, the implementation during machine room that described regular comparing module determination data are correct comprises:
Obtain the timestamp of the data content in each machine room, the machine room at data content place maximum for timestamp is defined as the correct machine room of data; And/or,
Obtain the incremental data of each machine room, calculate the md5 value that described incremental data is corresponding, the machine room corresponding to incremental data correct for md5 value is defined as the correct machine room of data.
9., according to the arbitrary described device of claim 6-8, wherein, comprise further:
Read requests receiver module, is suitable for after receiving data read request, determines that a machine room is as digital independent machine room, and read data content from multiple machine room from described digital independent machine room;
Wherein, described read requests receiver module determines described digital independent machine room according to one or more in following rule:
Described digital independent machine room is determined according to the type of service of data to be read;
According to preset order, each machine room is defined as digital independent machine room in turn; And,
Data access bag is sent to each machine room, the machine room returning data respond packet in Preset Time is defined as available machine room, the machine room of return data respond packet non-in Preset Time is defined as unavailable machine room, and from available machine room, determines described digital independent machine room further.
10. realize a system for multimachine room data consistency, it comprises: multiple machine room, and the client be connected respectively with described multiple machine room, wherein, described client comprise as arbitrary in claim 6-9 as described in the device realizing multimachine room data consistency.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510617174.7A CN105162879B (en) | 2015-09-24 | 2015-09-24 | Realize the method, apparatus and system of multimachine room data consistency |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510617174.7A CN105162879B (en) | 2015-09-24 | 2015-09-24 | Realize the method, apparatus and system of multimachine room data consistency |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105162879A true CN105162879A (en) | 2015-12-16 |
| CN105162879B CN105162879B (en) | 2019-03-12 |
Family
ID=54803631
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510617174.7A Active CN105162879B (en) | 2015-09-24 | 2015-09-24 | Realize the method, apparatus and system of multimachine room data consistency |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105162879B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105681428A (en) * | 2016-01-25 | 2016-06-15 | 浪潮软件股份有限公司 | User information synchronizing system, method and device |
| CN106385382A (en) * | 2016-09-05 | 2017-02-08 | 努比亚技术有限公司 | Data synchronization method, device and system |
| CN107038195A (en) * | 2015-12-17 | 2017-08-11 | 阿里巴巴集团控股有限公司 | Method of data synchronization and device |
| CN108256002A (en) * | 2017-12-31 | 2018-07-06 | 广东欧珀移动通信有限公司 | Across computer room method of data synchronization, device, system and server |
| CN108717384A (en) * | 2018-05-18 | 2018-10-30 | 阿里巴巴集团控股有限公司 | A kind of data back up method and device |
| CN109165261A (en) * | 2018-10-10 | 2019-01-08 | 深圳市欧瑞博科技有限公司 | Method of data synchronization, device, server and computer storage medium |
| CN109508912A (en) * | 2018-12-29 | 2019-03-22 | 广州虎牙信息科技有限公司 | A kind of business scheduling method, device, equipment and storage medium |
| WO2021189283A1 (en) * | 2020-03-25 | 2021-09-30 | 深圳市欢太科技有限公司 | Method and device for data processing, electronic device, and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102457549A (en) * | 2010-10-28 | 2012-05-16 | 盛乐信息技术（上海）有限公司 | Data synchronization system and method for multiple network storage devices |
| CN103747091A (en) * | 2014-01-16 | 2014-04-23 | 电信科学技术第一研究所 | Distributed data synchronization system and method for embedded equipment |
| CN104202375A (en) * | 2014-08-22 | 2014-12-10 | 广州华多网络科技有限公司 | Method and system for synchronous data |
| CN104376017A (en) * | 2013-08-15 | 2015-02-25 | 阿里巴巴集团控股有限公司 | Method and system for inter-database data synchronization |
| CN104834635A (en) * | 2014-02-07 | 2015-08-12 | 中国移动通信集团广东有限公司 | Data processing method and device |
-
2015
- 2015-09-24 CN CN201510617174.7A patent/CN105162879B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102457549A (en) * | 2010-10-28 | 2012-05-16 | 盛乐信息技术（上海）有限公司 | Data synchronization system and method for multiple network storage devices |
| CN104376017A (en) * | 2013-08-15 | 2015-02-25 | 阿里巴巴集团控股有限公司 | Method and system for inter-database data synchronization |
| CN103747091A (en) * | 2014-01-16 | 2014-04-23 | 电信科学技术第一研究所 | Distributed data synchronization system and method for embedded equipment |
| CN104834635A (en) * | 2014-02-07 | 2015-08-12 | 中国移动通信集团广东有限公司 | Data processing method and device |
| CN104202375A (en) * | 2014-08-22 | 2014-12-10 | 广州华多网络科技有限公司 | Method and system for synchronous data |
Non-Patent Citations (2)
| Title |
|---|
| 于晓鹏等: "基于消息队列的分布式系统数据一致性方法研究", 《吉林大学学报》 * |
| 代兵: "分布式消息队列Qbus介绍", 《百度文库》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107038195A (en) * | 2015-12-17 | 2017-08-11 | 阿里巴巴集团控股有限公司 | Method of data synchronization and device |
| CN107038195B (en) * | 2015-12-17 | 2020-07-03 | 阿里巴巴集团控股有限公司 | Data synchronization method and device |
| CN105681428A (en) * | 2016-01-25 | 2016-06-15 | 浪潮软件股份有限公司 | User information synchronizing system, method and device |
| CN106385382A (en) * | 2016-09-05 | 2017-02-08 | 努比亚技术有限公司 | Data synchronization method, device and system |
| CN106385382B (en) * | 2016-09-05 | 2020-06-23 | 福州佳软软件技术有限公司 | Data synchronization method, device and system |
| CN108256002A (en) * | 2017-12-31 | 2018-07-06 | 广东欧珀移动通信有限公司 | Across computer room method of data synchronization, device, system and server |
| CN108256002B (en) * | 2017-12-31 | 2021-05-11 | Oppo广东移动通信有限公司 | Cross-machine-room data synchronization method, device, system and server |
| CN108717384A (en) * | 2018-05-18 | 2018-10-30 | 阿里巴巴集团控股有限公司 | A kind of data back up method and device |
| CN108717384B (en) * | 2018-05-18 | 2021-11-23 | 创新先进技术有限公司 | Data backup method and device |
| CN109165261A (en) * | 2018-10-10 | 2019-01-08 | 深圳市欧瑞博科技有限公司 | Method of data synchronization, device, server and computer storage medium |
| CN109508912A (en) * | 2018-12-29 | 2019-03-22 | 广州虎牙信息科技有限公司 | A kind of business scheduling method, device, equipment and storage medium |
| WO2021189283A1 (en) * | 2020-03-25 | 2021-09-30 | 深圳市欢太科技有限公司 | Method and device for data processing, electronic device, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105162879B (en) | 2019-03-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105162879A (en) | Method, device and system for realizing data consistency among plurality of machine rooms | |
| CN104980519B (en) | Multimachine room storage system | |
| EP1402363B1 (en) | Method for ensuring operation during node failures and network partitions in a clustered message passing server | |
| US9367261B2 (en) | Computer system, data management method and data management program | |
| US20110225121A1 (en) | System for maintaining a distributed database using constraints | |
| KR101545626B1 (en) | System for interoperation between dds and dbms | |
| CN104967536A (en) | Method and device for realizing data consistency of multiple machine rooms | |
| CN102355369A (en) | Virtual clustered system as well as processing method and processing device thereof | |
| CN103761141A (en) | Method and device for realizing message queue | |
| CN110795395B (en) | File deployment system and file deployment method | |
| CN102955717A (en) | Message management equipment and method in distributed message processing system | |
| CN105205182A (en) | System deployed in multiple computer rooms and cross-computer-room business data processing method | |
| CN102981911A (en) | Distributed message handling system and device and method thereof | |
| CN103034541A (en) | Distributing type information system and equipment and method thereof | |
| CN102968457B (en) | Database method for switching between and system | |
| CN112052230B (en) | Multi-machine room data synchronization method, computing device and storage medium | |
| US20110225120A1 (en) | System for maintaining a distributed database using leases | |
| EP3803616A1 (en) | Change notifications for object storage | |
| CN109714430A (en) | Distributed caching method, device, computer system and storage medium | |
| CN113885797B (en) | Data storage method, device, equipment and storage medium | |
| CN113946427A (en) | Task processing method, processor and storage medium for multi-operating system | |
| US9578120B1 (en) | Messaging with key-value persistence | |
| CN112865927B (en) | Message delivery verification method, device, computer equipment and storage medium | |
| CN112363980A (en) | Data processing method and device for distributed system | |
| CN117331576A (en) | Application pre-download method, device, computer and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220711 Address after: Room 801, 8th floor, No. 104, floors 1-19, building 2, yard 6, Jiuxianqiao Road, Chaoyang District, Beijing 100015 Patentee after: BEIJING QIHOO TECHNOLOGY Co.,Ltd. Address before: 100088 room 112, block D, 28 new street, new street, Xicheng District, Beijing (Desheng Park) Patentee before: BEIJING QIHOO TECHNOLOGY Co.,Ltd. Patentee before: Qizhi software (Beijing) Co., Ltd |