CN105162879B - Realize the method, apparatus and system of multimachine room data consistency - Google Patents
Realize the method, apparatus and system of multimachine room data consistency Download PDFInfo
- Publication number
- CN105162879B CN105162879B CN201510617174.7A CN201510617174A CN105162879B CN 105162879 B CN105162879 B CN 105162879B CN 201510617174 A CN201510617174 A CN 201510617174A CN 105162879 B CN105162879 B CN 105162879B
- Authority
- CN
- China
- Prior art keywords
- data
- computer room
- computer
- room
- determined
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
Abstract
The invention discloses a kind of method, apparatus and system for realizing multimachine room data consistency, this method comprises: determining that computer room is written as data in a computer room from multiple computer rooms after receiving data write request；The data content write-in data write-in computer room being written into, also, preset public message queue is written in the data content being written into；Wherein, the data content in public message queue is used to carry out corresponding data update for other each computer rooms.The problem of data that the present invention is able to solve between multiple computer rooms in the prior art are difficult to realize consistency in the case where frequent updating.
Description
Technical field
The present invention relates to Internet technical fields, and in particular to a kind of method, apparatus for realizing multimachine room data consistency
And system.
Background technique
With becoming increasingly popular for network service, it is often necessary to store some common data (such as business datum) simultaneously
On the server of multiple computer rooms.For example, in order to provide identical service to user in all parts of the country, respectively in Beijing, Guangzhou
Three computer rooms are provided with Tibet, required service can be accessed by any computer room node in user, under normal circumstances user
Access nearest computer room node.At this moment, Beijing, the data stored in the regional computer room in Guangzhou and three, Tibet are all the same.
For another example, sometimes for prevent because in a computer room server hang due to lead to service disruption the case where generation, also can simultaneously portion
Multiple computer rooms for being stored with identical data are affixed one's name to, it can be provided after a computer room is hung by other computer room for user
By service.
In the above case said, the data needs in multiple computer rooms are consistent, once the data in each computer room occur not
Unanimous circumstances just will affect the normal use of user.Since business datum changes frequent occurrence, once the data in certain computer room
After change, therefore other computer rooms do not have to synchronize update in time and will lead to the inconsistent situation of data not to be had also at present
There is a kind of effective mechanism that can ensure that the data in multiple computer rooms are completely the same.
Summary of the invention
In view of the above problems, it proposes on the present invention overcomes the above problem or at least be partially solved in order to provide one kind
State the method, apparatus and system of the realization multimachine room data consistency of problem.
According to an aspect of the invention, there is provided a kind of method for realizing multimachine room data consistency comprising: it receives
To after data write request, determine that computer room is written as data in a computer room from multiple computer rooms；The data content being written into
The data content that data are written computer room, also, are written into is written, preset public message queue is written；Wherein, public message
Data content in queue is used to carry out corresponding data update for other each computer rooms.
Another aspect according to the present invention provides a kind of device for realizing multimachine room data consistency comprising: write-in
Request receiving module, suitable for determining that a computer room is written as data from multiple computer rooms after receiving data write request
Computer room；First Data write. module, suitable for the data content write-in data write-in computer room being written into；And second data write
Enter module, preset public message queue is written in the data content suitable for being written into, wherein the data in public message queue
Content is used to carry out corresponding data update for other each computer rooms.
Another aspect according to the present invention provides a kind of system for realizing multimachine room data consistency comprising: it is multiple
Computer room, and the client being respectively connected with multiple computer rooms, wherein client includes above-mentioned realization multimachine room data consistency
Device.
In the method, apparatus and system of realization multimachine room data consistency of the invention, asked when receiving data write-in
When asking, a computer room in multiple computer rooms is not write data only to, but also further write data into preset public message
In queue, so that other each computer rooms can carry out corresponding data update according to the public message queue.Since number occurring every time
Public message queue can be all updated when according to update accordingly, thus each computer room can carry out together according to the public message queue
Step updates, it is ensured that the data consistency in each computer room.
The above description is only an overview of the technical scheme of the present invention, in order to better understand the technical means of the present invention,
And it can be implemented in accordance with the contents of the specification, and in order to allow above and other objects of the present invention, feature and advantage can
It is clearer and more comprehensible, the followings are specific embodiments of the present invention.
Detailed description of the invention
By reading the following detailed description of the preferred embodiment, various other advantages and benefits are common for this field
Technical staff will become clear.The drawings are only for the purpose of illustrating a preferred embodiment, and is not considered as to the present invention
Limitation.And throughout the drawings, the same reference numbers will be used to refer to the same parts.In the accompanying drawings:
Fig. 1 shows the flow chart of the method according to an embodiment of the invention for realizing multimachine room data consistency；
Fig. 2 shows the processes of the method for realization multimachine room data consistency accord to a specific embodiment of that present invention
Figure；
Fig. 3 shows the schematic diagram of an application scenarios of embodiment shown in Fig. 2；
Fig. 4 shows the treatment process schematic diagram of data read request in embodiment shown in Fig. 2；
Fig. 5 shows the structure chart of the device according to an embodiment of the invention for realizing multimachine room data consistency；With
And
Fig. 6 shows the structure chart of the system according to an embodiment of the invention for realizing multimachine room data consistency.
Specific embodiment
Exemplary embodiments of the present disclosure are described in more detail below with reference to accompanying drawings.Although showing the disclosure in attached drawing
Exemplary embodiment, it being understood, however, that may be realized in various forms the disclosure without should be by embodiments set forth here
It is limited.On the contrary, these embodiments are provided to facilitate a more thoroughly understanding of the present invention, and can be by the scope of the present disclosure
It is fully disclosed to those skilled in the art.
The embodiment of the invention provides a kind of method, apparatus and system for realizing multimachine room data consistency, at least can
Solve the problems, such as that the data between multiple computer rooms in the prior art are difficult to realize consistency in the case where frequent updating.
Fig. 1 shows the flow chart of the method according to an embodiment of the invention for realizing multimachine room data consistency, such as
Shown in Fig. 1, this method comprises the following steps:
Step S110: after receiving data write request, determine a computer room as data writing machine from multiple computer rooms
Room.
It optionally, can be according to following rule when determining that computer room is written as data in a computer room from multiple computer rooms
One or more of come determine data be written computer room: according to the type of service of data to be written determine data be written computer room；It presses
Each computer room is determined as data write-in computer room in turn according to preset order；And access packet is sent to each computer room, it will be in preset time
The computer room for inside returning response bag is determined as available computer room, and the computer room of non-returning response packet within a preset time is determined as can not
With computer room, and further determine that computer room is written in data from available computer room.
Step S120: computer room is written in the data determined in the data content write step S110 being written into.
Step S130: preset public message queue is written in the data content being written into, wherein the public message queue
In data content be used to carry out corresponding data update for other each computer rooms.
Optionally, when updating public message queue, other each computer rooms are obtained public by data synchronizing process
Data content in message queue, and corresponding data update is carried out according to the data content got；Herein, public message
Queue is known as QBus queue, and data synchronizing process is known as QBusHandle process.
Optionally, the case where a certain computer room synchronized update failure or data are tampered in order to prevent, this method may be used also
Periodically the data content in each computer room is compared with further including steps of, when finding that data are inconsistent, is determined
The correct computer room of data, and data update is carried out to other each computer rooms according to the data content in the correct computer room of data；Wherein,
Determine that implementation when the correct computer room of data includes: the timestamp for obtaining the data content in each computer room, most by timestamp
Computer room where big data content is determined as the correct computer room of data；And/or the incremental data of each computer room is obtained, it calculates and increases
The corresponding md5 value of data is measured, computer room corresponding to the correct incremental data of md5 value is determined as the correct computer room of data.
In the method for realization multimachine room data consistency of the invention, when receiving data write request, not only will
A computer room in multiple computer rooms is written in data, but also further writes data into preset public message queue, so that
Other each computer rooms can carry out corresponding data update according to the public message queue.Shi Douhui is updated since data occurring every time
It is corresponding to update public message queue, thus each computer room can synchronize update according to the public message queue, it is ensured that
Data consistency in each computer room.
Fig. 2 shows the processes of the method for realization multimachine room data consistency accord to a specific embodiment of that present invention
Figure.In the embodiment shown in Figure 2, the executing subject of each step is the client being respectively connected with multiple computer rooms, the client
The data access demand from different calling service sides can be received and processed, which includes that data write-in needs
It asks and reading data demand.When specific implementation, can by client SDK provided by the client (Client SDK) come
Execute corresponding operation.Fig. 3 shows the schematic diagram of an application scenarios of the present embodiment, as shown in figure 3, client 31 can
Receive the data access request from first network equipment (web appliance) 32 and/or second network equipment 33, and client 31 with
First computer room 34, the second computer room 35 and third computer room 36 are respectively connected with.In addition, it will be appreciated to those of skill in the art that
More platform network equipments or more computer room can also be included in the application scenarios of the present embodiment.
As shown in Fig. 2, this method comprises the following steps:
Step S210: client 31 receives data write request.
In application scenarios shown in Fig. 3, the data write request from first network equipment (web appliance) 32 and/or
Second network equipment 33.In the present embodiment others application scenarios, which can be from other business
Called side, for example, the data write request can be sent by equipment such as user terminal or halfpaces；In addition, the data are write
Entering request may also originate from computer lab management person, for example, sending the data by management server by the administrator in multimachine room
Write request.Specifically, which includes newly-increased request of data, modification request of data and deletes request of data
Deng.
Step S220: client 31 determines that a computer room is made from multiple computer rooms according to the data write request received
Computer room is written for data.
Assuming that client 31 has received the data write request from first network equipment 32, it be according to the data
Write request modifies the data content in three computer rooms.At this moment, client 31 will be first from the first computer room 34,35 and of the second computer room
Select a computer room that computer room is written as data in third computer room 36.It, can be according to one in following rule when specific implementation
Or it is multiple come determine data be written computer room:
(1) determine that computer room is written in data according to the type of service of data to be written.For example, when disposing multiple computer rooms, often
A computer room lays particular emphasis on the different types of business datum of storage respectively, therefore, according to the business of this data to be written received
Type, which is assured that, is written computer room for which computer room as this data.
(2) each computer room is determined as data write-in computer room in turn according to preset order.For example, according to the first computer room 34-
The sequence of two computer room 35- third computer rooms 36 come determine data be written computer room, it is assumed that the last time receives institute after data write request
Determining data write-in computer room is the second computer room 35, then this receives identified data write-in computer room after data write request
It is then third computer room 36.
(3) access packet is sent to each computer room, the computer room for returning response bag within a preset time is determined as available computer room,
The computer room of non-returning response packet within a preset time is determined as unavailable computer room, and further determines data from available computer room
Computer room is written.
Three kinds of above-mentioned modes both can be used alone, and can also be used in combination.For example, the third mode can be with preceding two
Kind mode combines: when the data write-in computer room determined according to first way or the second way does not return within a preset time
When returning response bag, then other computer rooms for returning response bag are ultimately determined to data write-in computer room, to guarantee data writing machine
The availability in room.
Step S230: computer room is written in the data determined in the data content write step S220 that client 31 is written into.
For example, in this example, it is assumed that the data write-in computer room determined in step S220 is the first computer room 34, then it is objective
The first computer room 34 is written in the data content that family end 31 is written into.Wherein, when each computer room uses SSDB storage mode storage service
When data, the SSDB memory for realizing data storage is further included inside each computer room, therefore, client 31 will be to
The data content of write-in is written in the SSDB memory 341 inside the first computer room 34.
Step S240: the data content that client 31 is written into is written in preset QBus queue.
As shown in figure 3, client 31 is further connected with QBus queue, it will be appreciated to those of skill in the art that
In addition to QBus queue, other public message queues can also be used to realize the present invention, as long as can reach orderly storage message
Purpose.Wherein, QBus queue is a kind of Distributed Message Queue of first in first out, can be in message transmitting procedure
Make sender and recipient need not be same to provide asynchronous communication mechanism for program or component as the container for saving message
When it is online, need not also understand other side, so that it may mutually transmission message.When specific implementation, QBus queue can both have been taken point-to-point
(Point-to Point) mode can also take subscription (Subscribe) mode.In the present embodiment, it is with subscribing mode
Example is illustrated, also, QBus queue is stored in a special server, i.e., on QBus server 30, in actual conditions
In, directly QBus queue can also be stored in client 31.Wherein, QBus server 30 can receive client 31 etc. and set
Preparation come the message comprising data content to be written, and pass through QBus queue to data content included in the message into
Row persistent storage.Since QBus server 30 often receives multiple messages simultaneously, QBus server 30 can be same
When create multiple QBus queues, each QBus queue includes multiple for storing the part (partition) of different messages again.For
Convenient for management, type of message/message subject (topic) of each message of each QBus queue for storing is identical, each
QBus queue is distinguished by type of message/theme.Each QBus queue can provide subscription service to specified recipient,
For example, the first computer room 34, the second computer room 35 and third computer room 36 in Fig. 3 can subscribe to the message content in QBus queue, often
When QBus queue updates, the first computer room 34, the second computer room 35 and third computer room 36 being capable of timely learning this updates
Situation synchronizes update to obtain message content from QBus queue.When QBus queue is multiple, each computer room both may be used
To subscribe to whole QBus queues, it also can choose QBus queue in part therein and subscribed to.
Step S250: other each computer rooms carry out corresponding data update according to the data content in QBus queue.
Since each computer room all has subscribed QBus queue, after QBus queue updates in step S240, respectively
A computer room can all receive update message.Each computer room is according to the identification information for including in the update message to determine whether will be into
Row updates.The identification information for example can be timestamp information, message digest information etc..First computer room 34 is according in update message
Identification information judgment go out it (message content be written the received mistake message content in step S230 for client 31
In first computer room 34), therefore, the first computer room 34 ignores this update of QBus queue, without any processing.Second computer room, 35 He
Third computer room 36 goes out it according to the identification information judgment in update message and did not received the message content, therefore, the second computer room 35
It reads specific message content corresponding to this update message from QBus queue respectively with third computer room 36, and carries out correspondingly
It updates.When specific update, the second computer room 35 writes data content to be written included in the specific message content read
Enter in the SSDB memory 351 inside the second computer room 35, third computer room 36 will be included in the specific message content that read
In SSDB memory 361 inside data content write-in third computer room 36 to be written.
In addition, a data simultaneous module can also be separately provided in each computer room other than taking subscribing mode,
For detecting whether QBus queue updates, and actively obtained when detecting update in the message updated in QBus queue
Hold, which can be realized by a special process, such as QBusHandle process.
It since client 31 can repeatedly update QBus queue in actual conditions, and include multiple portions in QBus queue, often
Therefore, in the present embodiment how to the message read and not a part also relates to for storing at least one message
The problem of message read is managed.For the ease of managing, each part (partition) in QBus queue is by multiple
Segment (segment) composition, each segment is using a piece of news offset (offset) in the portion as filename.Its
In, offset is message id, can be used to the storage location of direct localization message, QBus server can be periodically as unit of segment
Delete the message read.When judging whether message has read, can be judged by the message status of record.
S210 to step S250 can be such that the data in multimachine room are consistent through the above steps.But still may be used
Data can be caused inconsistent due to cause specific, for example, mistake has occurred in a certain computer room at no point in the update process；In addition, QBus
Although queue is the queue of a first in first out, but be not precluded within the reversed order problem that will appear small probability in special circumstances,
Thus the data between multimachine room may also be caused inconsistent.The inconsistent feelings of data caused by order to prevent because of various situations
Condition, the method in the embodiment of the present invention can further include following step S260.Wherein, step S260 is one optional
The step of, the step independently of above-mentioned step S210 to step S250, be one according to the time come the step of triggering.
Step S260: being periodically compared the data content in each computer room, when finding that data are inconsistent, determines data
Correct computer room, and data update is carried out to other each computer rooms according to the data content in the correct computer room of data.
Wherein, step S260 can both be executed by client 31, can also be by some in three computer rooms
Computer room executes.The frequency periodically compared can according to need flexible setting, for example, when the data of each computer room update frequent,
Higher versus frequency is set, it is such as primary per comparing every other hour；When the data of each computer room update infrequently, it is arranged lower
Versus frequency such as every other day or compares in a week primary.
Determine that implementation when the correct computer room of data includes at least one of following two mode:
(1) timestamp of the data content in each computer room is obtained, the computer room where the maximum data content of timestamp is true
It is set to the correct computer room of data.
Computer room can generate at least one data record after more new data every time, and timestamp is included at least in the data record
Information, the renewal time to the unique identification data record.Therefore, it when finding that each machine room data is inconsistent, obtains each
Computer room where the maximum data content of timestamp is determined as the correct computer room of data by the timestamp of computer room data record, and
The corresponding data content in other computer rooms is covered with the data content in the computer room.
(2) incremental data for obtaining each computer room calculates the corresponding md5 value of incremental data, by the correct incremental data of md5 value
Corresponding computer room is determined as the correct computer room of data.
Wherein, incremental data refers to last ratio to the more new data generated later.For example, comparing primary answer daily
With in scene, after the completion of nine o'clock of a whole morning week compares, Tuesday need to only compare generation after nine o'clock of week a whole morning
Data content, and the data content before nine o'clock of week a whole morning is then not necessarily to compare again.Pass through the side of incremental raio pair
Comparison efficiency can be improved in formula, reduces workload, avoids repeating to compare.It in practical applications, can also be by increment alignments
It is used in combination with full dose alignments, it may be assumed that execute an incremental raio pair every the first preset time, held every the second preset time
Full dose of row compares, wherein the first preset time is less than the second preset time.For example, carrying out an incremental raio pair daily, often
Full dose of Zhou Jinhang compares, so that the data compared be prevented to be tampered.In addition, comparing effect when comparing in order to improve
Rate can be realized by way of comparing the corresponding md5 value of pieces of data record.After computer room every time more new data, meeting
At least one data record is generated, every data records a corresponding md5 value, which calculated by certain algorithm
Come, for being verified to this data record.So can only be compared corresponding to pieces of data record when comparing
Md5 value, if md5 value is identical, then it is assumed that data content is identical.Since md5 value is shorter, by md5 value be compared speed compared with
Fastly.When md5 value difference, it was both likely to be data record difference, it is also possible to be that data record is identical, only md5 value calculates
Therefore mistake will also further compare the data content of data record corresponding with the md5 value, the i.e. true value of data, if
True value is identical, it is determined that the data of each computer room are consistent；If true value is different, it is determined that the data of each computer room are inconsistent, and
When finding that data are inconsistent, the correct computer room of data is determined by modes such as data checks.For example, different in md5 value
In the case of, it can be directed to the different data record of md5 value, the corresponding md5 value of data record is recalculated, by md5 value
Computer room where correct data record is determined as the correct computer room of data, and updates other according to the data stored in the computer room
Data in each computer room, it is final to realize the consistent purpose of multimachine room data.
Above-mentioned two ways both can be used alone, and can also be used in combination, this is not limited by the present invention.Except this it
Outside, those skilled in the art can also take various other ways flexibly to determine the correct computer room of data.
Generally speaking, the synchronized update between each computer room data can be realized by step S210 to step S250, passes through
Step S260 can further ensure that the final consistency between each computer room data by asynchronous verification.
In addition, above-mentioned step S210 to step S250 essentially describes the treatment process of data write request, above-mentioned
In embodiment, the treatment process of data read request can further include.Fig. 4 shows the processed of data read request
Journey schematic diagram.As shown in figure 4, the treatment process of the data read request includes the following steps:
Step S410: client 31 receives data read request.
It is still illustrated by taking application scenarios shown in Fig. 3 as an example, which comes from first network equipment 32
And/or second network equipment 33.In the present embodiment others application scenarios, which can be from others
Calling service side, for example, the data read request can be sent by equipment such as user terminal or halfpaces；In addition, the number
It may also originate from computer lab management person according to read requests, for example, sending this by management server by the administrator in multimachine room
Data read request.
Step S420: client 31 determines that a computer room is made from multiple computer rooms according to the data read request received
For reading data computer room.
Wherein it is possible to determine reading data computer room according to one or more of following rule: according to data to be read
Type of service determine reading data computer room；Each computer room is determined as reading data computer room in turn according to preset order；And to
Each computer room sends access packet, and the computer room for returning response bag within a preset time is determined as available computer room, will be in preset time
Inside the computer room of non-returning response packet is determined as unavailable computer room, and reading data computer room is further determined from available computer room.On
The three kinds of modes stated both can be used alone, and can also be used in combination.For example, the third mode can mutually be tied with first two mode
It closes: when the reading data computer room determined according to first way or the second way non-returning response packet within a preset time
When, then other computer rooms for returning response bag are ultimately determined to reading data computer room, to guarantee the available of reading data computer room
Property.
Step S430: data content is read from reading data computer room.
For example, in this example, it is assumed that determining reading data computer room is the first computer room 34 in the step s 420, then it is objective
Data content is read from the first computer room 34 and is supplied to the network equipment for sending data read request in family end 31.Wherein, when each
When computer room uses SSDB storage mode storage service data, further included inside each computer room for realizing data storage
SSDB memory, therefore, client 31 read data directly from the SSDB memory 341 inside the first computer room 34.
Fig. 5 shows the structure chart of the device according to an embodiment of the invention for realizing multimachine room data consistency, such as
Shown in Fig. 5, which includes:
Write request receiving module 51, suitable for determining a machine from multiple computer rooms after receiving data write request
Computer room is written as data in room；
First Data write. module 52, suitable for the data content write-in data write-in computer room being written into；And
Preset public message queue is written in second Data write. module 53, the data content suitable for being written into, wherein
Data content in the public message queue is used to carry out corresponding data update for other each computer rooms.
Optionally, wherein write request receiving module determines that data are written according to one or more of following rule
Computer room: determine that computer room is written in data according to the type of service of data to be written；Each computer room is determined as in turn according to preset order
Computer room is written in data；And data access packet is sent to each computer room, the computer room of data response bag will be returned within a preset time
It is determined as available computer room, the computer room of non-returned data response bag within a preset time is determined as unavailable computer room, and further
Determine that computer room is written in data from available computer room.
Optionally, in order to further ensure that the data in each computer room are completely the same, which be can further include: fixed
Phase comparison module, when finding that data are inconsistent, determines data just suitable for periodically the data content in each computer room is compared
True computer room, and data update is carried out to other each computer rooms according to the data content in the correct computer room of data.Wherein, periodically compare
Implementation when the correct computer room of data, which includes: the timestamp for obtaining the data content in each computer room, to be determined to module, by when
Between stab maximum data content where computer room be determined as the correct computer room of data；And/or the incremental data of each computer room is obtained,
The corresponding md5 value of the incremental data is calculated, it is correct that computer room corresponding to the correct incremental data of md5 value is determined as data
Computer room.
In addition, the device can further include: read requests receiving module, suitable for receiving data read request
Afterwards, determine that a computer room reads data content as reading data computer room, and from reading data computer room from multiple computer rooms；Its
In, read requests receiving module determines the reading data computer room according to one or more of following rule: according to continuing
The type of service of access evidence determines reading data computer room；Each computer room is determined as reading data computer room in turn according to preset order；
And data access packet is sent to each computer room, the computer room for returning data response bag within a preset time is determined as available machine
The computer room of non-returned data response bag within a preset time is determined as unavailable computer room by room, and further from available computer room
Determine reading data computer room.
Fig. 6 shows the structure chart of the system according to an embodiment of the invention for realizing multimachine room data consistency, such as
Shown in Fig. 6, which includes: multiple computer rooms 61, and the client 62 being respectively connected with multiple computer rooms 61, wherein client
62 include the device of above-mentioned realization multimachine room data consistency.
Wherein, it can further include data simultaneous module in each computer room, data simultaneous module is suitable for disappearing public
Breath queue obtains the data content in public message queue when updating, and is carried out accordingly according to the data content got
Data update；Wherein, public message queue includes but is not limited to QBus queue, there is starting inside data simultaneous module
QBusHandle process, but it is not limited to QBusHandle process.
In the method, apparatus and system of realization multimachine room data consistency of the invention, asked when receiving data write-in
When asking, a computer room in multiple computer rooms is not write data only to, but also further write data into preset public message
In queue, so that other each computer rooms can carry out corresponding data update according to the public message queue.Since number occurring every time
Public message queue can be all updated when according to update accordingly, thus each computer room can carry out together according to the public message queue
Step updates, it is ensured that the data consistency in each computer room.
Algorithm and display are not inherently related to any particular computer, virtual system, or other device provided herein.
Various general-purpose systems can also be used together with teachings based herein.As described above, it constructs required by this kind of system
Structure be obvious.In addition, the present invention is also not directed to any particular programming language.It should be understood that can use various
Programming language realizes summary of the invention described herein, and the description done above to language-specific is to disclose this hair
Bright preferred forms.
In the instructions provided here, numerous specific details are set forth.It is to be appreciated, however, that implementation of the invention
Example can be practiced without these specific details.In some instances, well known method, structure is not been shown in detail
And technology, so as not to obscure the understanding of this specification.
Similarly, it should be understood that in order to simplify the disclosure and help to understand one or more of the various inventive aspects,
Above in the description of exemplary embodiment of the present invention, each feature of the invention is grouped together into single implementation sometimes
In example, figure or descriptions thereof.However, the disclosed method should not be interpreted as reflecting the following intention: i.e. required to protect
Shield the present invention claims features more more than feature expressly recited in each claim.More precisely, as following
Claims reflect as, inventive aspect is all features less than single embodiment disclosed above.Therefore,
Thus the claims for following specific embodiment are expressly incorporated in the specific embodiment, wherein each claim itself
All as a separate embodiment of the present invention.
Those skilled in the art will understand that can be carried out adaptively to the module in the equipment in embodiment
Change and they are arranged in one or more devices different from this embodiment.It can be the module or list in embodiment
Member or component are combined into a module or unit or component, and furthermore they can be divided into multiple submodule or subelement or
Sub-component.Other than such feature and/or at least some of process or unit exclude each other, it can use any
Combination is to all features disclosed in this specification (including adjoint claim, abstract and attached drawing) and so disclosed
All process or units of what method or apparatus are combined.Unless expressly stated otherwise, this specification is (including adjoint power
Benefit require, abstract and attached drawing) disclosed in each feature can carry out generation with an alternative feature that provides the same, equivalent, or similar purpose
It replaces.
In addition, it will be appreciated by those of skill in the art that although some embodiments described herein include other embodiments
In included certain features rather than other feature, but the combination of the feature of different embodiments mean it is of the invention
Within the scope of and form different embodiments.For example, in the following claims, embodiment claimed is appointed
Meaning one of can in any combination mode come using.
Various component embodiments of the invention can be implemented in hardware, or to run on one or more processors
Software module realize, or be implemented in a combination thereof.It will be understood by those of skill in the art that can be used in practice
Microprocessor or digital signal processor (DSP) realize multimachine room data consistency according to an embodiment of the present invention of realizing
The some or all functions of some or all components in system.The present invention is also implemented as being retouched here for executing
The some or all device or device programs (for example, computer program and computer program product) for the method stated.
It is such to realize that program of the invention can store on a computer-readable medium, or can have one or more signal
Form.Such signal can be downloaded from an internet website to obtain, be perhaps provided on the carrier signal or with it is any its
He provides form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and ability
Field technique personnel can be designed alternative embodiment without departing from the scope of the appended claims.In the claims,
Any reference symbol between parentheses should not be configured to limitations on claims.Word "comprising" does not exclude the presence of not
Element or step listed in the claims.Word "a" or "an" located in front of the element does not exclude the presence of multiple such
Element.The present invention can be by means of including the hardware of several different elements and being come by means of properly programmed computer real
It is existing.In the unit claims listing several devices, several in these devices can be through the same hardware branch
To embody.The use of word first, second, and third does not indicate any sequence.These words can be explained and be run after fame
Claim.
The invention discloses:
A1, a kind of method for realizing multimachine room data consistency comprising:
After receiving data write request, determine that computer room is written as data in a computer room from multiple computer rooms；
The data write-in computer room is written in the data content being written into, also, the data content to be written is write
Enter preset public message queue；
Wherein, the data content in the public message queue is used to carry out corresponding data update for other each computer rooms.
A2, method according to a1, wherein when updating the public message queue, other each computer rooms are logical
It crosses data synchronizing process and obtains data content in the public message queue, and carried out according to the data content got corresponding
Data update.
A3, method according to a1 or a2, wherein when a computer room determining from multiple computer rooms is as data writing machine
Fang Shi determines the data write-in computer room according to one or more of following rule:
The data write-in computer room is determined according to the type of service of data to be written；
Each computer room is determined as data write-in computer room in turn according to preset order；And
Access packet is sent to each computer room, the computer room for returning response bag within a preset time is determined as available computer room, it will
The computer room of non-returning response packet is determined as unavailable computer room within a preset time, and the number is further determined from available computer room
According to write-in computer room.
A4, according to any method of A1-A3, wherein further comprise:
Periodically the data content in each computer room is compared, when finding that data are inconsistent, determines the correct machine of data
Room, and data update is carried out to other each computer rooms according to the data content in the correct computer room of data；
Wherein it is determined that implementation when the correct computer room of data includes:
Computer room where the maximum data content of timestamp is determined as by the timestamp for obtaining the data content in each computer room
The correct computer room of data；And/or
The incremental data for obtaining each computer room calculates the corresponding md5 value of the incremental data, by the correct incremental number of md5 value
It is determined as the correct computer room of data according to corresponding computer room.
A5, according to any method of A1-A4, wherein further comprise:
After receiving data read request, from multiple computer rooms determine a computer room as reading data computer room, and from
Data content is read in the reading data computer room；
Wherein, the reading data computer room is determined according to one or more of following rule:
The reading data computer room is determined according to the type of service of data to be read；
Each computer room is determined as reading data computer room in turn according to preset order；And
Access packet is sent to each computer room, the computer room for returning response bag within a preset time is determined as available computer room, it will
The computer room of non-returning response packet is determined as unavailable computer room within a preset time, and the number is further determined from available computer room
According to reading computer room.
B6, a kind of device for realizing multimachine room data consistency comprising:
Write request receiving module, suitable for determining a computer room from multiple computer rooms after receiving data write request
Computer room is written as data；
The data write-in computer room is written in first Data write. module, the data content suitable for being written into；And
Second Data write. module, suitable for preset public message queue is written in the data content to be written,
In, the data content in the public message queue is used to carry out corresponding data update for other each computer rooms.
B7, the device according to B6, wherein said write request receiving module is according to one or more in following rule
It is a to determine data write-in computer room:
The data write-in computer room is determined according to the type of service of data to be written；
Each computer room is determined as data write-in computer room in turn according to preset order；And
Data access packet is sent to each computer room, the computer room for returning data response bag within a preset time is determined as can be used
The computer room of non-returned data response bag within a preset time is determined as unavailable computer room by computer room, and further from available computer room
Computer room is written in the middle determination data.
B8, the device according to B6 or B7, wherein further comprise:
Regular comparison module, suitable for periodically the data content in each computer room is compared, when finding that data are inconsistent,
It determines the correct computer room of data, and data update is carried out to other each computer rooms according to the data content in the correct computer room of data；
Wherein, the regular comparison module determines that implementation when the correct computer room of data includes:
Computer room where the maximum data content of timestamp is determined as by the timestamp for obtaining the data content in each computer room
The correct computer room of data；And/or
The incremental data for obtaining each computer room calculates the corresponding md5 value of the incremental data, by the correct incremental number of md5 value
It is determined as the correct computer room of data according to corresponding computer room.
B9, according to any device of B6-B8, wherein further comprise:
Read requests receiving module, suitable for determining a computer room from multiple computer rooms after receiving data read request
Data content is read as reading data computer room, and from the reading data computer room；
Wherein, the read requests receiving module determines the reading data according to one or more of following rule
Computer room:
The reading data computer room is determined according to the type of service of data to be read；
Each computer room is determined as reading data computer room in turn according to preset order；And
Data access packet is sent to each computer room, the computer room for returning data response bag within a preset time is determined as can be used
The computer room of non-returned data response bag within a preset time is determined as unavailable computer room by computer room, and further from available computer room
The middle determination reading data computer room.
C10, a kind of system for realizing multimachine room data consistency comprising: multiple computer rooms, and with the multiple computer room
The client being respectively connected with, wherein the client includes the dress of the realization multimachine room data consistency as described in B6-B9 is any
It sets.
C11, the system according to C10, wherein each computer room further comprises data simultaneous module, is suitable in institute
The data content obtained in the public message queue when public message queue updates is stated, and according in the data got
Hold and carries out corresponding data update.
Claims (10)
1. a kind of method for realizing multimachine room data consistency comprising:
After client receives data write request, determine that computer room is written as data in a computer room from multiple computer rooms；Wherein,
The client is respectively connected with multiple computer rooms, is asked for receiving and processing the data write-in from different calling service sides
It asks；
Data content that the client is written into is written the data and is written computer room, also, by the data to be written
Preset public message queue is written in content；Wherein, the public message queue is to disappear for orderly storing the distribution of message
Cease queue；
Wherein, the data content in the public message queue is used to carry out corresponding data update for other each computer rooms；Wherein,
When updating the public message queue, other each computer rooms obtain the public message queue by data synchronizing process
In data content, and corresponding data update is carried out according to the data content that gets；
Wherein, after the public message queue updates, each computer room receives update message, each computer room according to this more
Whether the identification information judgment for including in new information will be updated；Wherein, if according to the identification information judgment in update message
The computer room received mistake message content out then ignores this update of public message queue；If according in update message
Identification information judgment goes out the computer room and did not received the message content, then it is right that this update message institute is read from public message queue
The message content answered simultaneously is updated；Wherein, the identification information in update message includes timestamp information or message digest information.
2. according to the method described in claim 1, wherein, when determining a computer room as data write-in computer room from multiple computer rooms
When, the data write-in computer room is determined according to one or more of following rule:
The data write-in computer room is determined according to the type of service of data to be written；
Each computer room is determined as data write-in computer room in turn according to preset order；And
Access packet is sent to each computer room, the computer room for returning response bag within a preset time is determined as available computer room, it will be pre-
If the computer room of non-returning response packet is determined as unavailable computer room in the time, and further determines that the data are write from available computer room
Enter computer room.
3. method according to claim 1 or 2, wherein further comprise:
Periodically the data content in each computer room is compared, when finding that data are inconsistent, determines the correct computer room of data, and
Data update is carried out to other each computer rooms according to the data content in the correct computer room of data；
Wherein it is determined that implementation when the correct computer room of data includes:
Computer room where the maximum data content of timestamp is determined as data by the timestamp for obtaining the data content in each computer room
Correct computer room；And/or
The incremental data for obtaining each computer room calculates the corresponding md5 value of the incremental data, by the correct incremental data institute of md5 value
Corresponding computer room is determined as the correct computer room of data.
4. method according to claim 1 or 2, wherein further comprise:
After receiving data read request, a computer room is determined from multiple computer rooms as reading data computer room, and from described
Data content is read in reading data computer room；
Wherein, the reading data computer room is determined according to one or more of following rule:
The reading data computer room is determined according to the type of service of data to be read；
Each computer room is determined as reading data computer room in turn according to preset order；And
Access packet is sent to each computer room, the computer room for returning response bag within a preset time is determined as available computer room, it will be pre-
If the computer room of non-returning response packet is determined as unavailable computer room in the time, and further determines that the data are read from available computer room
Take computer room.
5. a kind of device for realizing multimachine room data consistency, which is client, and the client and multiple computer rooms are distinguished
It is connected, for receiving and processing the data write request from different calling service sides；Comprising:
Write request receiving module, suitable for determining a computer room conduct from multiple computer rooms after receiving data write request
Computer room is written in data；
The data write-in computer room is written in first Data write. module, the data content suitable for being written into；And
Second Data write. module, suitable for preset public message queue is written in the data content to be written, wherein institute
The data content in public message queue is stated to be used to carry out corresponding data update for other each computer rooms；Wherein, described public to disappear
Ceasing queue is the Distributed Message Queue for orderly storing message；
Wherein, when updating the public message queue, other each computer rooms obtain the public affairs by data synchronizing process
Data content in message queue altogether, and corresponding data update is carried out according to the data content got；
Wherein, after the public message queue updates, each computer room receives update message, each computer room according to this more
Whether the identification information judgment for including in new information will be updated；Wherein, if according to the identification information judgment in update message
The computer room received mistake message content out then ignores this update of public message queue；If according in update message
Identification information judgment goes out the computer room and did not received the message content, then it is right that this update message institute is read from public message queue
The message content answered simultaneously is updated；Wherein, the identification information in update message includes timestamp information or message digest information.
6. device according to claim 5, wherein said write request receiving module according to it is following rule in one or
It is multiple to determine data write-in computer room:
The data write-in computer room is determined according to the type of service of data to be written；
Each computer room is determined as data write-in computer room in turn according to preset order；And
Data access packet is sent to each computer room, the computer room for returning data response bag within a preset time is determined as available machine
The computer room of non-returned data response bag within a preset time is determined as unavailable computer room by room, and further from available computer room
Determine the data write-in computer room.
7. device according to claim 5 or 6, wherein further comprise:
Regular comparison module, when finding that data are inconsistent, is determined suitable for periodically the data content in each computer room is compared
The correct computer room of data, and data update is carried out to other each computer rooms according to the data content in the correct computer room of data；
Wherein, the regular comparison module determines that implementation when the correct computer room of data includes:
Computer room where the maximum data content of timestamp is determined as data by the timestamp for obtaining the data content in each computer room
Correct computer room；And/or
The incremental data for obtaining each computer room calculates the corresponding md5 value of the incremental data, by the correct incremental data institute of md5 value
Corresponding computer room is determined as the correct computer room of data.
8. device according to claim 5 or 6, wherein further comprise:
Read requests receiving module, suitable for determining a computer room conduct from multiple computer rooms after receiving data read request
Reading data computer room, and data content is read from the reading data computer room；
Wherein, the read requests receiving module determines the data reader according to one or more of following rule
Room:
The reading data computer room is determined according to the type of service of data to be read；
Each computer room is determined as reading data computer room in turn according to preset order；And
Data access packet is sent to each computer room, the computer room for returning data response bag within a preset time is determined as available machine
The computer room of non-returned data response bag within a preset time is determined as unavailable computer room by room, and further from available computer room
Determine the reading data computer room.
9. a kind of system for realizing multimachine room data consistency comprising: multiple computer rooms, and phase is distinguished with the multiple computer room
Client even, wherein the client includes the dress of the realization multimachine room data consistency as described in claim 5-8 is any
It sets.
10. system according to claim 9, wherein each computer room further comprises data simultaneous module, is suitable in institute
The data content obtained in the public message queue when public message queue updates is stated, and according in the data got
Hold and carries out corresponding data update.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510617174.7A CN105162879B (en) | 2015-09-24 | 2015-09-24 | Realize the method, apparatus and system of multimachine room data consistency |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510617174.7A CN105162879B (en) | 2015-09-24 | 2015-09-24 | Realize the method, apparatus and system of multimachine room data consistency |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105162879A CN105162879A (en) | 2015-12-16 |
| CN105162879B true CN105162879B (en) | 2019-03-12 |
Family
ID=54803631
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510617174.7A Active CN105162879B (en) | 2015-09-24 | 2015-09-24 | Realize the method, apparatus and system of multimachine room data consistency |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105162879B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107038195B (en) * | 2015-12-17 | 2020-07-03 | 阿里巴巴集团控股有限公司 | Data synchronization method and device |
| CN105681428A (en) * | 2016-01-25 | 2016-06-15 | 浪潮软件股份有限公司 | User information synchronizing system, method and device |
| CN106385382B (en) * | 2016-09-05 | 2020-06-23 | 福州佳软软件技术有限公司 | Data synchronization method, device and system |
| CN108256002B (en) * | 2017-12-31 | 2021-05-11 | Oppo广东移动通信有限公司 | Cross-machine-room data synchronization method, device, system and server |
| CN108717384B (en) * | 2018-05-18 | 2021-11-23 | 创新先进技术有限公司 | Data backup method and device |
| CN109165261A (en) * | 2018-10-10 | 2019-01-08 | 深圳市欧瑞博科技有限公司 | Method of data synchronization, device, server and computer storage medium |
| CN109508912B (en) * | 2018-12-29 | 2021-05-28 | 广州虎牙信息科技有限公司 | Service scheduling method, device, equipment and storage medium |
| CN115104295A (en) * | 2020-03-25 | 2022-09-23 | 深圳市欢太科技有限公司 | Data processing method, data processing device, electronic device and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102457549A (en) * | 2010-10-28 | 2012-05-16 | 盛乐信息技术（上海）有限公司 | Data synchronization system and method for multiple network storage devices |
| CN103747091A (en) * | 2014-01-16 | 2014-04-23 | 电信科学技术第一研究所 | Distributed data synchronization system and method for embedded equipment |
| CN104202375A (en) * | 2014-08-22 | 2014-12-10 | 广州华多网络科技有限公司 | Method and system for synchronous data |
| CN104376017A (en) * | 2013-08-15 | 2015-02-25 | 阿里巴巴集团控股有限公司 | Method and system for inter-database data synchronization |
| CN104834635A (en) * | 2014-02-07 | 2015-08-12 | 中国移动通信集团广东有限公司 | Data processing method and device |
-
2015
- 2015-09-24 CN CN201510617174.7A patent/CN105162879B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102457549A (en) * | 2010-10-28 | 2012-05-16 | 盛乐信息技术（上海）有限公司 | Data synchronization system and method for multiple network storage devices |
| CN104376017A (en) * | 2013-08-15 | 2015-02-25 | 阿里巴巴集团控股有限公司 | Method and system for inter-database data synchronization |
| CN103747091A (en) * | 2014-01-16 | 2014-04-23 | 电信科学技术第一研究所 | Distributed data synchronization system and method for embedded equipment |
| CN104834635A (en) * | 2014-02-07 | 2015-08-12 | 中国移动通信集团广东有限公司 | Data processing method and device |
| CN104202375A (en) * | 2014-08-22 | 2014-12-10 | 广州华多网络科技有限公司 | Method and system for synchronous data |
Non-Patent Citations (1)
| Title |
|---|
| 基于消息队列的分布式系统数据一致性方法研究;于晓鹏等;《吉林大学学报》;20110531;第29卷(第3期);第258-262页 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105162879A (en) | 2015-12-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105162879B (en) | Realize the method, apparatus and system of multimachine room data consistency | |
| CN104980519B (en) | Multimachine room storage system | |
| CN108566290B (en) | Service configuration management method, system, storage medium and server | |
| CN111538763B (en) | Method for determining master node in cluster, electronic equipment and storage medium | |
| US9367261B2 (en) | Computer system, data management method and data management program | |
| CN105515872B (en) | The update method of configuration information, apparatus and system | |
| CN109542865A (en) | Distributed cluster system configuration file synchronous method, device, system and medium | |
| KR101545626B1 (en) | System for interoperation between dds and dbms | |
| CN104932941B (en) | A kind of distributed message processing system and device and method therein | |
| CN107451147A (en) | A kind of method and apparatus of kafka clusters switching at runtime | |
| CN104011701A (en) | Content delivery network | |
| US11218541B2 (en) | Data storage method, storage server, and storage medium and system | |
| CN106506703A (en) | Based on the service discovery method of shared drive, apparatus and system, server | |
| CN102333029A (en) | Routing method in server cluster system | |
| CN109361525B (en) | Method, device, control terminal and medium for restarting distributed deployment of multiple services | |
| CN112328448A (en) | Zookeeper-based monitoring method, monitoring device, equipment and storage medium | |
| CN109167819B (en) | Data synchronization system, method, device and storage medium | |
| CN109391487A (en) | A kind of configuration update method and system | |
| US10091288B2 (en) | Ordered execution of tasks | |
| CN105373563B (en) | Database switching method and device | |
| WO2016147073A1 (en) | Method and apparatus for identifying changed mailboxes in an internet message access protocol (imap) list | |
| CN113190620B (en) | Method, device, equipment and storage medium for synchronizing data between Redis clusters | |
| CN113946427A (en) | Task processing method, processor and storage medium for multi-operating system | |
| CN110012050A (en) | Message Processing, storage method, apparatus and system | |
| CN112363980A (en) | Data processing method and device for distributed system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220711 Address after: Room 801, 8th floor, No. 104, floors 1-19, building 2, yard 6, Jiuxianqiao Road, Chaoyang District, Beijing 100015 Patentee after: BEIJING QIHOO TECHNOLOGY Co.,Ltd. Address before: 100088 room 112, block D, 28 new street, new street, Xicheng District, Beijing (Desheng Park) Patentee before: BEIJING QIHOO TECHNOLOGY Co.,Ltd. Patentee before: Qizhi software (Beijing) Co., Ltd |
|
| TR01 | Transfer of patent right |