Summary of the invention
The objective of the invention is at the deficiencies in the prior art, propose a kind of distributed real-time data replication synchronizing method, be called for short the DDR method.This method does not adopt database as storage medium, do not exist because of the database machine of delaying and cause the out of use situation of total system, simultaneously, real time of data transmission and reliability have been improved, satisfy ATC, CMS etc. in real time, the real time data synchronization replication needs in the magnificent mission system, the consistance that keeps data between principal and subordinate's process, and then improve total system reliability of operation, availability.
The objective of the invention is to reach like this: keep real time data synchronized process of operation on every processor of data sync at need, be called for short the DDR process, preserve in each DDR process with keep with other processor on the identical copying data of DDR process.Simultaneously, provide a function interface storehouse, abbreviation ddr interface storehouse is used for application program and data synchronizing process DDR carries out data interaction.Be to improve the performance of DDR process and the access of ddr interface database data, with the deposit data of DDR management of process in the shared drive (ShareMemory) that operating system provides.The data of DDR management of process are unit with the bar, the space size of every data is fixed as the N byte, be total to M bar data recording, be stored in the shared drive of a continuous M * N byte-sized, each bar data content includes the logical time LTS of these data, the application reference number UID and user data USER DATA three parts of renewal, wherein application reference number UID is made up of the process number PID of processor numbering and application program, keeps uniqueness.Each DDR process can be given other DDR process by the periodic broadcasting heartbeat message, to guarantee the keeping IP address list of all normal DDR processes of moving and current main attitude DDR process numbering in each DDR process.A main attitude DDR process can only be arranged in the whole DDR process group, if when main attitude or dereliction attitude process judge occurring more than 2, is main attitude with the DDR process that starts on the processor of an IP address minimum of main attitude election algorithm election, and all the other are from attitude; The data copy step comprises: real time data synchronized process DDR initialization; Main attitude election; The data consistency periodic test; Data Update; The Data Update notice; The Data Update conflict solves; Data read and DDR process withdraw from several sections, and wherein data read is that application program directly reads certain bar data by the ddr interface storehouse from local shared drive, does not produce access to netwoks.
The detailed process of data copy step is:
(1) real time data synchronized process DDR initialization:
A) program start, initialization shared drive data field,
B) master slave mode with process is changed to from the attitude state,
C) after the success of DDR process initiation, periodically to other DDR process broadcasting heartbeat message HBM,
D) receive other DDR process heartbeat message HBM, and upgrade the local activity tabulation, as the heartbeat message HBM message that exceeds schedule time and do not receive certain, the then corresponding information of this process of deletion in effort scale.
(2) main attitude election:
A) main attitude DDR process election algorithm: if receive that from attitude DDR process main DDR process will withdraw from message MDM message or exceeds schedule time and do not receive data consistent check message MCM message, then think main attitude process cisco unity malfunction, whether the IP address of checking this process place processor according to local activity DDR process list is minimum, judge whether this process should switch to main attitude, switch to main attitude as need, then broadcast transmission master DDR process alive message MEM message is main attitude with this machine DDR process switching simultaneously.
B) how main attitude DDR process collision is separated annual reporting law: if main attitude DDR process is received data consistent check message MCM message or main attitude DDR process alive message MEM, then think and have a plurality of main attitudes, therefore compare with the IP address of this machine IP address and the MCM message of receiving: if this machine IP is bigger, then thinking has other DDR process to be more suitable for main attitude, is from attitude with this machine DDR process switching; If this machine IP is less, then send MEM message to corresponding DDR process, other main attitude DDR process is done same judgement.
(3) data consistency periodic test:
Main attitude DDR process periodic broadcasting sends data consistent check message MCM, comprises the LTS and the UID field of every data of main attitude DDR process in the MCM message, and the DDR process will be done following the processing after receiving MCM message:
A) main attitude DDR process is received MCM message, then knows a plurality of main attitudes to occur, and the how main attitude DDR process collision that then starts in (2) step solves, thereby guarantees to have only a main attitude process.
B) each receives MCM message from attitude DDR process, then according to content in the MCM message one by one with the LTS of local data and UID relatively.
If the LTS value of this record in the MCM message is bigger, the data that main attitude DDR process is described are more recent version, send data re-transmission message RDM to main attitude DDR process, ask main attitude to retransmit this data, after main attitude DDR process is received RDM message, with data re-transmission response message RRM, broadcast the latest edition data of these data.
If the LTS value of this record in the MCM message is less, the data of then main attitude DDR process are legacy version, send RRM message to main attitude DDR process, to upgrade this data from attitude DDR process.
If this record in the MCM message equates with the LTS value of local record, but both UID values are unequal, illustrate to have data collision and also be not employed the program solution, send RDM message to main attitude DDR process, ask main attitude to retransmit this data,
After main attitude DDR process is received RDM message, with the latest edition data of these data of RRM information broadcast.
If this record in the MCM message all equates with the LTS value and the UID value of local record, both versions of data unanimities are described, do not do any processing.
(4) Data Update:
A) when application program need be upgraded certain bar data, call the renewal function that provides in ddr interface storehouse will be more new data send to local DDR process, local DDR process adopts more new data of following algorithm:
I) LTS with these data adds 1, and UID is updated to the UID of calling process, upgrades local data;
Ii) to other DDR process broadcast transmission Data Update UDM message, to upgrade this data;
B) the DDR process receives Data Update message UDM by following algorithm process:
I) as the LTS value of these data in the UDM message LTS value, illustrate that local data is newer, does not process less than local these data;
Ii), then use the Data Update local data among the UDM, comprise LTS and UID field as the LTS value of these data in the UDM message LTS value greater than local these data;
Iii) equal the LTS value of local these data, but the UID value is unequal, illustrates to produce the Data Update conflict, manage conflict by (6) step " Data Update conflict resolution algorithm ", and write down detailed daily record as the LTS value of these data in the UDM message;
Iv) other situation does not then process;
C) the DDR process receives data re-transmission response message RRM by following algorithm process:
I) as the LTS value of these data in the RRM message LTS value, illustrate that local data is newer, does not process less than local these data;
Ii), then use the Data Update local data among the RRM, comprise LTS and UID field as the LTS value of these data in the RRM message LTS value greater than local these data;
Iii) equal the LTS value of local these data, but the UID value is unequal, illustrates to produce the Data Update conflict, manage conflict by (6) step " Data Update conflict resolution algorithm ", and write down detailed daily record as the LTS value of these data in the RRM message.
(5) Data Update notice: the DDR process that receives Data Update sends the Data Update notification message to local ddr interface storehouse, and application program obtains this notification message by automatic regular polling or registered callbacks function mode.
(6) the Data Update conflict solves:
When the DDR process is received UDM or RRM message and detect Data Update when conflict and handle by following method:
A) local data with these data is saved in the colliding data tabulation;
B) with the local corresponding data of the Data Update of receiving;
C) the ddr interface storehouse of the Data Update message UID correspondence in colliding data sends the data collision notification message, solves this conflict by the local application of ddr interface storehouse correspondence;
D) send the Data Update notice to the ddr interface storehouse of local other application program.
(7) data read:
Application program directly reads certain bar data by the ddr interface storehouse from local shared drive.
(8) the DDR process withdraws from:
When the DDR process that is in main attitude withdraws from, broadcast transmission master DDR process will withdraw from message MDM message.
The good effect that the present invention has is:
1,, stopped because of the database then out of use situation of total system of machine of delaying, and data read is that application program directly reads by the ddr interface storehouse that do not produce access to netwoks, Data Update is fast, the real-time height from local shared drive without database.
2, DATA DISTRIBUTION is on every processor, and is consistent, and reliability is higher, is particularly suitable for real-time, distributed, magnificent mission system.
3, the local storage adopted the shared drive mode, makes between a plurality of application processes of this machine exchanges data speed application program faster, that support more.The shared drive mode is higher than mode efficient such as the file-sharing of this machine, TCP/IP.
Embodiment
The process of this distributed real-time data replication synchronizing method promptly is the implementation process of real time data synchronized process DDR process.
The message definition that is used for data interaction between the DDR process is as follows:
HBM:Heartbeat Message, DDR process heartbeat message is used for the states such as survival to other this machine of DDR process periodic report DDR process.HBM message data comprises: message sequence number, this machine IP, message generation time, master slave mode.
MCM:Master Check Message, data consistent check message, the data consistent check message of main DDR process periodic broadcasting is convenient to other DDR process check and new data more, to keep the global data unanimity.MCM message data comprises: the logical time LTS of message sequence number, this machine IP, message generation time, every data and application reference number UID field etc.
UDM:Update Data Message, Data Update message is used for the DDR process and Data Update situation and data updated is in time sent to other DDR process.UDM message comprises message sequence number, this machine IP address, message generation time, the numbering of data to be updated, logical time LTS, application reference number UID, user data user data etc.
RDM:Retransmission Data Message, data re-transmission message is used for the DDR process and retransmits certain bar data to certain DDR process requested; RDM message data comprises: the numbering of message sequence number, this machine IP address, message generation time, request retransmission data.
RRM:Reply Retransmission Message, the data re-transmission response message is used for receiving replying after the RDM message; RRM message data comprises: numbering, logical time LTS, application reference number UID, user data user data head and the user data of the data strip of message sequence number, this machine IP address, message generation time, re-transmission.
MDM:Master Dead Message, main DDR process will withdraw from message, send to other DDR process before the DDR process will withdraw from, and be used for other process initiation main frame election process; MDM message data comprises: message sequence number, this machine IP address, message generation time.
MEM:Master Exist Message, main DDR process alive message is used to solve the situation that has a plurality of main DDR processes simultaneously.MEM message data comprises: message sequence number, this machine IP address, message generation time.
The definition in ddr interface storehouse
It is that ddr interface storehouse mode is finished by the function interface storehouse is provided that application program and data synchronizing process DDR carry out data interaction.In the ddr interface storehouse promising application program provide that data are obtained, interface functions such as Data Update, automatic regular polling and register update notice readjustment.
The ddr interface built-in function is defined as:
(1) data are obtained function definition and are:
Int Get (int nddrNo, char*pUserdata, int nUserdataLength), this function take out nddrNo bar data and data content are put into pUserdata from local shared drive data, data are the longest to be the nUserdataLength byte.Application program directly by ddr interface storehouse reading of data from local shared drive, does not produce access to netwoks, both can reduce network overhead, can improve the real-time of data read again.
(2) the Data Update function definition is:
In the nddrNo bar data, data are the longest to be the nUserdataLength byte int Set (int nddrNo, const char*pUserdata, int nUserdataLength), this function copy the Data Update among the pUserdata to local data in.
(3) the automatic regular polling function definition is:
How many bar data variation int GetNotify (int*changedRecords) has returned according to function, and the numbering of delta data is put into the changedRecords array.Application program can be called the data that the get function is obtained concrete variation according to the data number in the changedRecords array.When certain bar data changed, the notification message that local transmission comprises data number was in the formation in the ddr interface storehouse of local application, and application program adopts the automatic regular polling mode to obtain the data message of variation.
(4) the registered callbacks function is:
Int RegisterNotifyCallback (void*funcptr), funcptr are shape such as void func (int nRecordNo) function pointer, the data number of nRecordNo for changing.When notification message, the function (funcptr) of the automatic invokes application registration in ddr interface storehouse is finished function corresponding and is handled.
The concrete steps that data are duplicated in the present embodiment are:
(1) real time data synchronized process DDR initialization:
A) program start, according to configuration parameter, initialization shared drive data field,
B) master slave mode with process is changed to from the attitude state,
C) after the DDR process initiation success, be 2 seconds the cycle to other DDR process broadcasting heartbeat message HBM,
D) receive other DDR process heartbeat message HBM, and upgrade the local activity tabulation,, then in effort scale, delete the corresponding information of this process as surpassing the heartbeat message HBM message of not receiving certain in 5 seconds of regulation.
(2) main attitude election:
A) main attitude DDR process election:
If receive that from attitude DDR process main DDR process will withdraw from message MDM message or exceeds schedule time, generally be decided to be 2.5~3 seconds, do not receive data consistent check message MCM message, then think main attitude process cisco unity malfunction.Therefore the relation of minimum IP address in this machine of DDR process check IP address and the local activity DDR process list:
If this machine IP address minimum thinks that then this process should switch to main attitude, broadcast transmission master DDR process alive message MEM message is main attitude with this machine DDR process switching simultaneously.
If this machine IP address is not minimum, then be left intact.
B) how main attitude DDR process collision solves:
If main attitude DDR process is received data consistent check message MCM message or main attitude DDR process alive message MEM, then think and have a plurality of main attitudes, therefore compare with the IP address of this machine IP address and the MCM message of receiving: if this machine IP is bigger, then thinking has other DDR process to be more suitable for main attitude, is from attitude with this machine DDR process switching; If this machine IP is less, then send MEM message to corresponding DDR process, other main attitude DDR process is done same judgement.
(3) data consistency periodic test:
Regular 1 second broadcast transmission data consistent check message of main attitude DDR process MCM message comprises the LTS and the UID field of every data of main attitude DDR process in the MCM message, the DDR process will be done following the processing after receiving MCM message:
A) main attitude DDR process is received MCM message, then knows a plurality of main attitudes to occur, and the how main attitude DDR process collision that starts in (2) step solves, and guarantees to have only a main attitude process at last.
B) each receives MCM message from attitude DDR process, then according to content in the MCM message one by one with the LTS of local data and UID relatively.
If LTS (MCM)>LTS (local record), this of then main attitude DDR process is recorded as more recent version, send RDM message to main attitude DDR process, ask main attitude DDR process to retransmit this data, after main attitude DDR process is received RDM message, random delay 0--50 millisecond is after the time, with the latest edition data of these data of RRM information broadcast.The purpose that sends RRM message behind the employing random delay time is that many RDM message compression from attitude DDR process possible receiving are a RRM acknowledgement messaging, reduces network traffics and DDR process processing time.
If LTS (MCM)<LTS (local record), the data of then main attitude DDR process are legacy version, send RRM message to main attitude DDR process, to upgrade this data from attitude DDR process.
If a LTS (MCM)=LTS (local record), and UID (MCM)!=UID (localrecord) illustrates to have data collision, but the program that also is not employed solves, send RDM message to main attitude DDR process, ask main attitude to retransmit this data, after main attitude DDR process is received RDM message, with the latest edition data of these data of RRM information broadcast.
If LTS (MCM)=LTS (local record), and UID (MCM)=UID (localrecord) illustrate both versions of data unanimities, do not do any processing.
(4) Data Update:
A) when application program need be upgraded certain bar data, the Data Update Set function that calls the ddr interface storehouse sends to local DDR process with data to be updated, and local DDR process adopts more new data of following algorithm:
I) LTS with these data adds 1, and UID is updated to the UID of calling process, upgrades local data;
Ii) to other DDR process broadcast transmission Data Update UDM message;
B) the DDR process receives Data Update message UDM or data re-transmission response message RRM message by following algorithm process:
I) as LTS (UDM)<LTS (local record) or LTS (RRM)<LTS (local record), do not process;
Ii) as LTS (UDM)>LTS (local record) or LTS (RRM)>LTS (local record) then with the local corresponding data of the Data Update among UDM or the RRM, comprise LTS and UID field;
Iii) as LTS (UDM)=LTS (local record), and UID (UDM)!=UID (localrecord); Or LTS (RRM)=LTS (local record), an and UID (RRM)!=UID (localrecord) illustrates to produce the Data Update conflict, goes on foot " Data Update conflict resolution algorithm " according to (6) and manages conflict, and write down detailed daily record;
Iv) other situation does not then process.
(5) Data Update notice: the DDR process that receives Data Update sends the Data Update notification message to local ddr interface storehouse, and application program obtains this notification message by automatic regular polling or registered callbacks function mode.
(6) the Data Update conflict solves:
When the DDR process is received UDM or RRM message and detect Data Update when conflict, at this moment handle by following method:
A) local data with these data is saved in the colliding data tabulation;
B) with the local corresponding data of the Data Update of receiving;
C) the ddr interface storehouse of the UID of the Data Update person in colliding data correspondence sends the data collision notification message, solves this conflict by the local application of ddr interface storehouse correspondence;
D) send the Data Update notice to the ddr interface storehouse of local other application program.
(7) data read:
Application program directly reads the data strip data that need from local shared drive by the ddr interface storehouse
(8) the DDR process withdraws from: when the DDR process that is in main attitude withdraws from, broadcast transmission master DDR process will withdraw from message MDM message, and this message is represented withdrawing from of whole process.