CN101771567A - Method for lasting data based on blog or dual-system warm backing up for autonomous internal memory database - Google Patents
Method for lasting data based on blog or dual-system warm backing up for autonomous internal memory database Download PDFInfo
- Publication number
- CN101771567A CN101771567A CN200910212673A CN200910212673A CN101771567A CN 101771567 A CN101771567 A CN 101771567A CN 200910212673 A CN200910212673 A CN 200910212673A CN 200910212673 A CN200910212673 A CN 200910212673A CN 101771567 A CN101771567 A CN 101771567A
- Authority
- CN
- China
- Prior art keywords
- internal memory
- memory database
- takes place
- daily record
- journal file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a method for lasting data based on a blog or dual-system warm backing up for an autonomous internal memory database which is fit for a telecom charging system. The method comprises the following steps of: (a) recording an element of a change of a host machine internal memory as a packet, (b) sending the packet to a daemon process rcvmsg by a first MdbServer server process through an information array, (c) storing cache data in other threads into the blog by a last thread of the daemon process rcvmsg, (d) transmitting the blog to a spare machine from the host machine, (e) arranging a received blog file by the daemon process synfile in the spare machine; and (f) writing a packet to be synchronized into a storage unit of the spare machine. The invention is characterized by simple realization and higher efficiency.
Description
Technical field
The invention relates to a kind of telecommunications industry billing and accounting system, and particularly relevant for lasting data or the double hot standby method of a kind of autonomous internal memory database based on daily record.
Background technology
Existing autonomous internal memory database product MDB, do not support lasting data and two-node cluster hot backup, the backup of data (to disk file) need be carried out by the manual backup command of carrying out, this backup of intervening artificially is very dangerous, when service system collapses, very easily cause losing of critical data in the internal memory.In addition on the one hand, as yet not between convalescence, business is forced to interrupt, and causes damage to the user at the fault main frame.Can consult on the net at present about the data of memory database two-node cluster hot backup seldom.(as TimeTen, Altibase) etc., its realization mechanism is not clear, is difficult to use for reference for the ripe commercial memory database of main flow.
Summary of the invention
It is simple to the purpose of this invention is to provide a kind of realization, and the higher autonomous internal memory database of efficient is based on the lasting data or the double hot standby method of daily record.
For reaching above-mentioned purpose, the invention provides lasting data or the double hot standby method of a kind of autonomous internal memory database based on daily record, be applicable to the telecommunications industry billing and accounting system.The telecommunications industry billing and accounting system comprises the main frame and the standby host of communicating by letter with client process, main frame comprises that a MdbServer service processes, rcvmsg write the daily record process and first memory database, the one MdbServer service processes is communicated by letter with client process by ICP/IP protocol, and standby host comprises synfile synchronized process, the 2nd MdbServer service processes and storage element.
Autonomous internal memory database comprises the following steps: based on the lasting data or the double hot standby method of daily record
(a). the key element that host memory is taken place in change is recorded as a bag;
(b). the first MdbServer service processes is sent to finger daemon rcvmsg by message queue with bag;
(c). last thread of finger daemon rcvmsg is with data cached being saved in the journal file in all the other threads;
(d). journal file is transferred to standby host by main frame;
(e). the journal file that in standby host, utilizes finger daemon synfile row reclosing to receive;
(f). will need synchronous bag to write the storage element of standby host.
Among the present invention, wherein be to utilize general ftp program that journal file is transferred to standby host by main frame in the step (d).
Among the present invention, the key element that change takes place in the step (a) is as follows:
The table space coding of change takes place;
The original position of change takes place;
The length of change takes place;
The content of change takes place;
The serial number of change takes place, and this serial number must be according to the sequencing of operation, must be continuously, and can not the number of jumping.
Among the present invention, wherein be to utilize finger daemon synfile in internal memory, to organize journal file and arrange heavily in the step (e) according to the sequencing of serial number.
In sum, the present invention is the relative position that upgrades according to memory database, and the record update content is to journal file, be synchronized to the disk mirroring file or the memory database of standby host again according to relative position by journal file, comparatively speaking, has realization simply, the characteristics that efficient is higher.More briefly, when main frame took place to upgrade, 2 memory databases (or memory database and disk file) were according to corresponding one by one carry out synchronous of physical location.
Description of drawings
Fig. 1 carries out the system architecture diagram of lasting data or two-node cluster hot backup for the telecommunications industry billing and accounting system.
Fig. 2 is that autonomous internal memory database is based on the lasting data of daily record or the flow chart of double hot standby method.
Embodiment
For above and other objects of the present invention, feature and advantage can be become apparent, preferred embodiment cited below particularly, and conjunction with figs. are described in detail below.
Fig. 1 carries out the system architecture diagram of lasting data or two-node cluster hot backup for the telecommunications industry billing and accounting system.As shown in Figure 1, the telecommunications industry billing and accounting system comprises the A main frame 2 and the B main frame (standby host) 3 of communicating by letter with client process 1 respectively.A main frame 2 comprises that a MdbServer service processes 21, rcvmsg write daily record process 22, A memory database 23.The one MdbServer service processes 21 is communicated by letter with client process 1 by ICP/IP protocol.The one MdbServer service processes 21 is by the DIRECT API Access A memory database 23 of memory database, the change records of internal memory was bag when the one MdbServer service processes 21 was will be with A memory database 23 mutual, message queue and rcvmsg by IPC communicate, have some special threads to be responsible for receiving the buffer memory that these wrap thread among the rcvmsg, last thread is responsible for regularly the information in other thread buffer memorys being write journal file among the rcvmsg.B main frame 3 comprises synfile synchronized process 31, the 2nd MdbServer service processes 32, storage element 33, and wherein storage element 33 can be disk file or memory database.After the synfile synchronized process is written into internal memory row weight with journal file, write storage element 33 (as memory database) by DIRECT API, the 2nd MdbServer process 32 externally provides the access interface for storage element 33 (as memory database), can be undertaken by the DIRECT API of memory database alternately between the 2nd MdbServer process 32 and the storage element 33 (as memory database).
Fig. 2 is that autonomous internal memory database is based on the lasting data of daily record or the flow chart of double hot standby method.Please also refer to Fig. 1 to Fig. 2, autonomous internal memory database comprises the following steps: based on the lasting data or the double hot standby method of daily record
The key element that change takes place for S101, record host memory is a bag.Here mainly contain following key element and need record: A, the table space that takes place to change are encoded; The original position of B, generation change; The length of C, generation change; The content of D, generation change; E: the serial number of change takes place, and this serial number must be according to the sequencing of operation, must be continuously, can not the number of jumping, more than 5 key elements be recorded as a bag.
S103, a MdbServer service processes are sent to finger daemon rcvmsg by message queue with bag.In order to improve the transmitting efficiency of message queue, system adopts a plurality of message queues to send and receive message, and quantity can dispose, and general recommendations is not less than 10.Because message queue is that operating system guarantees synchronous between each process in essence, ideally same time read-write only has 1 process at certain message queue of read-write, and then efficient wants high relatively.In addition, an operation, such as inserting data, may produce a plurality of packets, because it is made up of a plurality of operations, such as: need to upgrade the record number of controlled area, allocation of space and the data that need carry out the data field write, needing the index information of renewal index area etc. our hypothesis has 3 bags and need write message queue, writes fashionable will the circulation according to the mode that 1 bag writes a formation and writes formation.Guarantee that as far as possible message queue can be by the utilization of equilibrium.The A main frame has 1 finger daemon rcvmsg to be responsible for the reception bag specially, and this process is the finger daemon of a multithreading.Thread Count=message queue number+1.Be man-to-man relation between message queue and the thread, thread receives from message queue after the data, leaves in the buffer memory of thread inside.
Last thread of S105, finger daemon rcvmsg is with data cached being saved in the journal file in all the other threads, and the quantity according to the bag that is provided with in the configuration file during preservation writes journal file 24.
S107, journal file is transferred to standby host by main frame.Journal file is transferred to B main frame (standby host) by the A main frame and finishes by general ftp program (gftp).
S109, the journal file that in standby host, utilizes finger daemon synfile rearrangement to receive.Synfile will organize journal file 34 and arrange weight according to the sequencing of serial number in internal memory.The principle that row is heavy is roughly as follows: because the writing to write according to relative position in internal memory and (comprised the table space coding in the bag of bag, side-play amount and length), be not that each bag all needs to write disk file or internal memory, such as for same table space, identical side-play amount and length, 3 renewals are arranged, as long as write last.Can effectively reduce like this and write indegree, thereby improve the efficient that writes.
S111, will need synchronous bag to write the storage element of standby host.If default is for being synchronized to the disk file of B main frame (standby host), synfile then will need synchronous bag content to encode according to table space, and side-play amount, length write corresponding disk file correspondence position, thereby finish data synchronization.If default is for being synchronized to the memory database of B main frame (standby host), synfile then will need the content of synchronous bag according to table space coding+side-play amount+length, call the correspondence position in the correspondence table space of DIRECT api interface write memory database, thereby finish data synchronization.It should be noted that, the message queue series DIRECT API that system adopts is the IPC message queue DIRECT API that meets the posix standard under the UNIX, on main flow unix system environment, has good portability, concrete DIRECT API is a common practise to the people who has common knowledge in this area, no longer describes here.
In the A hostdown, need switch under the situation of B main frame (standby host), need to check and confirm:
1: finger daemon rcvmsg accepts writing of message procedure has not had journal file under the Log Directory;
2: the journal file under the working directory of finger daemon synfile synchronized process all disposes.
If satisfied above-mentioned 2 points, then application can be switched to B main frame (standby host).
10000 records of test insert/update/delete, every is write down commit once:
| ??Insert(us) | ??Update(us) | ??Delete(us) | |
| NG (NG is the abbreviation of NGBOSS, is the business support system of China Mobile) requirement | ???80 | ???60 | ???50 |
| Free of data enduring quality MDB performance (band two-node cluster hot backup) | ??11??26 | ??30??33 | ??37??43 |
As can be seen from the above table, use two-node cluster hot backup, performance has certain decline, but can satisfy the requirement of NG for memory database on the whole.
In sum, the present invention is the relative position that upgrades according to memory database, and the record update content is to journal file, be synchronized to the disk mirroring file or the memory database of standby host again according to relative position by journal file, comparatively speaking, has realization simply, the characteristics that efficient is higher.More briefly, when main frame took place to upgrade, the memory database of B main frame (standby host) or disk mirroring file were according to corresponding one by one carry out synchronous of physical location.
Concrete case study on implementation only is a preferable case study on implementation of the present invention described in the present invention, is not to be used for limiting practical range of the present invention.Be that all equivalences of doing according to the content of the present patent application claim change and modification, all should be as technology category of the present invention.
Claims (5)
1. an autonomous internal memory database is based on the lasting data or the double hot standby method of daily record, be applicable to the telecommunications industry billing and accounting system, the telecommunications industry billing and accounting system comprises the main frame and the standby host of communicating by letter with client process, main frame comprises that a MdbServer service processes, rcvmsg write the daily record process and first memory database, the one MdbServer service processes is communicated by letter with client process by ICP/IP protocol, standby host comprises synfile synchronized process, the 2nd MdbServer service processes and storage element, it is characterized in that double hot standby method comprises the following steps:
(a). the key element that host memory is taken place in change is recorded as a bag;
(b). the first MdbServer service processes is sent to finger daemon rcvmsg by message queue with bag;
(c). last thread of finger daemon rcvmsg is with data cached being saved in the journal file in all the other threads;
(d). journal file is transferred to standby host by main frame;
(e). the journal file that in standby host, utilizes finger daemon synfile row reclosing to receive;
(f). will need synchronous bag to write the storage element of standby host.
2. autonomous internal memory database according to claim 1 is characterized in that based on the lasting data or the double hot standby method of daily record, wherein is to utilize general ftp program that journal file is transferred to standby host by main frame in the step (d).
3. autonomous internal memory database according to claim 1 is characterized in that based on the lasting data or the double hot standby method of daily record the key element that change takes place in the step (a) is as follows:
The table space coding of change takes place;
The original position of change takes place;
The length of change takes place;
The content of change takes place;
The serial number of change takes place, and this serial number must be according to the sequencing of operation, must be continuously, and can not the number of jumping.
4. autonomous internal memory database according to claim 3 is based on the lasting data or the double hot standby method of daily record, it is characterized in that, wherein be to utilize finger daemon synfile to organize journal file according to the sequencing of serial number in internal memory and arrange heavily in the step (e).
5. autonomous internal memory database according to claim 3 is based on the lasting data or the double hot standby method of daily record, it is characterized in that wherein said operation is the writing or the insertion of index area index information of the writing of distribution, recorded content, controlled area record sum of memory space.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200910212673A CN101771567A (en) | 2009-11-10 | 2009-11-10 | Method for lasting data based on blog or dual-system warm backing up for autonomous internal memory database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200910212673A CN101771567A (en) | 2009-11-10 | 2009-11-10 | Method for lasting data based on blog or dual-system warm backing up for autonomous internal memory database |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN101771567A true CN101771567A (en) | 2010-07-07 |
Family
ID=42504183
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN200910212673A Pending CN101771567A (en) | 2009-11-10 | 2009-11-10 | Method for lasting data based on blog or dual-system warm backing up for autonomous internal memory database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101771567A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102855163A (en) * | 2011-06-27 | 2013-01-02 | 华为软件技术有限公司 | Memory database hot-standby method and host |
| CN105872574A (en) * | 2016-03-30 | 2016-08-17 | 乐视控股（北京）有限公司 | Live broadcast message release optimization method and system |
| CN106021348A (en) * | 2016-05-09 | 2016-10-12 | 深圳市百米生活股份有限公司 | A log file rolling storage system |
| CN106202307A (en) * | 2016-07-01 | 2016-12-07 | 百势软件(北京)有限公司 | A kind of batch log preservation method and device |
| CN109491611A (en) * | 2018-11-07 | 2019-03-19 | 郑州云海信息技术有限公司 | A kind of metadata rule method, device and equipment |
| CN110597904A (en) * | 2018-05-25 | 2019-12-20 | 海能达通信股份有限公司 | Data synchronization method, standby machine and host machine |
-
2009
- 2009-11-10 CN CN200910212673A patent/CN101771567A/en active Pending
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102855163A (en) * | 2011-06-27 | 2013-01-02 | 华为软件技术有限公司 | Memory database hot-standby method and host |
| CN102855163B (en) * | 2011-06-27 | 2016-03-30 | 华为软件技术有限公司 | A kind of memory database hot-standby method and main frame |
| CN105872574A (en) * | 2016-03-30 | 2016-08-17 | 乐视控股（北京）有限公司 | Live broadcast message release optimization method and system |
| CN106021348A (en) * | 2016-05-09 | 2016-10-12 | 深圳市百米生活股份有限公司 | A log file rolling storage system |
| CN106202307A (en) * | 2016-07-01 | 2016-12-07 | 百势软件(北京)有限公司 | A kind of batch log preservation method and device |
| CN110597904A (en) * | 2018-05-25 | 2019-12-20 | 海能达通信股份有限公司 | Data synchronization method, standby machine and host machine |
| CN110597904B (en) * | 2018-05-25 | 2023-11-24 | 海能达通信股份有限公司 | Data synchronization method, standby machine and host machine |
| CN109491611A (en) * | 2018-11-07 | 2019-03-19 | 郑州云海信息技术有限公司 | A kind of metadata rule method, device and equipment |
| CN109491611B (en) * | 2018-11-07 | 2021-11-09 | 郑州云海信息技术有限公司 | Metadata dropping method, device and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101997918B (en) | Method for allocating mass storage resources according to needs in heterogeneous SAN (Storage Area Network) environment | |
| CN101854388B (en) | Method and system concurrently accessing a large amount of small documents in cluster storage | |
| CN101233498B (en) | Data operations in flash memories utilizing direct data file stroage | |
| CN102662992B (en) | Method and device for storing and accessing massive small files | |
| CN105122241B (en) | With database engine and the Database Systems for being independently distributed formula storage service | |
| CN102222085B (en) | Data de-duplication method based on combination of similarity and locality | |
| US9436571B2 (en) | Estimating data storage device lifespan | |
| CN101771567A (en) | Method for lasting data based on blog or dual-system warm backing up for autonomous internal memory database | |
| CN104978362B (en) | Data migration method, device and the meta data server of distributed file system | |
| CN103229171B (en) | Snapshot based replication | |
| CN106708427A (en) | Storage method suitable for key value pair data | |
| CN105404469A (en) | Video data storage method and system | |
| US9424156B2 (en) | Identifying a potential failure event for a data storage device | |
| CN104765575A (en) | Information storage processing method | |
| CN103139300A (en) | Virtual machine image management optimization method based on data de-duplication | |
| CN101231619A (en) | Method for managing dynamic internal memory base on discontinuous page | |
| US9557938B2 (en) | Data retrieval based on storage device activation schedules | |
| CN107357680A (en) | Data processing equipment and the method for data processing | |
| CN101344861A (en) | Smart card internal memory management method | |
| CN103986779B (en) | A kind of application data transmission method, device and terminal device | |
| CN102779138A (en) | Hard disk access method of real time data | |
| US9436524B2 (en) | Managing archival storage | |
| CN108089825B (en) | Storage system based on distributed cluster | |
| US20100153346A1 (en) | Data integrity in a database environment through background synchronization | |
| CN104765574A (en) | Data cloud storage method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Open date: 20100707 |