CN110442560A - Method, apparatus, server and storage medium are recurred in a kind of log - Google Patents

Method, apparatus, server and storage medium are recurred in a kind of log Download PDF

Info

Publication number
CN110442560A
CN110442560A CN201910748717.7A CN201910748717A CN110442560A CN 110442560 A CN110442560 A CN 110442560A CN 201910748717 A CN201910748717 A CN 201910748717A CN 110442560 A CN110442560 A CN 110442560A
Authority
CN
China
Prior art keywords
log
redo
database
page
redo log
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910748717.7A
Other languages
Chinese (zh)
Other versions
CN110442560B (en
Inventor
韩朱忠
王巍
王海龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Dameng Database Co Ltd
Original Assignee
Shanghai Dameng Database Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Dameng Database Co Ltd filed Critical Shanghai Dameng Database Co Ltd
Priority to CN201910748717.7A priority Critical patent/CN110442560B/en
Publication of CN110442560A publication Critical patent/CN110442560A/en
Application granted granted Critical
Publication of CN110442560B publication Critical patent/CN110442560B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/1805Append-only file systems, e.g. using logs or journals to store data
    • G06F16/1815Journaling file systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/182Distributed file systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28Databases characterised by their database models, e.g. relational or object models
    • G06F16/284Relational databases

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The embodiment of the invention discloses a kind of logs to recur method, apparatus, server and storage medium, wherein this method is applied to primary database, this method comprises: if it is determined that when current database example modifies target data page for the first time, generation is redo log, and is redo log wherein redoing log and synchronizing to redo log including page with conventional;Transmission is redo log to standby database, so that standby database root carries out log recurrence according to the Log Types redo log.The technical solution of the embodiment of the present invention, primary database is when generation is redo log, by increase a kind of new page synchronize redo log redo log type, standby database can be made not need additional logic judgment, according to Log Types can parallel processing page synchronize redoing log except redoing log, greatly improve the recurrence efficiency and performance of standby database.

Description

Method, apparatus, server and storage medium are recurred in a kind of log
Technical field
The present embodiments relate to database technical field more particularly to a kind of log recur method, apparatus, server and Storage medium.
Background technique
Relational database management system is usually used to reform (REDO) log to guarantee the persistence of affairs, any modification number According to operation, can all generate REDO log in internal system, and after ensuring REDO log brush, the data page of modification is just written Disk.Log serial number (Log Sequence Number, LSN) is one of relational database management system internal maintenance BIGINT Type Value, for identifying REDO log recording, each LSN value represents a system inside relational database management system The set of column modification data manipulation.
In the single-node data library database of a database instance (include), LSN value, which has, to be increased continuously and global Unique characteristic, when data modification generates REDO log, system can distribute a LSN value, and the size of LSN value represents data and repairs The sequencing changed.When the system failure is restarted, REDO log is successively reformed according to LSN value from small to large, so that it may in strict accordance with The modification sequence of data, the state before restoring data to the system failure.Shared storing data library cluster may include multiple numbers According to library example, user can connect any database example starting affairs, modifies data and generates REDO log, each database Example independent maintenance log information, the LSN value inside database instance, which also has, is increased continuously characteristic, unique trait.For area Divide the sequencing of modification, it is ensured that the LSN value that the same data page of disparate databases example modifications generates is incremental.Therefore, The LSN value of each data instance is not necessarily continuously in shared storing data library cluster, does not also have uniqueness.
Data guardian system is the current relatively common solution for database high availability, usually by a master Database (i.e. the host of database) and one or several for database (i.e. the standby host of database) composition.Standby database is logical Cross the REDO log that recurrence primary database generates keeps data synchronous with primary database.Inside data guardian system, if main Database is shared storing data library cluster, then will receive the REDO day that multiple database instances send over for database Will, in order to guarantee data consistency, it is necessary to REDO log successively be reformed according to LSN value from small to large, otherwise can destroy data Correctness.In the prior art, a kind of feasible method is after receiving the REDO log of primary database, by the REDO log of multichannel REDO log is successively recurred again after merger sequence, but this method has the following problems: because of multiple data in primary database The REDO log of library example be it is independent, after standby database receives the REDO log of data instance generation, need clear Know whether other database instances produce REDO log, be otherwise unable to complete merger sequence, and standby database need by It is recurred again after multichannel REDO log merger sequence, treatment effeciency is unable to meet demand, and repeatability can be guaranteed.
Summary of the invention
The embodiment of the present invention provides a kind of log and recurs method, apparatus, server and storage medium, to optimize in database Scheme is recurred in the log of standby host, is improved log and is recurred efficiency.
In a first aspect, the embodiment of the invention provides a kind of logs to recur method, it is applied to primary database, comprising:
If it is determined that generation is redo log when current database example modifies target data page for the first time, wherein described It redoes log to synchronize to redo log including page and be redo log with conventional;
It is redo log described in transmission to standby database, so that the Log Types that the standby database root is redo log described in Carry out log recurrence.
Second aspect, the embodiment of the invention also provides a kind of logs to recur method, is applied to standby database, comprising:
Obtain redoing log for primary database transmission;
According to the Log Types redo log, log recurrence is carried out to described redo log, it is described to redo log packet It includes page and synchronizes and redo log and conventional redo log.
The third aspect, the embodiment of the invention also provides a kind of logs to recur device, is configured at primary database, comprising:
Log generation module, for if it is determined that being generated when current database example modifies target data page for the first time Redo log, wherein it is described redo log to synchronize including page redo log and conventional redo log;
Log sending module, for send it is described redo log to standby database so that described in the standby database root evidence The Log Types redo log carry out log recurrence.
Further, the log generation module includes modifying determination unit for the first time, the determination unit tool of modification for the first time Body is used for:
If it is determined that target data page brush, alternatively, upper one in the non-brush of the target data page and global buffer The corresponding database instance of modification record is different from the current database example, it is determined that the current database example is to institute Target data page is stated to be modified for the first time.
Further, the log generation module includes that page synchronizes and redoes log unit, and the page, which synchronizes, redoes log list Member is specifically used for:
Page log serial number of the target data page before the modification for the first time is determined as synchronization log sequence number, Described in synchronization log sequence number increments and unique;
Upper modification note based on the synchronization log sequence number and the target data page before the modification for the first time Corresponding database instance is recorded, the page is generated and synchronizes and redo log.
Further, described device further include:
Conventional log module, for if it is determined that the current database example non-repair for the first time to the target data page When changing, conventional redo log is generated.
Fourth aspect, the embodiment of the invention also provides a kind of logs to recur device, is configured at standby database, comprising:
Log acquisition module, for obtaining redoing log for primary database transmission;
Module is recurred in log, and the Log Types for redoing log according to carry out log weight to described redo log Drill, it is described redo log to synchronize including page redo log and conventional redo log.
Further, the log recurrence module includes:
First recur unit, if for it is described redo log to synchronize for the page redo log, day is reformed according to The synchronization log sequence number of will and the comparison result for recurring completion log serial number carry out log recurrence to described redo log.
Further, the first recurrence unit is specifically used for:
If described recur completes log serial number less than the synchronization log sequence number redo log, waiting behaviour is executed Make, until described recur completes log serial number more than or equal to the synchronization log sequence number redo log.
Further, the log recurrence module includes:
Second recurs unit, if redoing log for described redo log for the routine, parallel to the conventional weight It does log and carries out log recurrence, wherein the quantity routinely redo log is at least one.
5th aspect, the embodiment of the invention also provides a kind of server, the server includes:
One or more processors;
Storage device, for storing one or more programs;
When one or more of programs are executed by one or more of processors, so that one or more of processing Device realizes that method is recurred in log as described in relation to the first aspect, or method is recurred in the log as described in second aspect.
6th aspect, the embodiment of the invention also provides a kind of computer readable storage mediums, are stored thereon with computer Program realizes that method is recurred in log as described in relation to the first aspect when the program is executed by processor, or as described in second aspect Log recur method.
The embodiment of the present invention is determining that current database example modifies target data page for the first time by primary database When, generation is redo log, and transmission is redo log to standby database, so that standby database root is carried out according to the Log Types redo log Log is recurred, and is redo log wherein redoing log and synchronizing to redo log including page with conventional.The technical solution of the embodiment of the present invention, Primary database when generation is redo log, by increase a kind of new page synchronize redo log redo log type, can make Standby database does not need additional logic judgment, according to Log Types can parallel processing page synchronize the weight except redoing log Log is done, the recurrence efficiency and performance of standby database are greatly improved.
Detailed description of the invention
Fig. 1 is the flow chart of the log recurrence method in the embodiment of the present invention one;
Fig. 2 is the schematic diagram of the data guardian system in the embodiment of the present invention one;
Fig. 3 is the flow chart of the data modification process in the embodiment of the present invention one;
Fig. 4 is the flow chart of the log recurrence method in the embodiment of the present invention two;
Fig. 5 is the structural schematic diagram of the log recurrence device in the embodiment of the present invention three;
Fig. 6 is the structural schematic diagram of the log recurrence device in the embodiment of the present invention four;
Fig. 7 is the structural schematic diagram of the server in the embodiment of the present invention five.
Specific embodiment
The present invention is described in further detail with reference to the accompanying drawings and examples.It is understood that this place is retouched The specific embodiment stated is used only for explaining the present invention rather than limiting the invention.It also should be noted that in order to just Only the parts related to the present invention are shown in description, attached drawing rather than entire infrastructure.
Embodiment one
Fig. 1 is the flow chart of the log recurrence method in the embodiment of the present invention one, and the present embodiment is applicable to database In redo log the case where carrying out log recurrence, this method can recur device by the log that is configured in primary database and hold Row, which can be realized by the way of software and/or hardware, for example, the device is configured in server.
Fig. 2 is the schematic diagram of the data guardian system in the embodiment of the present invention one, which is only one and shows The quantity of example, standby database can be multiple.Data guardian system in figure includes a primary database 101 and a standby data Library 102, wherein primary database 101 is the shared storing data library cluster including two database instances, primary database 101 In include database instance 1 and database instance 2, each database instance individually generates and manages redoing log for itself.It is main Database instance 1 and redoing log for database instance 2 can be sent in standby database 102 by database 101, standby database 102 can redo log progress log recurrence to what is received, realize and keep data synchronous with primary database 101.
Method is recurred to log with the data guardian system in Fig. 2 in the present embodiment to be illustrated.
As shown in Figure 1, this method can specifically include:
S110, if it is determined that when current database example modifies target data page for the first time, generation is redo log.
Wherein, to be relational database server provide the example of data service for user to database instance, and user can be with The data service operation internal data therein provided by database instance.To the number for including in primary database in the present embodiment It is not construed as limiting according to the quantity of library example.Data page can be the basic unit of management memory space in database instance.It reforms (REDO) log may include page synchronize redo log and it is conventional redo log, it is a kind of newly-increased reform that page, which is synchronized and redo log, Log Types, page, which is synchronized and is redo log, to be the last modification relevant information before currently modify for data page Log recording, page synchronizes redo log in synchronization log sequence number increments and unique.
Further, it determines that current database example modifies target data page for the first time, may include: if it is determined that mesh Data page brush is marked, alternatively, the corresponding database instance of a upper modification record in the non-brush of target data page and global buffer It is different from current database example, it is determined that current database example modifies target data page for the first time.Wherein brush is Finger stores target data page into disk.
By inquiry global buffer in brush record, can determine target data page whether brush, however, it is determined that number of targets According to page brush, it is determined that other database instances except current database example modify to the target data page And brush, it needs to load newest target data page from disk, thus may determine that current database example is to current newest Target data page modification for the first time.Or, however, it is determined that the non-brush of target data page can then inquire the modification in global buffer Record.If a upper modification record corresponding database instance of the target data page in global buffer and current database example Difference, the i.e. corresponding database instance of Last modification of the target data page before current database example modifications are not worked as Preceding database instance can then determine that the target data page is modified by other database instances except current database example It crosses, needs to obtain newest target data page from the corresponding database instance of the modification record, thus may determine that current number According to the modification for the first time that library example is to current newest target data page.For example, database instance 1 has modified data page p1, still Non- brush, the corresponding database instance of the upper modification record recorded in global buffer at this time (Last modification record) are number According to library example 1, for the database instance other than database instance 1, then data page p1 is modified, as modified for the first time.
Further, generation page, which synchronizes, redoes log, and may include S111-S112 (not shown), specific:
S111, page log serial number of the target data page before modifying for the first time is determined as synchronization log sequence number.
Wherein, page synchronize redo log be it is a kind of it is newly-increased redo log type, synchronization log sequence number increments and unique, Position in journal file can be before routinely redoing log.It may include mesh that page, which synchronizes and redoes log the content of middle record, Mark file identification (ID), target data page page number, number of targets belonging to the mark (ID) of the affiliated table space of data page, target data page According to the synchronization log sequence number of page newest place database instance number and target data page, wherein table space belonging to target data page Mark can indicate that file identification belonging to target data page can be indicated using fil_id, target data page page using ts_id Number can using page_no indicate, the newest place database instance number of target data page can using ep_se indicate, number of targets It can be indicated using page_lsn according to the synchronization log sequence number of page.
It is available to be recorded in target data page before current database example modify for the first time to target data page This page of log serial number is determined as synchronization log sequence number after modifying for the first time by the page log serial number on head.
S112, it is corresponded to based on the upper modification record of synchronization log sequence number and target data page before modifying for the first time Database instance, generate page and synchronize and redoes log.
After determining synchronization log sequence number, however, it is determined that target data page brush does not know last modification target then Which the database instance of data page is specifically, need at this time for all database example except current database example it is equal It generates page to synchronize and redoes log, that is, generates multiple pages and synchronize and redo log.If it is determined that the non-brush of target data page, then can pass through The modification record of target data page determines before current database example is modified for the first time in global buffer, and a upper modification record is corresponding Database instance database instance number, the database instance number be the newest place database instance number of target data page, A page is only generated at this time to synchronize and redo log.
Furthermore, it is desirable to determine file identification and target belonging to the mark of table space belonging to target data page, target data page Data page page number.Based on table belonging to synchronization log sequence number, the newest place database instance number of target data page, target data page File identification and target data page page number belonging to the mark in space, target data page, generate page and synchronize and redoes log.
Furthermore, however, it is determined that current database example to target data page carry out it is non-modify for the first time when, generate conventional reforms day Will.When current database example carries out n-th modification to target data page, redoing log for generation only includes conventional reform day Will, wherein N is the integer more than or equal to 2.
S120, transmission are redo log to standby database, so that standby database root carries out day according to the Log Types redo log Will is recurred.
Specifically, this can be redo log after primary database generation is redo log and be sent to standby database, so that standby Database root carries out log recurrence according to the Log Types redo log.It is redo log if redoing log and being synchronized for page, waits page It synchronizes and redoes log the newest place database instance of target data page of middle record and recurred and then to current database example Carry out log recurrence;It is redo log if redoing log to be conventional, log recurrence is directly carried out by parallel thread.Routine is reformed The quantity of log can ensure that the performance of recurrence by parallel processing to be multiple.
In addition, using the page log serial number being recorded on data top margin in the present embodiment, dynamic modification system has been distributed Max log sequence number, it can be ensured that each database instance generated when modifying identical data page page synchronize redo log in it is same Log serial number is walked, has and is incremented by and unique characteristic.Therefore, page is synchronized and is redo log, it can be according to synchronization log sequence The sequence of row number carries out log recurrence.
Fig. 3 is the flow chart of the data modification process in the embodiment of the present invention one, referring to Fig. 3, current number in primary database The process modified according to library example to target data page is as follows: S11, initializing local log serial number.Initialize current number It is 0 according to the local log serial number in the example of library, local log serial number can be indicated using PTX_LSN.S12, target is obtained The modification authority of data page.S13, newest target data page is obtained.It is loaded by caching syncretizing mechanism from other examples or disk Newest target data page.S14, modification target data page and generation are redo log.Current database example modifications target data page And it generates page and synchronizes and redo log and conventional redo log.S15, judge whether page log serial number is greater than local logged sequence Number.The page log serial number recorded on target data top margin before modifying is read, page log serial number can use PAGE_LSN table Show, if PAGE_LSN > PTX_LSN, illustrates that other database instances modified the target data page, execute S16, otherwise hold Row S17.S16, local log serial number is modified.PTX_LSN=PAGE_LSN+1 is set.S17, whether modify and terminate.If PAGE_LSN≤PTX_LSN then illustrates to modify the target data page without other database instances, it is determined whether modification terminates, If so, executing S18, otherwise returns and execute S12.S18, judge whether system distribution log serial number is less than local logged sequence Number.Comparison system distributes log serial number and local log serial number, and system distribution log serial number can use CUR_LSN table Show, if CUR_LSN < PTX_LSN, illustrates that other database instances modified the target data page, then execute S19, otherwise, such as Up to the present fruit CUR_LSN >=PTX_LSN, is not yet modified after illustrating that target data page last change is complete, then holds Row S20.S19, modification system distribute log serial number.CUR_LSN=PTX_LSN is set.S20, local log serial number is modified Log serial number is distributed with system.PTX_LSN=CUR_LSN+1 is modified, new CUR_LSN is itself to add one.S21, page is determined Log serial number.Page log serial number PAGE_LSN is equal with PTX_LSN.S22, the modification authority for discharging target data page.
One database instance is before modifying data page, it is ensured that first obtains the modification authority of data page, and obtains newest Data page, after modifying data page, the page for generating database example, which synchronizes, to be redo log, while being read and being saved data top margin The page log serial number of upper record saves most big page log serial number if modification is related to multiple data pages.Complete data page After modification, the most big page log serial number being collected into is modified according to this, the currently allocated maximum system of adjustment system distributes day Will sequence number redoes log sequence number to determine that page synchronizes the page redo log and synchronizes.
Illustratively, if in primary database including three database instances: database instance 1, database instance 2 and data Library example 3, CUR_LSN=1000 in database instance 1, CUR_LSN=2000 in database instance 2, in database instance 3 CUR_LSN=3000.Database instance 1 modifies data page P1 (P1 first revision loads, PAGE_LSN=800 from disk), PTX_LSN=801 is arranged after reading data page in initial setting up PTX_LSN=0, after the completion of data page P1 modification, modification CUR_ LSN=1001, PTX_LSN=1001 modify PAGE_LSN=1001, and two pages of generation are synchronized and redo log, respectively (database instance 2,800), (database instance 3,800).Database instance 3 continues to modify data page P1, from database instance 1 PAGE_LSN=1001 is read, PTX_LSN=1002 is set, after the completion of data page P1 modification, modification CUR_LSN=3001, PTX_LSN=3001, modification PAGE_LSN=3001.And generate a page synchronize redo log for (database instance 1, 1001).Database instance 2 continues to modify data page P1, reads PAGE_LSN=3001 from database instance 3, PTX_LSN is arranged After the completion of=3002, data page P1 modification, PTX_LSN > CUR_LSN is found, modify CUR_LSN=3002, modify PAGE_LSN =3002, and generation one is synchronized and is redo log as (database instance 3,3001).
The technical solution of the present embodiment is determining that current database example is first to the progress of target data page by primary database When secondary modification, generation is redo log, and transmission is redo log to standby database, so that standby database root is according to the log class redo log Type carries out log recurrence, redoes log wherein redoing log and synchronizing to redo log including page with conventional.In the present embodiment, master data Library when generation is redo log, by increase a kind of new page synchronize redo log redo log type, standby data can be made Library does not need additional logic judgment, according to Log Types can parallel processing page synchronize and reform day except redoing log Will greatly improves the recurrence efficiency and performance of standby database.
Embodiment two
Fig. 4 is the flow chart of the log recurrence method in the embodiment of the present invention two.The present embodiment is applicable to database In redo log the case where carrying out log recurrence, this method can recur device by the log being configured in standby database and hold Row, which can be realized by the way of software and/or hardware, for example, the device is configured in server.Such as Fig. 4 institute Show, the method for the present embodiment specifically includes:
S210, redoing log for primary database transmission is obtained.
Wherein, redo log may include that page synchronizes and redoes log and conventional redo log.
Whether S220, to redo log be that page is synchronized and redo log.
According to the Log Types redo log, log recurrence is carried out to redoing log, first determines the log class redo log Type determines to redo log according to log content whether be that page is synchronized and redo log, if redoing log is that page is synchronized and redo log, S230 is then executed, is redo log if redoing log to be conventional, executes S240.
S230, the comparison result that log serial number is completed according to the synchronization log sequence number and recurrence that redo log, counterweight It does log and carries out log recurrence.
Standby database can be one recurrence task system of each primary database example allocation when carrying out log recurrence, The recurrence task system can recorde the case where correspondence database example log is recurred.Completion log serial number is wherein recurred to attach most importance to The newest log serial number that log recurrence is completed that task system is the storage of correspondence database example is drilled, can specifically be passed through APPLY_LSN [ep_seqno] indicates that the recurrence that meaning is database instance ep_seqno completes log serial number and is APPLY_LSN。
It is less than the synchronization log sequence number redo log if recurring and completing log serial number, executes waiting operation, until It recurs and completes log serial number more than or equal to the synchronization log sequence number redo log.If standby database processing is to reforming Page in log, which synchronizes, to be redo log, then may determine that recurrence complete log serial number whether be greater than or equal to redo log it is same Log serial number is walked, is less than the synchronization log sequence number redo log if recurring and completing log serial number, database of descriptions is real There are log serial numbers to be equal to redoing log for synchronization log sequence number by example ep_seqno, and current recurrence session can wait pending datas The recurrence thread of library example ep_seqno carries out log recurrence, completes and log serial number is greater than or equal to and reforms day until recurring The synchronization log sequence number of will, continues to recur.It is greater than or equal to the synchronization redo log if recurring and completing log serial number Log serial number, then database of descriptions example ep_seqno had recurred the weight that log serial number is equal to synchronization log sequence number Log is done, is withouted waiting for.
Illustratively, if database instance 2 generates database instance 1 page when modifying target data page, which synchronizes, is reformed day Will, the routine that standby database first receives database instance 1 are redo log, then are received the page of database instance 2 and synchronized and reform Log and it is conventional redo log, even if to recur speed fast for 2 thread of database instance under complications, encounter page and synchronize and reform day Will judges that recurring completion log serial number synchronizes the synchronization log sequence number redo log less than page, then meeting of database instance 2 etc. Pending data library example 1 recurs thread and recurs completion, is further continued for the recurrence itself routinely redo log.
If target data page brush, the page that standby database receives database instance 2, which synchronizes, to be redo log and conventional weight Log is done, and recurs completion log serial number and synchronizes the synchronization log sequence number redo log less than page, but due to long-range The principle of reading requirement log elder generation brush, it can be ensured that other database instances did not modify the target data page, counted at this time It is withouted waiting for according to library example 2.
S240, log recurrence is carried out to conventional redo log parallel.
The quantity wherein routinely redo log is at least one.Standby database by parallel thread to it is conventional redo log into Row log is recurred.
The technical solution of the present embodiment is redo log by what standby database obtained that primary database sends, according to reforming day The Log Types of will carry out log recurrence to redoing log, and redo log to synchronize including page and redo log and conventional redo log. In the present embodiment, due to include in redoing log a kind of new page synchronize redo log redo log type, standby database is not Need additional logic judgment, directly can according to Log Types can parallel processing page synchronize reforming except redoing log Log greatly improves the recurrence efficiency and performance of standby database.
Embodiment three
Fig. 5 is the structural schematic diagram of the log recurrence device in the embodiment of the present invention three, which recurs device and can set It is placed in primary database, the present embodiment is applicable to redo log the case where carrying out log recurrence in database.The present invention is real It applies log provided by example and recurs log recurrence method provided by the executable embodiment of the present invention one of device, have execution method Corresponding functional module and beneficial effect.
The device specifically includes log generation module 310 and log sending module 320, in which:
Log generation module 310, for if it is determined that being given birth to when current database example modifies target data page for the first time At redoing log, redo log wherein redoing log and synchronizing to redo log including page with conventional;
Log sending module 320 is redo log for sending to standby database, so that standby database root evidence was redo log Log Types carry out log recurrence.
The technical solution of the present embodiment is determining that current database example is first to the progress of target data page by primary database When secondary modification, generation is redo log, and transmission is redo log to standby database, so that standby database root is according to the log class redo log Type carries out log recurrence, redoes log wherein redoing log and synchronizing to redo log including page with conventional.In the present embodiment, master data Library when generation is redo log, by increase a kind of new page synchronize redo log redo log type, standby data can be made Library does not need additional logic judgment, according to Log Types can parallel processing page synchronize and reform day except redoing log Will greatly improves the recurrence efficiency and performance of standby database.
Further, log generation module 310 includes modifying determination unit for the first time, modifies determination unit for the first time and specifically uses In:
If it is determined that target data page brush, alternatively, a upper modification record in the non-brush of target data page and global buffer Corresponding database instance is different from current database example, it is determined that current database example carries out for the first time target data page Modification.
Further, log generation module 310 includes that page synchronizes and redoes log unit, and page synchronizes that redo log unit specific For:
Page log serial number of the target data page before modifying for the first time is determined as synchronization log sequence number, wherein synchronous day Will sequence number increments and unique;
The corresponding data of a upper modification record based on synchronization log sequence number and target data page before modifying for the first time Library example, generation page, which synchronizes, to be redo log.
Further, the device further include:
Conventional log module, for if it is determined that current database example to target data page carry out it is non-modify for the first time when, it is raw It is redo log at conventional.
Device is recurred in log provided by the embodiment of the present invention can be performed the recurrence of log provided by the embodiment of the present invention one Method has the corresponding functional module of execution method and beneficial effect.
Example IV
Fig. 6 is the structural schematic diagram of the log recurrence device in the embodiment of the present invention four, which recurs device and can set It is placed in standby database, the present embodiment is applicable to redo log the case where carrying out log recurrence in database.The present invention is real It applies log provided by example and recurs log recurrence method provided by the executable embodiment of the present invention two of device, have execution method Corresponding functional module and beneficial effect.
The device specifically includes log acquisition module 410 and module 420 is recurred in log, in which:
Log acquisition module 410, for obtaining redoing log for primary database transmission;
Module 420 is recurred in log, for carrying out log recurrence to redoing log, weight according to the Log Types redo log Doing log includes that page synchronizes and redoes log and conventional redo log.
The technical solution of the present embodiment is redo log by what standby database obtained that primary database sends, according to reforming day The Log Types of will carry out log recurrence to redoing log, and redo log to synchronize including page and redo log and conventional redo log. In the present embodiment, due to include in redoing log a kind of new page synchronize redo log redo log type, standby database is not Need additional logic judgment, directly can according to Log Types can parallel processing page synchronize reforming except redoing log Log greatly improves the recurrence efficiency and performance of standby database.
Further, log recurrence module 420 includes:
First recurs unit, if redoing log for redoing log to synchronize for page, according to the synchronization log redo log Sequence number and the comparison result for recurring completion log serial number carry out log recurrence to redoing log.
Further, the first recurrence unit is specifically used for:
It is less than the synchronization log sequence number redo log if recurring and completing log serial number, executes waiting operation, until It recurs and completes log serial number more than or equal to the synchronization log sequence number redo log.
Further, log recurrence module 420 includes:
Second recurs unit, if redoing log for redoing log to be conventional, carries out day to conventional redo log parallel Will is recurred, wherein the quantity routinely redo log is at least one.
Device is recurred in log provided by the embodiment of the present invention can be performed the recurrence of log provided by the embodiment of the present invention two Method has the corresponding functional module of execution method and beneficial effect.
Embodiment five
Fig. 7 is the structural schematic diagram of the server in the embodiment of the present invention five.Fig. 7, which is shown, to be suitable for being used to realizing the present invention The block diagram of the exemplary servers 512 of embodiment.The server 512 that Fig. 7 is shown is only an example, should not be to the present invention The function and use scope of embodiment bring any restrictions.
As shown in fig. 7, server 512 is showed in the form of generic server.The component of server 512 may include but not Be limited to: one or more processor 516, storage device 528 connect different system components (including storage device 528 and processing Device 516) bus 518.
Bus 518 indicates one of a few class bus structures or a variety of, including storage device bus or storage device control Device processed, peripheral bus, graphics acceleration port, processor or total using the local of any bus structures in a variety of bus structures Line.For example, these architectures include but is not limited to industry standard architecture (Industry Subversive Alliance, ISA) bus, microchannel architecture (Micro Channel Architecture, MAC) bus is enhanced Isa bus, Video Electronics Standards Association (Video Electronics Standards Association, VESA) local are total Line and peripheral component interconnection (Peripheral Component Interconnect, PCI) bus.
Server 512 typically comprises a variety of computer system readable media.These media can be it is any being capable of bedding and clothing The usable medium that business device 512 accesses, including volatile and non-volatile media, moveable and immovable medium.
Storage device 528 may include the computer system readable media of form of volatile memory, such as arbitrary access Memory (Random Access Memory, RAM) 530 and/or cache memory 532.Server 512 can be further Including other removable/nonremovable, volatile/non-volatile computer system storage mediums.Only as an example, storage system System 534 can be used for reading and writing immovable, non-volatile magnetic media (Fig. 7 do not show, commonly referred to as " hard disk drive ").To the greatest extent It is not shown in pipe Fig. 7, the disc driver for reading and writing to removable non-volatile magnetic disk (such as " floppy disk ") can be provided, with And to removable anonvolatile optical disk, such as CD-ROM (Compact Disc Read-Only Memory, CD-ROM), number Word optic disk (Digital Video Disc-Read Only Memory, DVD-ROM) or other optical mediums) read-write CD Driver.In these cases, each driver can be connected by one or more data media interfaces with bus 518. Storage device 528 may include at least one program product, which has one group of (for example, at least one) program module, These program modules are configured to perform the function of various embodiments of the present invention.
Program/utility 540 with one group of (at least one) program module 542 can store in such as storage dress It sets in 528, such program module 542 includes but is not limited to operating system, one or more application program, other program moulds It may include the realization of network environment in block and program data, each of these examples or certain combination.Program module 542 usually execute function and/or method in embodiment described in the invention.
Server 512 can also be with one or more external equipments 514 (such as keyboard, direction terminal, display 524 etc.) Communication, can also be enabled a user to one or more terminal interact with the server 512 communicate, and/or with make the clothes Any terminal (such as network interface card, modem etc.) that business device 512 can be communicated with one or more of the other computing terminal Communication.This communication can be carried out by input/output (I/O) interface 522.Also, server 512 can also be suitable by network Orchestration 520 and one or more network (such as local area network (Local Area Network, LAN), wide area network (Wide Area Network, WAN) and/or public network, such as internet) communication.As shown in fig. 7, network adapter 520 passes through bus 518 It is communicated with other modules of server 512.It should be understood that although not shown in the drawings, can be used in conjunction with server 512 other hard Part and/or software module, including but not limited to: microcode, terminal driver, redundant processor, external disk drive array, magnetic Disk array (Redundant Arrays of Independent Disks, RAID) system, tape drive and data backup Storage system etc..
The program that processor 516 is stored in storage device 528 by operation, thereby executing various function application and number According to processing, such as realize that method is recurred in log provided by any embodiment of the invention.
Embodiment six
The embodiment of the present invention six additionally provides a kind of computer readable storage medium, is stored thereon with computer program, should Realize that method is recurred in log provided by any embodiment of the invention such as when program is executed by processor.
The computer storage medium of the embodiment of the present invention, can be using any of one or more computer-readable media Combination.Computer-readable medium can be computer-readable signal media or computer readable storage medium.It is computer-readable Storage medium for example may be-but not limited to-the system of electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor, device or Device, or any above combination.The more specific example (non exhaustive list) of computer readable storage medium includes: tool There are electrical connection, the portable computer diskette, hard disk, random access memory (RAM), read-only memory of one or more conducting wires (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD- ROM), light storage device, magnetic memory device or above-mentioned any appropriate combination.In this document, computer-readable storage Medium can be any tangible medium for including or store program, which can be commanded execution system, device or device Using or it is in connection.
Computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal, Wherein carry computer-readable program code.The data-signal of this propagation can take various forms, including but unlimited In electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be that computer can Any computer-readable medium other than storage medium is read, which can send, propagates or transmit and be used for By the use of instruction execution system, device or device or program in connection.
The program code for including on computer-readable medium can transmit with any suitable medium, including --- but it is unlimited In wireless, electric wire, optical cable, RF etc. or above-mentioned any appropriate combination.
The computer for executing operation of the present invention can be write with one or more programming languages or combinations thereof Program code, described program design language include object oriented program language-such as Java, Smalltalk, C++, It further include conventional procedural programming language-such as " C " language or similar programming language.Program code can be with It fully executes, partly execute on the user computer on the user computer, being executed as an independent software package, portion Divide and partially executes or executed on remote computer or terminal completely on the remote computer on the user computer.It is relating to And in the situation of remote computer, remote computer can pass through the network of any kind --- including local area network (LAN) or extensively Domain net (WAN)-be connected to subscriber computer, or, it may be connected to outer computer (such as provided using Internet service Quotient is connected by internet).
Note that the above is only a better embodiment of the present invention and the applied technical principle.It will be appreciated by those skilled in the art that The invention is not limited to the specific embodiments described herein, be able to carry out for a person skilled in the art it is various it is apparent variation, It readjusts and substitutes without departing from protection scope of the present invention.Therefore, although being carried out by above embodiments to the present invention It is described in further detail, but the present invention is not limited to the above embodiments only, without departing from the inventive concept, also It may include more other equivalent embodiments, and the scope of the invention is determined by the scope of the appended claims.

Claims (12)

1. method is recurred in a kind of log, which is characterized in that be applied to primary database, comprising:
If it is determined that generation is redo log when current database example modifies target data page for the first time, wherein described reform Log includes that page synchronizes and redoes log and conventional redo log;
It is redo log described in transmission to standby database, so that the Log Types that the standby database root is redo log described in carry out Log is recurred.
2. the method according to claim 1, wherein the determining current database example to target data page into Row is modified for the first time, comprising:
If it is determined that target data page brush, alternatively, a upper modification in the non-brush of the target data page and global buffer It is different from the current database example to record corresponding database instance, it is determined that the current database example is to the mesh Mark data page is modified for the first time.
3. being redo log the method according to claim 1, wherein generating page and synchronizing, comprising:
Page log serial number of the target data page before the modification for the first time is determined as synchronization log sequence number, wherein institute State synchronization log sequence number increments and unique;
A upper modification record pair based on the synchronization log sequence number and the target data page before the modification for the first time The database instance answered, generates the page and synchronizes and redo log.
4. the method according to claim 1, wherein further include:
If it is determined that the current database example to the target data page carry out it is non-modify for the first time when, generate conventional reforms day Will.
5. method is recurred in a kind of log, which is characterized in that be applied to standby database, comprising:
Obtain redoing log for primary database transmission;
According to the Log Types redo log, log recurrence is carried out to described redo log, it is described to redo log including page It synchronizes to redo log and be redo log with conventional.
6. according to the method described in claim 5, it is characterized in that, according to the Log Types redo log, to described heavy It does log and carries out log recurrence, comprising:
If it is described redo log to synchronize for the page redo log, the synchronization log sequence number redo log according to again The comparison result for completing log serial number is drilled, log recurrence is carried out to described redo log.
7. according to the method described in claim 6, it is characterized in that, according to the synchronization log sequence number redo log and again The comparison result for completing log serial number is drilled, log recurrence is carried out to described redo log, comprising:
If described recur completes log serial number less than the synchronization log sequence number redo log, waiting operation is executed, Until described recur completes log serial number more than or equal to the synchronization log sequence number redo log.
8. according to the method described in claim 5, it is characterized in that, according to the Log Types redo log, to described heavy It does log and carries out log recurrence, comprising:
If described redo log redoes log for the routine, the routine is redo log carry out log recurrence parallel, Described in the quantity that routinely redoes log be at least one.
9. device is recurred in a kind of log, which is characterized in that be configured at primary database, comprising:
Log generation module, for if it is determined that generation is reformed when current database example modifies target data page for the first time Log, wherein it is described redo log to synchronize including page redo log and conventional redo log;
Log sending module, for send it is described redo log to standby database, so as to be reformed described in the standby database root evidence The Log Types of log carry out log recurrence.
10. device is recurred in a kind of log, which is characterized in that be configured at standby database, comprising:
Log acquisition module, for obtaining redoing log for primary database transmission;
Module is recurred in log, and the Log Types for redoing log according to carry out log recurrence, institute to described redo log It states to redo log to synchronize to redo log including page and be redo log with conventional.
11. a kind of server, which is characterized in that the server includes:
One or more processors;
Storage device, for storing one or more programs;
When one or more of programs are executed by one or more of processors, so that one or more of processors are real Method is recurred in the now log as described in any in claim 1-4, or realizes the log as described in any in claim 5-8 Recurrence method.
12. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is by processor It realizes that method is recurred in the log as described in any in claim 1-4 when execution, or realizes such as institute any in claim 5-8 Method is recurred in the log stated.
CN201910748717.7A 2019-08-14 2019-08-14 Log replay method, device, server and storage medium Active CN110442560B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910748717.7A CN110442560B (en) 2019-08-14 2019-08-14 Log replay method, device, server and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910748717.7A CN110442560B (en) 2019-08-14 2019-08-14 Log replay method, device, server and storage medium

Publications (2)

Publication Number Publication Date
CN110442560A true CN110442560A (en) 2019-11-12
CN110442560B CN110442560B (en) 2022-03-08

Family

ID=68435397

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910748717.7A Active CN110442560B (en) 2019-08-14 2019-08-14 Log replay method, device, server and storage medium

Country Status (1)

Country Link
CN (1) CN110442560B (en)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111290881A (en) * 2020-01-21 2020-06-16 上海达梦数据库有限公司 Data recovery method, device, equipment and storage medium
CN111324665A (en) * 2020-01-23 2020-06-23 阿里巴巴集团控股有限公司 Log playback method and device
CN111639087A (en) * 2020-05-28 2020-09-08 北京金山云网络技术有限公司 Data updating method and device in database and electronic equipment
CN112416654A (en) * 2020-11-26 2021-02-26 上海达梦数据库有限公司 Database log replay method, device, equipment and storage medium
CN112596953A (en) * 2020-12-25 2021-04-02 上海达梦数据库有限公司 Database log replay method, device, equipment and storage medium
CN112637284A (en) * 2020-12-09 2021-04-09 北京金山云网络技术有限公司 Redo log storage method and device, electronic device and storage medium
CN113239120A (en) * 2021-06-07 2021-08-10 上海达梦数据库有限公司 Log synchronization method, device, equipment and storage medium
WO2022002103A1 (en) * 2020-06-30 2022-01-06 华为技术有限公司 Method for playing back log on data node, data node, and system
CN113987078A (en) * 2021-12-24 2022-01-28 中兴通讯股份有限公司 Data synchronization method, device and computer readable storage medium
CN114647624A (en) * 2022-05-11 2022-06-21 成都云祺科技有限公司 Method, system and storage medium for capturing database consistent point in block-level CDP
CN115114370A (en) * 2022-01-20 2022-09-27 腾讯科技(深圳)有限公司 Synchronization method and device for master database and slave database, electronic equipment and storage medium
CN115454717A (en) * 2022-09-16 2022-12-09 广州鼎甲计算机科技有限公司 Database real-time backup method and device, computer equipment and storage medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102629250A (en) * 2012-02-28 2012-08-08 杭州丰城信息技术有限公司 Recovery method of redo log files for main memory database
CN103412803A (en) * 2013-08-15 2013-11-27 华为技术有限公司 Data recovering method and device
CN103793479A (en) * 2014-01-14 2014-05-14 上海上讯信息技术股份有限公司 Log management method and log management system
CN106776130A (en) * 2016-11-30 2017-05-31 华为技术有限公司 A kind of journal recovery method, storage device and memory node
US20170177444A1 (en) * 2015-12-21 2017-06-22 Industrial Technology Research Institute Database batch update method, data redo/undo log producing method and memory storage apparatus
CN108664359A (en) * 2018-05-23 2018-10-16 上海达梦数据库有限公司 A kind of database restoring method, device, equipment and storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102629250A (en) * 2012-02-28 2012-08-08 杭州丰城信息技术有限公司 Recovery method of redo log files for main memory database
CN103412803A (en) * 2013-08-15 2013-11-27 华为技术有限公司 Data recovering method and device
CN103793479A (en) * 2014-01-14 2014-05-14 上海上讯信息技术股份有限公司 Log management method and log management system
US20170177444A1 (en) * 2015-12-21 2017-06-22 Industrial Technology Research Institute Database batch update method, data redo/undo log producing method and memory storage apparatus
CN106776130A (en) * 2016-11-30 2017-05-31 华为技术有限公司 A kind of journal recovery method, storage device and memory node
CN108664359A (en) * 2018-05-23 2018-10-16 上海达梦数据库有限公司 A kind of database restoring method, device, equipment and storage medium

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
何儒汉: "达梦数据库备份与恢复子系统的设计与实现", 《中国优秀博硕士学位论文全文数据库 (硕士) 信息科技辑》 *
成雅: "基于Oracle Streams的数据库实时备份与恢复技术研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111290881A (en) * 2020-01-21 2020-06-16 上海达梦数据库有限公司 Data recovery method, device, equipment and storage medium
CN111290881B (en) * 2020-01-21 2023-09-19 上海达梦数据库有限公司 Data recovery method, device, equipment and storage medium
CN111324665B (en) * 2020-01-23 2023-06-27 阿里巴巴集团控股有限公司 Log playback method and device
CN111324665A (en) * 2020-01-23 2020-06-23 阿里巴巴集团控股有限公司 Log playback method and device
CN111639087A (en) * 2020-05-28 2020-09-08 北京金山云网络技术有限公司 Data updating method and device in database and electronic equipment
CN111639087B (en) * 2020-05-28 2023-09-08 北京金山云网络技术有限公司 Data updating method and device in database and electronic equipment
WO2022002103A1 (en) * 2020-06-30 2022-01-06 华为技术有限公司 Method for playing back log on data node, data node, and system
CN112416654A (en) * 2020-11-26 2021-02-26 上海达梦数据库有限公司 Database log replay method, device, equipment and storage medium
CN112416654B (en) * 2020-11-26 2024-04-09 上海达梦数据库有限公司 Database log replay method, device, equipment and storage medium
CN112637284A (en) * 2020-12-09 2021-04-09 北京金山云网络技术有限公司 Redo log storage method and device, electronic device and storage medium
CN112596953B (en) * 2020-12-25 2023-08-29 上海达梦数据库有限公司 Database log replay method, device, equipment and storage medium
CN112596953A (en) * 2020-12-25 2021-04-02 上海达梦数据库有限公司 Database log replay method, device, equipment and storage medium
CN113239120B (en) * 2021-06-07 2023-08-18 上海达梦数据库有限公司 Log synchronization method, device, equipment and storage medium
CN113239120A (en) * 2021-06-07 2021-08-10 上海达梦数据库有限公司 Log synchronization method, device, equipment and storage medium
CN113987078B (en) * 2021-12-24 2022-04-19 中兴通讯股份有限公司 Data synchronization method, device and computer readable storage medium
CN113987078A (en) * 2021-12-24 2022-01-28 中兴通讯股份有限公司 Data synchronization method, device and computer readable storage medium
CN115114370A (en) * 2022-01-20 2022-09-27 腾讯科技(深圳)有限公司 Synchronization method and device for master database and slave database, electronic equipment and storage medium
CN115114370B (en) * 2022-01-20 2023-06-13 腾讯科技(深圳)有限公司 Master-slave database synchronization method and device, electronic equipment and storage medium
CN114647624A (en) * 2022-05-11 2022-06-21 成都云祺科技有限公司 Method, system and storage medium for capturing database consistent point in block-level CDP
CN115454717A (en) * 2022-09-16 2022-12-09 广州鼎甲计算机科技有限公司 Database real-time backup method and device, computer equipment and storage medium
CN115454717B (en) * 2022-09-16 2023-05-05 广州鼎甲计算机科技有限公司 Database real-time backup method and device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN110442560B (en) 2022-03-08

Similar Documents

Publication Publication Date Title
CN110442560A (en) Method, apparatus, server and storage medium are recurred in a kind of log
CN108537543B (en) Parallel processing method, device, equipment and storage medium for blockchain data
US8468313B2 (en) Asynchronous replication with write concurrency grouping
CN103793425B (en) Data processing method and device for distributed system
CN107113341B (en) System for high throughput processing of transactions in a distributed relational database management system for data partitioning
JP6375056B2 (en) Apparatus, method, storage medium, and computer program for coordinating parallel execution and cancellation of commands in storage cluster system
CN109960671B (en) Data transmission system, method and computer equipment
CN109951547A (en) Transactions requests method for parallel processing, device, equipment and medium
CN109710190A (en) A kind of date storage method, device, equipment and storage medium
CN110648178A (en) Method for increasing kafka consumption capacity
CN100442238C (en) Method and apparatus for asynchronous replication of data
CN105518641A (en) Method, device, and system for peer-to-peer data replication and method, device, and system for master node switching
CN102867035A (en) High-availability method and device of distributed document system cluster
RU2475818C2 (en) Data exchange methods and systems
CN113168652A (en) Block chain transaction processing system and method
CN102479118B (en) Incremental backup method for data blocks
CN109446202A (en) Identifier allocation method, device, server and storage medium
CN113051102B (en) File backup method, device, system, storage medium and computer equipment
CN113407637A (en) Data synchronization method and device, electronic equipment and storage medium
US20150370711A1 (en) Method and apparatus for cache memory data processing
CN111163149B (en) Intelligent contract platform method based on block chain
KR102326280B1 (en) Method, apparatus, device and medium for processing data
CN109284108A (en) Date storage method, device, electronic equipment and storage medium
CN109347899A (en) The method of daily record data is written in distributed memory system
CN109800110A (en) Data back up method, system and electronic equipment, storage medium and back-up device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant