CN109635038B - Remote double-reading and writing method for structured data - Google Patents
Remote double-reading and writing method for structured data Download PDFInfo
- Publication number
- CN109635038B CN109635038B CN201811383969.6A CN201811383969A CN109635038B CN 109635038 B CN109635038 B CN 109635038B CN 201811383969 A CN201811383969 A CN 201811383969A CN 109635038 B CN109635038 B CN 109635038B
- Authority
- CN
- China
- Prior art keywords
- data
- database table
- log
- user
- authority
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a remote double-reading and writing method of structured data, which comprises the following steps: step S10, dividing the users into local groups and allopatric groups; step S20, respectively creating database tables on local and off-site database servers, and respectively setting a main key by adopting an odd-even separation strategy; step S30, the user sends a read-write data request to the service system, the service system identifies the grouping of the user, and gives corresponding user authority according to the grouping; and step S40, the user performs read-write operation on the data according to the authority, and copies the data change log generated by the database table in the read-write operation process to the database server of the opposite terminal. The invention has the advantages that: the bidirectional copy of the remote double-live technology is realized.
Description
Technical Field
The invention relates to the technical field of information, in particular to a remote double-reading and writing method for structured data.
Background
With the rapid development of disaster recovery technology, data replication enters a rapid development stage, and especially for structured data (such as Oracle), technical products such as RAC, ADG, OGG and the like appear in sequence: RAC is used for realizing Oracle multi-instance cluster, ADG is used for realizing same-city double-activity disaster recovery in different places, and OGG is used for realizing different-place double-activity.
Along with the high requirements of core services of various industries on service continuity, the rapid development and application of the remote double-active technology are promoted, and remote double-active logic replication of the structured database can be realized based on the remote double-active technology.
However, the traditional remote double-activity technology has the following disadvantages: the traditional remote dual-active technology can only realize unidirectional copy, generally only a master library can read and write, a slave library only reads, and the copy direction needs to be manually adjusted after dual-active switching occurs, so that the risk of service switching is increased, meanwhile, the time of service switching is also increased, and disaster recovery indicators (RTO) is influenced.
Disclosure of Invention
The technical problem to be solved by the invention is to provide a remote double-reading and writing method of structured data, so as to realize bidirectional copying of remote double-live technology.
The invention is realized in the following way: a different-place double reading and writing method for structured data comprises the following steps:
step S10, dividing the users into local groups and allopatric groups;
step S20, respectively creating database tables on local and off-site database servers, and respectively setting a primary key by adopting an odd-even separation strategy;
step S30, the user sends read-write data request to the service system, the service system identifies the grouping of the user, and gives corresponding user authority according to the grouping;
and step S40, the user performs read-write operation on the data according to the authority, and copies the data change log generated by the database table in the read-write operation process to the database server of the opposite terminal.
Further, the step S10 is specifically:
the users are divided into local packets and allopatric packets by a scheduling control service.
Further, the step S20 is specifically:
creating a first database table on a local database server, wherein odd rows of the first database table are set as first primary keys; and creating a second database table on the database server at the different place, wherein even rows of the second database table are set as a second main key.
Further, setting the odd row of the first database table as a first primary key specifically includes: odd rows of the first database table are set as first main keys, and the step length between the first main keys is 2;
the setting of the even rows of the second database table as the second primary key specifically includes: even rows of the second database table are set as second primary keys, and the step length between the second primary keys is 2.
Further, the step S30 is specifically:
the user sends a request for reading and writing data in the database server to the service system, and the service system judges the grouping of the user through the scheduling control service and gives the user authority.
Further, the permission is a read-write permission or a read-only permission.
Further, the step S40 is specifically:
step S41, if the user belongs to the local group, the user carries out read-write operation to the data according to the authority, and the local database server generates a first main key in the odd-numbered row according to the first database table in the process of reading and writing the data, and adds or changes the data;
step S42, the data copying program scans the first database table at regular time, captures the log of data change and stores the log into the cache;
and step S43, the loader reads the log in the cache, converts the log into SQL statements, and loads the log of the data change into odd rows of the second database table through the SQL statements.
Further, the step S40 is specifically:
step S410, if the user belongs to the grouping in different places, the user carries out read-write operation on the data according to the authority, and a database server in different places generates a second main key in an even row according to a second database table in the process of reading and writing the data, so as to newly add or change the data;
step S420, the data copying program scans a second database table at regular time, captures a log of data change and stores the log into a cache;
and step S430, the loader reads the log in the cache, converts the log into an SQL statement, and loads the log with changed data into an even row of the first database table through the SQL statement.
Further, the data replication program is Logmnr.
The invention has the advantages that:
1. the bidirectional copy of the remote double-live technology is realized.
2. The odd rows of the first database table are set as the first main keys, the even rows of the second database table are set as the second main keys, namely the main keys of the local and the different places adopt an odd-even separation strategy, so that the split brain problem generated in the data copying process is avoided, namely the data copying program cannot decide which data is the standard because the same main key is changed locally and the different places simultaneously.
3. The strategy of odd-even separation is adopted by the local and allopatric main keys, so that simultaneous read-write operation of the local and allopatric database servers is realized, the service continuity index is greatly improved, and the upgrading and reconstruction cost is low.
4. The step length of each first main key is set to be 2, and the step length of each second main key is set to be 2, so that system resources are effectively saved.
5. And users are grouped by scheduling control service, so that the global load is reasonably and effectively distributed.
6. The copying direction can be switched at any time in different places by setting the even number of the first database table as the first main key and the odd number of the second database table as the second main key.
Drawings
The invention will be further described with reference to the following examples with reference to the accompanying drawings.
FIG. 1 is a flow chart of a method for remotely reading and writing structured data according to the present invention.
FIG. 2 is a schematic block diagram of a remote dual read-write method for structured data according to the present invention.
Detailed Description
Referring to fig. 1 to fig. 2, a preferred embodiment of a remote dual read/write method for structured data according to the present invention includes the following steps:
step S10, dividing the users into local groups and allopatric groups;
step S20, respectively creating database tables on local and off-site database servers, and respectively setting a main key by adopting an odd-even separation strategy;
step S30, the user sends read-write data request to the service system, the service system identifies the grouping of the user, and gives corresponding user authority according to the grouping; the security of the database server is improved by setting the user authority;
and step S40, the user performs read-write operation on the data according to the authority, and copies the data change log generated by the database table in the read-write operation process to the database server of the opposite terminal.
The step S10 specifically includes:
the users are divided into local packets and allopatric packets by a scheduling control service. Grouping can be performed according to regions or modulus rules and the like, users are grouped through scheduling control services, and overall load is reasonably and effectively distributed.
The step S20 specifically includes:
creating a first database table on a local database server, wherein odd rows of the first database table are set as first primary keys; and creating a second database table on a database server at a different place, wherein an even row of the second database table is set as a second main key. The method has the advantages that the method is characterized in that the odd-even separation strategy is adopted for the local and remote main keys, so that the split brain problem generated in the data copying process is avoided, namely the local and remote main keys are changed simultaneously, so that the data copying program cannot decide which data is used as the standard; and the simultaneous read-write operation of the local database server and the remote database server is realized, so that the service continuity index is greatly improved, and the upgrading and reconstruction cost is low.
A database primary key, refers to a column or combination of columns whose value uniquely identifies each row in a table, by which the physical integrity of the table is enforced, the primary key being primarily a foreign key association for other tables, as well as modification and deletion of the record.
The setting of the odd row of the first database table as a first primary key specifically includes: odd rows of the first database table are set as first main keys, and the step length between the first main keys is 2;
the setting of the even rows of the second database table as the second primary key specifically includes: even rows of the second database table are set as second primary keys, and the step length between the second primary keys is 2.
The step length of each first main key is set to be 2, and the step length of each second main key is set to be 2, so that system resources are effectively saved.
The step S30 specifically includes:
the user initiates a request for reading and writing data in the database server to the service system, and the service system judges the group where the user is located and gives the user authority through the scheduling control service.
The authority is a read-write authority or a read-only authority.
The step S40 specifically includes:
step S41, if the user belongs to the local group, the user reads and writes the data according to the authority, the local database server generates a first main key in the odd-numbered row according to the first database table in the process of reading and writing the data, and adds or changes the data;
step S42, the data copying program scans the first database table at regular time, captures the log of data change and stores the log into the cache;
and step S43, the loader reads the log in the cache, converts the log into SQL statements, and loads the log of the data change into odd rows of the second database table through the SQL statements.
The step S40 specifically includes:
step S410, if the user belongs to the grouping at different places, the user reads and writes the data according to the authority, and the database server at different places generates a second main key at an even row according to a second database table in the process of reading and writing the data, and adds or changes the data;
step S420, the data copying program scans a second database table at regular time, captures a log of data change and stores the log into a cache;
and step S430, the loader reads the log in the cache, converts the log into an SQL statement, and loads the log with changed data into an even row of the first database table through the SQL statement.
The data replication program is Logmnr. Logmnr is a self-contained log mining tool of oracle and can be used for problem diagnosis of DDL and DML operations of oracle data.
When the copy direction needs to be switched, the even row of the first database table is set as the first main key, and the odd row of the second database table is set as the second main key.
In summary, the invention has the advantages that:
1. the bidirectional copy of the remote double-live technology is realized.
2. The odd rows of the first database table are set as the first main keys, the even rows of the second database table are set as the second main keys, namely the main keys of the local and the foreign places adopt an odd-even separation strategy, so that the split brain problem generated in the data copying process is avoided, namely the data copying program cannot decide which data to be based on when the same main key is changed locally and the foreign places at the same time.
3. The strategy of odd-even separation is adopted by the local and allopatric main keys, so that simultaneous read-write operation of the local and allopatric database servers is realized, the service continuity index is greatly improved, and the upgrading and reconstruction cost is low.
4. The step length of each first main key is set to be 2, and the step length of each second main key is set to be 2, so that system resources are effectively saved.
5. And users are grouped by scheduling control service, so that the global load is reasonably and effectively distributed.
6. The copying direction can be switched at any time in different places by setting the even number of the first database table as the first main key and the odd number of the second database table as the second main key.
Although specific embodiments of the invention have been described above, it will be understood by those skilled in the art that the specific embodiments described are illustrative only and are not limiting upon the scope of the invention, and that equivalent modifications and variations can be made by those skilled in the art without departing from the spirit of the invention, which is to be limited only by the appended claims.
Claims (3)
1. A different-place double-reading and writing method for structured data is characterized in that: the method comprises the following steps:
step S10, dividing the users into local groups and allopatric groups through the scheduling control service;
step S20, a first database table is established on a local database server, odd rows of the first database table are set as first main keys, and the step length between the first main keys is 2; creating a second database table on a database server at a different place, wherein even rows of the second database table are set as second main keys, and the step length between the second main keys is 2;
step S30, the user sends a request for reading and writing data in the database server to the service system, and the service system judges the grouping of the user through the dispatching control service and gives the user authority;
step S40, the user carries out read-write operation to the data according to the authority, and copies the data change log generated by the database table in the read-write operation process to the database server of the opposite terminal; when the copying direction is switched, setting an even row of the first database table as a first main key, and setting an odd row of the second database table as a second main key;
the step S40 specifically includes:
step S41, if the user belongs to the local group, the user reads and writes the data according to the authority, the local database server generates a first main key in the odd-numbered row according to the first database table in the process of reading and writing the data, and adds or changes the data;
step S42, the data copying program scans the first database table at regular time, captures the log of data change and stores the log into the cache;
step S43, the loader reads the log in the cache and converts the log into SQL statements, and the loader loads the log with changed data into odd rows of the second database table through the SQL statements;
step S410, if the user belongs to the grouping in different places, the user carries out read-write operation on the data according to the authority, and a database server in different places generates a second main key in an even row according to a second database table in the process of reading and writing the data, so as to newly add or change the data;
step S420, the data copying program scans a second database table at regular time, captures a log of data change and stores the log into a cache;
and step S430, the loader reads the log in the cache, converts the log into an SQL statement, and loads the log with changed data into an even row of the first database table through the SQL statement.
2. The allopatric dual read-write method of a structured data as claimed in claim 1 wherein: the authority is a read-write authority or a read-only authority.
3. The allopatric dual read-write method of a structured data as claimed in claim 1 wherein: the data replication program is Logmnr.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811383969.6A CN109635038B (en) | 2018-11-20 | 2018-11-20 | Remote double-reading and writing method for structured data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811383969.6A CN109635038B (en) | 2018-11-20 | 2018-11-20 | Remote double-reading and writing method for structured data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109635038A CN109635038A (en) | 2019-04-16 |
| CN109635038B true CN109635038B (en) | 2022-08-19 |
Family
ID=66068453
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811383969.6A Active CN109635038B (en) | 2018-11-20 | 2018-11-20 | Remote double-reading and writing method for structured data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109635038B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110489092B (en) * | 2019-08-27 | 2022-08-16 | 四川长虹电器股份有限公司 | Method for solving read data delay problem under database read-write separation architecture |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7313657B1 (en) * | 2005-03-24 | 2007-12-25 | Sprint Communications Company L.P. | Conflict avoidance in data store replication |
| US8050406B2 (en) * | 2005-06-07 | 2011-11-01 | Sony Corporation | Key table and authorization table management |
| CN102025753B (en) * | 2009-09-17 | 2013-10-23 | 阿里巴巴集团控股有限公司 | Load balancing method and equipment for data resources of servers |
| CN103106200B (en) * | 2011-11-09 | 2017-08-01 | 上海盛大网络发展有限公司 | Non-relational database synchronization system and double write synchronous method |
| CN104243195A (en) * | 2013-06-19 | 2014-12-24 | 国家电网公司 | Remote disaster recovery processing method and device |
| CN103488721B (en) * | 2013-09-12 | 2017-01-18 | 京信通信系统（中国）有限公司 | Database bisynchronous method and system for master and slave boards |
| CN105205182B (en) * | 2015-10-28 | 2019-02-01 | 北京奇虎科技有限公司 | Multimachine room deployment system and business data processing method across computer room |
| CN106802891A (en) * | 2015-11-26 | 2017-06-06 | 中国电信股份有限公司 | The querying method of the non-burst field of distributed data base, system and equipment |
| CN106254100B (en) * | 2016-07-27 | 2019-04-16 | 腾讯科技（深圳）有限公司 | A kind of data disaster tolerance methods, devices and systems |
| CN108228755A (en) * | 2017-12-21 | 2018-06-29 | 江苏瑞中数据股份有限公司 | The data of MySQL database based on daily record analytic technique to Hadoop platform synchronize clone method |
| CN108595485A (en) * | 2018-03-14 | 2018-09-28 | 北京用友政务软件有限公司 | A kind of data-updating method and system at Double Data center |
-
2018
- 2018-11-20 CN CN201811383969.6A patent/CN109635038B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109635038A (en) | 2019-04-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9898549B1 (en) | Tenant-aware database for software as a service | |
| Schwartz et al. | High performance MySQL: optimization, backups, and replication | |
| CN104040481B (en) | Method and system for merging, storing and retrieving incremental backup data | |
| CA2758518C (en) | Asynchronous distributed object uploading for replicated content addressable storage clusters | |
| US9990366B2 (en) | Virtual partitions in virtual databases | |
| US7698319B2 (en) | Database system management method, database system, database device, and backup program | |
| CN105556519A (en) | Multi-version concurrency control on in-memory snapshot store of ORACLE in-memory database | |
| US20190034453A1 (en) | Flexible synchronous file system replication | |
| WO2012108015A1 (en) | Data synchronization method, data synchronization program, and data synchronization control device | |
| US20150006485A1 (en) | High Scalability Data Management Techniques for Representing, Editing, and Accessing Data | |
| US11914569B2 (en) | Light weight redundancy tool for performing transactions | |
| Kleppmann | Designing data-intensive applications | |
| CN113868028A (en) | Method for replaying log on data node, data node and system | |
| US20190340272A1 (en) | Systems and related methods for updating attributes of nodes and links in a hierarchical data structure | |
| CN109635038B (en) | Remote double-reading and writing method for structured data | |
| CN116303789A (en) | Parallel synchronization method and device for multi-fragment multi-copy database and readable medium | |
| CN115658391A (en) | Backup recovery method of WAL mechanism based on QianBase MPP database | |
| JP2007264814A (en) | Program for creating replication data, program for reflecting replication data, and database device | |
| CN114153857A (en) | Data synchronization method, data synchronization apparatus, and computer-readable storage medium | |
| US7600149B2 (en) | Failure transparency for update applications under single-master configuration | |
| CN114297196A (en) | Metadata storage method and device, electronic equipment and storage medium | |
| CN115114258A (en) | Data copying method and device, electronic equipment and computer storage medium | |
| CN117170942B (en) | Database backup method based on file system snapshot and related equipment | |
| CN111459949B (en) | Data processing method, device and equipment for database and index updating method | |
| US20240168664A1 (en) | Storage scheme transition via storage structure expansion in a storage system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |