CN111737341A - 基于日志解析的插入操作同步优化方法和数据同步系统 - Google Patents
基于日志解析的插入操作同步优化方法和数据同步系统 Download PDFInfo
- Publication number
- CN111737341A CN111737341A CN202010328622.2A CN202010328622A CN111737341A CN 111737341 A CN111737341 A CN 111737341A CN 202010328622 A CN202010328622 A CN 202010328622A CN 111737341 A CN111737341 A CN 111737341A
- Authority
- CN
- China
- Prior art keywords
- target
- value
- distribution
- transaction
- null
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003780 insertion Methods 0.000 title claims abstract description 187
- 230000037431 insertion Effects 0.000 title claims abstract description 187
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000005457 optimization Methods 0.000 title claims abstract description 38
- 238000004458 analytical method Methods 0.000 title claims abstract description 11
- 230000001360 synchronised effect Effects 0.000 claims abstract description 72
- 230000001172 regenerating effect Effects 0.000 claims abstract description 14
- 230000015654 memory Effects 0.000 claims description 14
- 239000003550 marker Substances 0.000 claims description 10
- 238000010586 diagram Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1734—Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Computer Security & Cryptography (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
Claims (10)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010328622.2A CN111737341B (zh) | 2020-04-23 | 2020-04-23 | 基于日志解析的插入操作同步优化方法和数据同步系统 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010328622.2A CN111737341B (zh) | 2020-04-23 | 2020-04-23 | 基于日志解析的插入操作同步优化方法和数据同步系统 |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111737341A true CN111737341A (zh) | 2020-10-02 |
CN111737341B CN111737341B (zh) | 2024-02-09 |
Family
ID=72646712
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010328622.2A Active CN111737341B (zh) | 2020-04-23 | 2020-04-23 | 基于日志解析的插入操作同步优化方法和数据同步系统 |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111737341B (zh) |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5590324A (en) * | 1995-02-07 | 1996-12-31 | International Business Machines Corporation | Optimization of SQL queries using universal quantifiers, set intersection, and max/min aggregation in the presence of nullable columns |
US20100030817A1 (en) * | 2008-07-31 | 2010-02-04 | Sybase, Inc | Statement categorization and normalization |
US20150149409A1 (en) * | 2013-11-22 | 2015-05-28 | Joo Yeon Lee | Dml replication with logical log shipping |
CN105786955A (zh) * | 2015-01-08 | 2016-07-20 | 国际商业机器公司 | 数据库管理系统中的数据复制 |
CN109933630A (zh) * | 2019-03-19 | 2019-06-25 | 武汉达梦数据库有限公司 | 数据库数据实时同步方法及设备 |
US20190317865A1 (en) * | 2018-04-16 | 2019-10-17 | Sap Se | Database Recovery and Optimization with Batch Processing |
CN110968593A (zh) * | 2019-12-10 | 2020-04-07 | 上海达梦数据库有限公司 | 数据库sql语句优化方法、装置、设备和存储介质 |
-
2020
- 2020-04-23 CN CN202010328622.2A patent/CN111737341B/zh active Active
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5590324A (en) * | 1995-02-07 | 1996-12-31 | International Business Machines Corporation | Optimization of SQL queries using universal quantifiers, set intersection, and max/min aggregation in the presence of nullable columns |
US20100030817A1 (en) * | 2008-07-31 | 2010-02-04 | Sybase, Inc | Statement categorization and normalization |
US20150149409A1 (en) * | 2013-11-22 | 2015-05-28 | Joo Yeon Lee | Dml replication with logical log shipping |
CN105786955A (zh) * | 2015-01-08 | 2016-07-20 | 国际商业机器公司 | 数据库管理系统中的数据复制 |
US20190317865A1 (en) * | 2018-04-16 | 2019-10-17 | Sap Se | Database Recovery and Optimization with Batch Processing |
CN109933630A (zh) * | 2019-03-19 | 2019-06-25 | 武汉达梦数据库有限公司 | 数据库数据实时同步方法及设备 |
CN110968593A (zh) * | 2019-12-10 | 2020-04-07 | 上海达梦数据库有限公司 | 数据库sql语句优化方法、装置、设备和存储介质 |
Non-Patent Citations (1)
Title |
---|
李仕伟;周坤;刘新蕊;李宝林;: "MySQL数据库优化技术", 信息与电脑(理论版), no. 12, pages 173 - 174 * |
Also Published As
Publication number | Publication date |
---|---|
CN111737341B (zh) | 2024-02-09 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN105335403B (zh) | 数据库访问方法及装置、数据库系统 | |
CN112286941B (zh) | 一种基于Binlog+HBase+Hive的大数据同步方法和装置 | |
CN110262929B (zh) | 一种保证复制事务一致性的方法以及相应的复制装置 | |
CN108121827B (zh) | 一种全量数据的同步方法及装置 | |
CN111767327B (zh) | 数据流间具有依赖关系的数据仓库构建方法与系统 | |
CN109086382B (zh) | 一种数据同步方法、装置、设备及存储介质 | |
KR20140053888A (ko) | 판식 파일중 구조화 정보 획득방법 및 장치 | |
CN111694798B (zh) | 一种基于日志解析的数据同步方法及数据同步系统 | |
CN109144950B (zh) | 业务数据的存储方法和装置 | |
CN111813803B (zh) | 语句块执行计划的生成方法、装置、设备和存储介质 | |
CN114547208A (zh) | 用于全链路追踪事务的方法及原生分布式数据库 | |
CN112559473A (zh) | 一种基于优先级的双路同步方法和同步系统 | |
CN114398873A (zh) | 一种敏感词的处理方法及处理装置 | |
CN112000649A (zh) | 一种基于map reduce的增量数据同步的方法和装置 | |
CN111858504B (zh) | 基于日志解析同步的操作合并执行方法和数据同步系统 | |
CN113177095A (zh) | 一种企业知识管理方法、系统、电子设备及存储介质 | |
CN111737341B (zh) | 基于日志解析的插入操作同步优化方法和数据同步系统 | |
CN111159020B (zh) | 一种应用于同步软件测试的方法和装置 | |
CN110196786B (zh) | 数据库回滚同步中内存的控制方法及设备 | |
CN106293862B (zh) | 一种可扩展标记语言xml数据的解析方法和装置 | |
CN109508244B (zh) | 数据处理方法及计算机可读介质 | |
CN105630997A (zh) | 一种数据并行处理方法、装置及设备 | |
CN105512237A (zh) | 一种复杂结构的数据导入系统 | |
CN112035464B (zh) | 一种基于日志解析的数据同步的过滤方法和同步装置 | |
CN111241193B (zh) | 一种基于日志解析的数据库添加列同步方法和装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
CB02 | Change of applicant information | ||
CB02 | Change of applicant information |
Address after: 430000 16-19 / F, building C3, future technology building, 999 Gaoxin Avenue, Donghu New Technology Development Zone, Wuhan, Hubei Province Applicant after: Wuhan dream database Co.,Ltd. Address before: 430000 16-19 / F, building C3, future technology building, 999 Gaoxin Avenue, Donghu New Technology Development Zone, Wuhan, Hubei Province Applicant before: WUHAN DAMENG DATABASE Co.,Ltd. |
|
CB03 | Change of inventor or designer information | ||
CB03 | Change of inventor or designer information |
Inventor after: Huang Haiming Inventor after: Sun Feng Inventor after: Yu Yuanlan Inventor before: Fu Quan Inventor before: Huang Haiming Inventor before: Sun Feng Inventor before: Yu Yuanlan |
|
GR01 | Patent grant | ||
GR01 | Patent grant |