CN109800272A - Data cached synchronous method, server, application system and storage device - Google Patents
Data cached synchronous method, server, application system and storage device Download PDFInfo
- Publication number
- CN109800272A CN109800272A CN201910138892.4A CN201910138892A CN109800272A CN 109800272 A CN109800272 A CN 109800272A CN 201910138892 A CN201910138892 A CN 201910138892A CN 109800272 A CN109800272 A CN 109800272A
- Authority
- CN
- China
- Prior art keywords
- data
- server
- database
- synchronized
- cache
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
The invention discloses a kind of data cached synchronous method, server, application system and storage devices, which comprises middleware server obtains the change log of system database, parses change log to obtain the data modification information of system database；Middleware server obtains data to be synchronized using data modification information from tables of data；Data to be synchronized are synchronized to cache database according to the mapping relations between tables of data and cache key by middleware server, wherein the mapping table being stored between tables of data and cache key in middleware server.By the above-mentioned means, the present invention can guarantee the data consistency of system database and cache database.
Description
Technical field
The present invention relates to Internet technical fields, more particularly to a kind of data cached synchronous method, server, using system
System and storage device.
Background technique
Now with the increase of the user volume of Internet application, especially possess the Internet application of mass users, for
The high concurrent of application system, High Availabitity propose increasingly higher demands.In order to promote the performance of application system, reduce to system number
According to the direct access in library, it will usually use cache database in application system, the data in system database can be put into
In cache database, data are directly obtained from cache database when needing to obtain data, to improve the reading speed of data.
When the data in system database have made caching in cache database, need to guarantee in cache database it is data cached with
The consistency of source data in system database.Existing caching is using passive more new strategy, when user data changes,
Corresponding caching is actively deleted, caching is just write data into when user inquires again.Under this mode, there is high concurrent in system
In the case where, database will face biggish pressure, easily cause performance bottleneck.
In addition, existing system database carries out data backup, principal and subordinate's progress each other between database using double main schemes
Synchronous duplication.When there is network jitter, there are certain time-delays between master-slave database.At this time if delay machine needs occurs in master library
Failure transfer, it is impossible to ensure principal and subordinate's data consistency, there are affairs risk of missing.The consistency of data is poor, and in failure
It needs the confirmation of manual sorting reconciliation result to lose affairs in recovery process, and carries out corresponding Supplementing Data.
Summary of the invention
The invention mainly solves the technical problem of providing a kind of data cached synchronous method, server, application system and
Storage device can guarantee the data consistency of system database and cache database.
In order to solve the above technical problems, one technical scheme adopted by the invention is that: a kind of data cached side of synchronization is provided
Method, which comprises middleware server obtains the change log of system database, parses change log to obtain system number
According to the data modification information in library；Middleware server obtains data to be synchronized using data modification information from tables of data；It is intermediate
Data to be synchronized are synchronized to cache database according to the mapping relations between tables of data and cache key by part server, wherein in
Between mapping table between tables of data and cache key is stored in part server.
In order to solve the above technical problems, another technical solution used in the present invention is: a kind of method of data synchronization is provided,
The described method includes: distributed application server obtains the nodal information of system database, and the nodal information is sent to
System server and middleware server, system server switches the connection of host node according to nodal information, and other are respectively saved
The data of point are synchronized in new host node, and middleware server positions host node according to nodal information, and obtains host node
Change log parses change log to obtain the data modification information of system database；Using data modification information from tables of data
It is middle to obtain data to be synchronized, and according to the mapping relations between tables of data and cache key be synchronized to data to be synchronized data cached
Library, wherein the mapping table being stored in middleware server between tables of data and cache key.
In order to solve the above technical problems, another technical solution used in the present invention is: providing a kind of server, the clothes
Business device includes processor and memory, and processor couples memory, and memory is stored with program, and processor is for executing the journey
Sequence realizes above-mentioned data cached synchronous method.
In order to solve the above technical problems, another technical solution used in the present invention is: a kind of application system is provided, it is described
System includes: system server, distributed application server, middleware server, system database, cache database, system
Server couples distributed application server, system database and cache database, and distributed application server couples middleware
Server and system database, middleware server coupling system database and cache database；System server is used for read/write
The data of system database and/or the data in reading cache data library；Distributed application server is for obtaining system database
Nodal information, and nodal information is sent to system server and middleware server；Middleware server is for obtaining system
System databases comparison log parses change log to obtain the data modification information of system database, utilizes data modification information
Data to be synchronized are obtained from tables of data, and are synchronized to data to be synchronized according to the mapping relations between tables of data and cache key
Cache database, wherein the mapping table being stored in middleware server between tables of data and cache key.
In order to solve the above technical problems, another technical solution used in the present invention is: providing a kind of with store function
Device, described device is stored with program, and described program, which is performed, can be realized above-mentioned method of data synchronization.
The beneficial effects of the present invention are: being in contrast to the prior art, the present invention is monitored by middleware server is
The change log of system database, parsing obtains change record, and is stored between tables of data and cache key in middleware server
Mapping table, server-side can by the change real-time update of data to cache, it can be ensured that system database and caching number
According to the consistency of the data in library.
Detailed description of the invention
Fig. 1 is the flow diagram of the data cached synchronous method first embodiment of the present invention；
Fig. 2 is the structural schematic diagram of application system first embodiment of the present invention；
Fig. 3 is the configuration diagram of application system second embodiment of the present invention；
Fig. 4 is the flow diagram of the application method of data synchronization first embodiment；
Fig. 5 is the structural schematic diagram of the application server first embodiment；
Fig. 6 is the structural schematic diagram for the device first embodiment that the application has store function.
Specific embodiment
It is right as follows in conjunction with drawings and embodiments to keep the purpose, technical solution and effect of the application clearer, clear
The application is further described.
The application provides a kind of data cached synchronous method, at least can be applied in high concurrent application system.Please refer to figure
1, Fig. 1 is the flow diagram of the data cached synchronous method first embodiment of the present invention.In this embodiment, data cached
Synchronous method includes the following steps:
S101: middleware server obtains the change log of system database, parses change log to obtain system data
The data modification information in library.
Wherein, system database is relevant database, such as MySQL relevant database, and system database is for depositing
Store up the significant data of application system.The change log of system database is binary log (Binlog log), Binlog log
It is the most important log of MySQL, it has recorded all DDL and DML (in addition to data query sentence) sentence, is remembered with event form
Record, the also time comprising consumption performed by sentence.Such as Binlog the log recording increase of system database, modification and deletion
Etc. data modification information.
S102: middleware server obtains data to be synchronized using data modification information from tables of data.
Wherein, when obtaining the data modification information of system database, while synchronous configuration information is obtained, it is synchronous to match confidence
Breath include in the address information of cache database, which database and database which tables of data needs synchronize caching and
The synchronization rules etc. of synchronization caching.The corresponding data name of data in change data is obtained, in the tables of data of synchronous configuration information
In search whether the record to match with the data name, if so, then the corresponding data of the data name are data to be synchronized.
S103: data to be synchronized are synchronized to slow by middleware server according to the mapping relations between tables of data and cache key
Deposit data library, wherein the mapping table being stored in middleware server between tables of data and cache key.
Wherein, a cache entry can be uniquely determined using the key of cache entry, utilizes reflecting between tables of data and cache key
Relationship is penetrated, the corresponding write-in cache database of data to be synchronized can be updated cache entry, data synchronization caching is improved
Reliability.
Wherein, cache database can be Redis database, and Redis is Remote DictionaryServer (long-range
Data service) abbreviation, be a memory cache database, belong to key-value type database.Redis database performance
It is powerful, there are more advantages.Completely in memory such as Redis database, processing speed is very fast.Redis also supports data to hold
Longization can store the data in memory onto disk, convenient quick-recovery fast under the emergency cases such as delay machine.Redis is supported
Data type abundant except the data of simple k/v type, while also providing list, set, zset, the data structures such as hash
Storage.It can guarantee data consistency, all Redis operations are atoms, it ensure that if two clients access simultaneously
Redis server will obtain updated value.
Wherein, in one embodiment, the change log of system database is obtained using middleware server.Middleware can
To be Cannal middleware, Canal can disguise oneself as oneself the slave node (Slave) of MySQL, and obtain from host node (Master)
Binlog is taken, is used after parsing and storage for downstream consumption end.Canal includes two component parts: server-side and client.Clothes
Business end is responsible for being connected to different MySQL examples, and is one event message queue of each Example maintenance；Client can then be ordered
The data altering event in these queues is read, handles and stores in data warehouse.
Existing Canal frame only supports tcp, kafka, RocketMQ Three models, wherein Kafka is LinkedIn
The distributed post of open source-subscription message system is the mode based on Pull to handle information consumption, pursues high-throughput；
RocketMQ is the message-oriented middleware of Ali's open source, it is pure Java exploitation, has high-throughput, high availability, is suitble to big rule
The characteristics of mould distributed system is applied.Canal frame is optimized in the application, can support Redis mode, is had
Body, the program of original connection Canal for being used to read Canal server-side message positioned at Canal client is integrated into Canal
In server-side, make it that can complete the transmission and reading of message in Canal server-side one end, reduces time delay, and then can support
Redis mode.In this embodiment, by the change log of middleware server monitoring system database, parsing, which obtains, to be become
The mapping table for more recording, and being stored in middleware server between tables of data and cache key, server-side can be by number
According to change real-time update to cache, it can be ensured that the consistency of the data of system database and cache database.
Wherein, in one embodiment, if there is multiple cache databases, middleware server can be according to tables of data
Data to be synchronized are synchronized to multiple cache databases in different computer rooms by the mapping relations between cache key.
Fig. 2 and Fig. 3 are please referred to, Fig. 2 is the structural schematic diagram of application system first embodiment of the present invention, and Fig. 3 is this hair
The configuration diagram of bright application system second embodiment.The application provides a kind of application system, such as can be virtual coin management
System.In this embodiment, application system 200 includes system server 10, distributed application server 20, middleware services
Device 30, system database 40, cache database 50, system server 10 couple distributed application server 20, system database
40 and cache database 50, distributed application server 20 couples middleware server 30 and system database 40, middleware clothes
Business 30 coupling system database 40 of device, cache database 50.Wherein, system server 10, distributed application server 20, centre
Part server 30 can be independent server, be also possible to the server cluster of multiple server compositions.System database 40,
Cache database 50 is formed by independent server or server set group configuration.System server 10, Distributed Application clothes
Business device 20, middleware server 30 are also possible to independent server or server cluster.
In the application system, system database is MySQL relevant database, and middleware is Canal middleware, caching
Database is Redis database.
Wherein, the system database of existing application system generally uses double main schemes, principal and subordinate's progress each other between database
Synchronous duplication.But when network jitter occur, there are certain time-delays between master-slave database.At this time if master library delay machine occurs and needs
Failure is wanted to shift, it is impossible to ensure principal and subordinate's data consistency, there are affairs risk of missing.People is also needed in failover procedure
Affairs are lost in carefully and neatly done reason reconciliation result confirmation, and carry out corresponding Supplementing Data.In order to solve this technical problem, the application exists
MGR mode is used in system database, guarantees the strong consistency of data write-in by distributed consensus algorithm inside MySQL.
MGR is a high-availability cluster scheme MySQLGroup Replication, referred to as: MGR (group duplication), is assisted based on Paxos
The state machine of view replicates, and being able to solve not can guarantee based on Data Consistency in the duplication synchronous with half of traditional asynchronous replication
The case where, the field for also allowing MySQL database to be related to is wider.Group duplication can be run under single holotype, can also be mostly main
It is run under mode.The system database of the application application system uses MGR mode, can guarantee the strong consistency of data write-in,
Artificial degree of participation is also reduced, system availability is improved.
Wherein, existing application system breaks down in MGR cluster or when delay machine, and cluster can re-elect host node and reach
To the function of restoring automatically, but the function of these fault recoveries, it can not be perceived for client, it is therefore desirable in visitor
Family end independently customizes fail-over feature.In order to solve this technical problem, the application passes through in system server and middleware
The client of integrated distributed application service on server monitors MGR using distributed application services (zookeeper, ZK)
Then MGR update is sent to system server and middleware server by the Status Change of cluster, so that it perceives MGR in time
The change of cluster.
Specifically, ZK server-side (mgr-monitor) obtains MGR clustered node information by executing sql, and by host node
For information persistence to ZK node, nodal information includes the status information and current primary node of each node of system database cluster
Information.The client (mgr-monitor-client) of distributed application services is integrated in the application, so that should journey
Sequence monitors node variation by ZK client can feel in real time when master library breaks down or host node switching occurs in delay machine
Know.I.e. system server can switch in time main section by the nodal information of the system database received after host node delay machine
The connection of point, and the data of other each nodes are synchronized in new host node, guarantee the consistency of data.Preferably
In, application program is monitored corresponding node by ZK using integrated mgr-motor-client and is changed, when the event of MGR cluster internal
, can be with real-time perception when node switching occur in barrier or delay machine, and handled by program and automatically switch database, it is fast to reach failure
The purpose of quick-recovery.
Client (the mgr-monitor- of distributed application services is also integrated in middleware server simultaneously
Client), so that middleware server monitors node variation, when switching host node, timely tracking and positioning host node obtains master
The change log of node.Parsing change log to obtain the data modification information of system database, using data modification information from
Data to be synchronized are obtained in tables of data, and according to the mapping relations between tables of data and cache key are synchronized to data to be synchronized slow
Deposit data library.Guarantee the consistency of the data of system database and cache database.Detailed process please refers to above-mentioned caching number
According to the description of synchronous method embodiment, details are not described herein.
Wherein, in existing application system, caching is deployed in single machine room, and program is deployed in two-shipper room, and which results in must
The routine call caching that so there is computer room is to belong to across computer room calling, it is easy for the inquiry business in the case of high concurrent because
Network Packet Loss between computer room causes success rate to decline, and influences user experience.In order to solve this technical problem, the application is adopted
Cache database, and the synchronous platform based on Canal frame are all deployed in two computer rooms, by monitoring MGR cluster master
The binlog of node is changed, and by the change immediate updating of data to the cache server of two computer rooms, system server is according to machine
Room configures preferential attachment with the cache database of computer room, reduces across computer room calling, decreases because surpassing caused by Network Packet Loss
When/failure.
Wherein, in one embodiment, it in order to further enhance system performance, is additionally provided in the application system of the application
Configurationization manages platform, is managed with the update of the configuration informations such as nodal information, cache information to system database.This is matched
Setting management platform can be a unit in system server, be also possible to independent server.
Based on this, the application also provides a kind of method of data synchronization, which can be applied to above-mentioned application system
System guarantees the data consistency of system database and cache database.Referring to Fig. 4, Fig. 4 is the application method of data synchronization
The flow diagram of one embodiment, in this embodiment, method of data synchronization includes the following steps:
Distributed application server obtains the nodal information of system database, and nodal information is sent to system server
And middleware server.
Wherein it is possible to be distributed application server active push nodal information to system server and middleware services
Device；It is also possible to after system server and middleware server request, retransmits nodal information.
System server switches the connection of host node according to nodal information, and the data of other each nodes is synchronized to new
In host node.
Middleware server positions host node according to nodal information, and obtains the change log of host node, parsing change day
Will is to obtain the data modification information of system database and the nodal information of system database；Using data modification information from data
Data to be synchronized are obtained in table, and data to be synchronized are synchronized to caching number according to the mapping relations between tables of data and cache key
According to library.Wherein, the mapping table being stored in the middleware server between tables of data and cache key.
Wherein, system database is relevant database, such as MySQL relevant database, and system database is for depositing
Store up the significant data of application system.The change log of system database is binary log (Binlog log), records system
The data modification information such as increase, modification and deletion of database.
Specific implementation process please refers to the description of above embodiment, and details are not described herein.
Based on this, the application also provides a kind of server, referring to Fig. 5, Fig. 5 is the application server first embodiment
Structural schematic diagram, in this embodiment, server 50 include processor 501 and memory 502, processor 501 coupling deposit
Reservoir 502, memory 502 are stored with program, and processor 501 realizes such as above-mentioned data cached synchronization for executing described program
Method.The server can be above-mentioned middleware server.The server can be independent server or server cluster.
Based on above-mentioned method of data synchronization, the application also provides a kind of device with store function, referring to Fig. 6, Fig. 6
It is the structural schematic diagram for the device first embodiment that the application has store function.In this embodiment, storage device 60
It is stored with program 601, program 601, which is performed, realizes above-mentioned method of data synchronization.Specific work process and the above method are implemented
It is consistent in example, therefore details are not described herein, please refers to the explanation of the above corresponding method step in detail.Wherein with the dress of store function
Set can be portable storage media for example USB flash disk, CD, mobile hard disk, read-only memory (ROM, Read-Only Memory), with
Machine accesses the various media that can store program code such as memory (RAM, Random Access Memory), magnetic disk, can also
To be terminal, server etc..
In several embodiments provided herein, it should be understood that disclosed system, device and method can
To realize by another way.For example, device embodiments described above are only schematical, for example, the mould
The division of block or unit, only a kind of logical function partition, there may be another division manner in actual implementation, such as multiple
Unit or assembly can be combined or can be integrated into another system, or some features can be ignored or not executed.It is another
Point, shown or discussed mutual coupling, direct-coupling or communication connection can be through some interfaces, device or
The indirect coupling or communication connection of unit can be electrical property, mechanical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.Some or all of unit therein can be selected to realize present embodiment scheme according to the actual needs
Purpose.
In addition, each functional unit in each embodiment of the application can integrate in one processing unit, it can also
To be that each unit physically exists alone, can also be integrated in one unit with two or more units.It is above-mentioned integrated
Unit both can take the form of hardware realization, can also realize in the form of software functional units.
If the integrated unit is realized in the form of SFU software functional unit and sells or use as independent product
When, it can store in a computer readable storage medium.Based on this understanding, the technical solution of the application is substantially
The all or part of the part that contributes to existing technology or the technical solution can be in the form of software products in other words
It embodies, which is stored in a storage medium, including some instructions are used so that a computer
It is each that equipment (can be personal computer, server or the network equipment etc.) or processor (processor) execute the application
The all or part of the steps of embodiment the method.
The foregoing is merely presently filed embodiments, are not intended to limit the scope of the patents of the application, all to utilize this
Equivalent structure or equivalent flow shift made by application specification and accompanying drawing content, it is relevant to be applied directly or indirectly in other
Technical field is included within the scope of the present invention.
Claims (13)
1. a kind of data cached synchronous method, which is characterized in that the described method includes:
Middleware server obtains the change log of system database, parses the change log to obtain the system database
Data modification information；
The middleware server obtains data to be synchronized using the data modification information from tables of data；
The data to be synchronized are synchronized to slow by the middleware server according to the mapping relations between tables of data and cache key
Deposit data library, wherein the mapping table being stored in the middleware server between the tables of data and cache key.
2. data cached synchronous method according to claim 1, which is characterized in that the middleware server is according to data
The data to be synchronized are synchronized to cache database and include: by the mapping relations between table and cache key
The data to be synchronized are synchronized to not by the middleware server according to the mapping relations between tables of data and cache key
With multiple cache databases in computer room.
3. a kind of server, which is characterized in that the server includes processor and memory, is deposited described in the processor coupling
Reservoir, the memory are stored with program, and the processor is realized as claimed in claim 1 or 2 for executing described program
Data cached synchronous method.
4. a kind of method of data synchronization, which is characterized in that the described method includes:
Distributed application server obtains the nodal information of system database, and the nodal information is sent to system server
And middleware server；
The system server switches the connection of host node according to the nodal information, and the data of other each nodes are synchronized to
In new host node；
The middleware server positions host node according to the nodal information, and obtains the change log of the host node, solves
The change log is analysed to obtain the data modification information of the system database；Using the data modification information from tables of data
It is middle to obtain data to be synchronized, and the data to be synchronized are synchronized to caching according to the mapping relations between tables of data and cache key
Database, wherein the mapping table being stored in the middleware server between tables of data and cache key.
5. method of data synchronization according to claim 4, which is characterized in that the method also includes: the distribution is answered
Persistence is carried out to the nodal information with server, the nodal information includes the shape of each node of system database cluster
The information of state information and current primary node.
6. a kind of application system, which is characterized in that the system comprises: system server, distributed application server, middleware
Server, system database, cache database, the system server couple the distributed application server, system data
Library and cache database, the distributed application server couples the middleware server and the system database, described
Middleware server couples the system database and cache database；
Data of the system server for system database described in read/write, and/or read the number of the cache database
According to；
The distributed application server is used to obtain the nodal information of the system database, and the nodal information is sent
To the system server and the middleware server；
For the middleware server for obtaining the system database change log, it is described to obtain to parse the change log
The data modification information of system database obtains data to be synchronized using the data modification information from tables of data, and according to
The data to be synchronized are synchronized to cache database by the mapping relations between tables of data and cache key, wherein the middleware
The mapping table being stored in server between tables of data and cache key.
7. application system according to claim 6, which is characterized in that the middleware server is for obtaining the system
Databases comparison log includes:
The middleware server is also used to receive the nodal information of the system database, and is positioned according to the nodal information
Host node obtains the change log of the host node.
8. application system according to claim 6, which is characterized in that the middleware server is according to tables of data and caching
The data to be synchronized are synchronized to cache database and include: by the mapping relations between key
The data to be synchronized are synchronized to not by the middleware server according to the mapping relations between tables of data and cache key
With multiple cache databases in computer room.
9. application system according to claim 8, which is characterized in that the system server preferentially calls same computer room
Data in the cache database.
10. application system according to claim 6, which is characterized in that the system server is also used to receive the system
The nodal information of system database, and according to the connection of nodal information switching host node, the data of other each nodes are synchronous
Into new host node.
11. application system according to claim 6, which is characterized in that the application system further includes management server, institute
Management server is stated for being managed to the nodal information of the system database, the configuration information of cache information.
12. application system according to claim 6, which is characterized in that the system database is relevant database, institute
State the synchronous consistency of data that system database realizes each node of system database cluster using the mode of group duplication；It is described
Middleware is Canal middleware, and the cache database is Redis database.
13. a kind of device with store function, which is characterized in that described device is stored with program, and described program is performed
It can be realized such as the described in any item method of data synchronization of claim 1-2,4-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910138892.4A CN109800272A (en) | 2019-02-25 | 2019-02-25 | Data cached synchronous method, server, application system and storage device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910138892.4A CN109800272A (en) | 2019-02-25 | 2019-02-25 | Data cached synchronous method, server, application system and storage device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109800272A true CN109800272A (en) | 2019-05-24 |
Family
ID=66562314
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910138892.4A Pending CN109800272A (en) | 2019-02-25 | 2019-02-25 | Data cached synchronous method, server, application system and storage device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109800272A (en) |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110287273A (en) * | 2019-07-03 | 2019-09-27 | 成都路行通信息技术有限公司 | The synchronous method and system of data is realized based on MySQL database binlog variation is monitored |
| CN110362628A (en) * | 2019-06-20 | 2019-10-22 | 视联动力信息技术股份有限公司 | A kind of synchronous method and device cascading file |
| CN110661869A (en) * | 2019-09-26 | 2020-01-07 | 苏州浪潮智能科技有限公司 | Node online method, system, device, equipment and computer storage medium |
| CN110781243A (en) * | 2019-11-06 | 2020-02-11 | 杭州安恒信息技术股份有限公司 | Incremental synchronization method and system for double main data of relational database |
| CN110851528A (en) * | 2019-10-09 | 2020-02-28 | 网联清算有限公司 | Database synchronization method and device, storage medium and computer equipment |
| CN111046106A (en) * | 2019-12-19 | 2020-04-21 | 杭州中恒电气股份有限公司 | Cache data synchronization method, device, equipment and medium |
| CN111078717A (en) * | 2019-12-20 | 2020-04-28 | 深圳前海环融联易信息科技服务有限公司 | Database and cache consistency synchronization method and device, computer equipment and storage medium |
| CN111159144A (en) * | 2019-11-27 | 2020-05-15 | 北京中交兴路信息科技有限公司 | Cache system and method |
| CN111274254A (en) * | 2020-01-16 | 2020-06-12 | 上海米哈游天命科技有限公司 | Data updating method, device, equipment and storage medium |
| CN111506668A (en) * | 2020-07-01 | 2020-08-07 | 西安安森智能仪器股份有限公司 | Robot cluster intelligent data synchronization method and system |
| CN111949731A (en) * | 2020-08-14 | 2020-11-17 | 厦门美图之家科技有限公司 | Data synchronization method and device, electronic equipment and storage medium |
| CN112035495A (en) * | 2020-08-20 | 2020-12-04 | 深圳乐信软件技术有限公司 | Data synchronization method, device, server and storage medium |
| CN112419018A (en) * | 2020-01-03 | 2021-02-26 | 上海哔哩哔哩科技有限公司 | General data reconciliation method, server and storage medium in distributed environment |
| CN112561449A (en) * | 2020-12-25 | 2021-03-26 | 云汉芯城（上海）互联网科技股份有限公司 | Method and system for synchronizing inventory information |
| CN112632189A (en) * | 2020-12-26 | 2021-04-09 | 中国农业银行股份有限公司 | Data synchronization system and method |
| CN112685431A (en) * | 2020-12-29 | 2021-04-20 | 京东数字科技控股股份有限公司 | Asynchronous caching method, device, system, electronic equipment and storage medium |
| CN112732751A (en) * | 2020-12-30 | 2021-04-30 | 北京懿医云科技有限公司 | Medical data processing method, device, storage medium and equipment |
| CN112749233A (en) * | 2021-01-15 | 2021-05-04 | 京东数字科技控股股份有限公司 | Data synchronization method and device, electronic equipment and storage medium |
| CN112905691A (en) * | 2019-11-19 | 2021-06-04 | 中盈优创资讯科技有限公司 | Information synchronization method and device |
| CN113656503A (en) * | 2021-08-20 | 2021-11-16 | 北京健康之家科技有限公司 | Data synchronization method, device and system and computer readable storage medium |
| WO2023082800A1 (en) * | 2021-11-11 | 2023-05-19 | 中兴通讯股份有限公司 | Main node selection method, distributed database and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8788458B2 (en) * | 2009-12-30 | 2014-07-22 | Sybase, Inc. | Data caching for mobile applications |
| CN105740383A (en) * | 2016-01-27 | 2016-07-06 | 中国科学院计算技术研究所 | Method and system for realizing data consistency of database and cache in big data platform |
| CN106959996A (en) * | 2017-01-20 | 2017-07-18 | 华数传媒网络有限公司 | A kind of back-end data synchronous method based on internet television |
| CN108920670A (en) * | 2018-07-06 | 2018-11-30 | 深圳市小牛在线互联网信息咨询有限公司 | Cache synchronization method, device, system and storage medium |
| CN109241072A (en) * | 2018-08-31 | 2019-01-18 | 携程计算机技术（上海）有限公司 | Buffering updating method and system based on Canal |
| CN109284334A (en) * | 2018-09-05 | 2019-01-29 | 拉扎斯网络科技（上海）有限公司 | Real-time data base synchronous method, device, electronic equipment and storage medium |
-
2019
- 2019-02-25 CN CN201910138892.4A patent/CN109800272A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8788458B2 (en) * | 2009-12-30 | 2014-07-22 | Sybase, Inc. | Data caching for mobile applications |
| CN105740383A (en) * | 2016-01-27 | 2016-07-06 | 中国科学院计算技术研究所 | Method and system for realizing data consistency of database and cache in big data platform |
| CN106959996A (en) * | 2017-01-20 | 2017-07-18 | 华数传媒网络有限公司 | A kind of back-end data synchronous method based on internet television |
| CN108920670A (en) * | 2018-07-06 | 2018-11-30 | 深圳市小牛在线互联网信息咨询有限公司 | Cache synchronization method, device, system and storage medium |
| CN109241072A (en) * | 2018-08-31 | 2019-01-18 | 携程计算机技术（上海）有限公司 | Buffering updating method and system based on Canal |
| CN109284334A (en) * | 2018-09-05 | 2019-01-29 | 拉扎斯网络科技（上海）有限公司 | Real-time data base synchronous method, device, electronic equipment and storage medium |
Non-Patent Citations (4)
| Title |
|---|
| GARFIELDER007: ""使用Binlog和Canal从MySQL抽取数据"", 《HTTPS://BLOG.CSDN.NET/GARFIELDER007/ARTICLE/DETAILS/ 79284264》 * |
| 乱月灵猫: ""使用Binlog和Canal从MySQL抽取数据"", 《HTTPS://WWW.CNBLOGS.COM/43726581GAVIN/P/9043958.HTML》 * |
| 周静等: "《基于Hadoop的大数据平台构建》", 30 November 2018 * |
| 薄荷脑: ""使用Binlog和Canal从MySQL抽取数据"", 《HTTPS://BLOG.CSDN.NET/ZJERRYJ/ARTICLE/DETAILS/77152226》 * |
Cited By (27)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110362628A (en) * | 2019-06-20 | 2019-10-22 | 视联动力信息技术股份有限公司 | A kind of synchronous method and device cascading file |
| CN110287273A (en) * | 2019-07-03 | 2019-09-27 | 成都路行通信息技术有限公司 | The synchronous method and system of data is realized based on MySQL database binlog variation is monitored |
| CN110661869A (en) * | 2019-09-26 | 2020-01-07 | 苏州浪潮智能科技有限公司 | Node online method, system, device, equipment and computer storage medium |
| CN110851528A (en) * | 2019-10-09 | 2020-02-28 | 网联清算有限公司 | Database synchronization method and device, storage medium and computer equipment |
| CN110851528B (en) * | 2019-10-09 | 2022-09-02 | 网联清算有限公司 | Database synchronization method and device, storage medium and computer equipment |
| CN110781243A (en) * | 2019-11-06 | 2020-02-11 | 杭州安恒信息技术股份有限公司 | Incremental synchronization method and system for double main data of relational database |
| CN112905691A (en) * | 2019-11-19 | 2021-06-04 | 中盈优创资讯科技有限公司 | Information synchronization method and device |
| CN111159144A (en) * | 2019-11-27 | 2020-05-15 | 北京中交兴路信息科技有限公司 | Cache system and method |
| CN111159144B (en) * | 2019-11-27 | 2023-09-08 | 北京中交兴路信息科技有限公司 | Caching system and method |
| CN111046106A (en) * | 2019-12-19 | 2020-04-21 | 杭州中恒电气股份有限公司 | Cache data synchronization method, device, equipment and medium |
| CN111078717A (en) * | 2019-12-20 | 2020-04-28 | 深圳前海环融联易信息科技服务有限公司 | Database and cache consistency synchronization method and device, computer equipment and storage medium |
| CN112419018A (en) * | 2020-01-03 | 2021-02-26 | 上海哔哩哔哩科技有限公司 | General data reconciliation method, server and storage medium in distributed environment |
| CN112419018B (en) * | 2020-01-03 | 2022-11-18 | 上海哔哩哔哩科技有限公司 | General data reconciliation method, server and storage medium in distributed environment |
| CN111274254A (en) * | 2020-01-16 | 2020-06-12 | 上海米哈游天命科技有限公司 | Data updating method, device, equipment and storage medium |
| CN111506668A (en) * | 2020-07-01 | 2020-08-07 | 西安安森智能仪器股份有限公司 | Robot cluster intelligent data synchronization method and system |
| CN111506668B (en) * | 2020-07-01 | 2024-02-02 | 西安安森智能仪器股份有限公司 | Robot cluster intelligent data synchronization method and system |
| CN111949731A (en) * | 2020-08-14 | 2020-11-17 | 厦门美图之家科技有限公司 | Data synchronization method and device, electronic equipment and storage medium |
| CN112035495B (en) * | 2020-08-20 | 2024-02-23 | 深圳乐信软件技术有限公司 | Data synchronization method, device, server and storage medium |
| CN112035495A (en) * | 2020-08-20 | 2020-12-04 | 深圳乐信软件技术有限公司 | Data synchronization method, device, server and storage medium |
| CN112561449A (en) * | 2020-12-25 | 2021-03-26 | 云汉芯城（上海）互联网科技股份有限公司 | Method and system for synchronizing inventory information |
| CN112561449B (en) * | 2020-12-25 | 2022-05-10 | 云汉芯城（上海）互联网科技股份有限公司 | Method and system for synchronizing inventory information |
| CN112632189A (en) * | 2020-12-26 | 2021-04-09 | 中国农业银行股份有限公司 | Data synchronization system and method |
| CN112685431A (en) * | 2020-12-29 | 2021-04-20 | 京东数字科技控股股份有限公司 | Asynchronous caching method, device, system, electronic equipment and storage medium |
| CN112732751A (en) * | 2020-12-30 | 2021-04-30 | 北京懿医云科技有限公司 | Medical data processing method, device, storage medium and equipment |
| CN112749233A (en) * | 2021-01-15 | 2021-05-04 | 京东数字科技控股股份有限公司 | Data synchronization method and device, electronic equipment and storage medium |
| CN113656503A (en) * | 2021-08-20 | 2021-11-16 | 北京健康之家科技有限公司 | Data synchronization method, device and system and computer readable storage medium |
| WO2023082800A1 (en) * | 2021-11-11 | 2023-05-19 | 中兴通讯股份有限公司 | Main node selection method, distributed database and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109800272A (en) | Data cached synchronous method, server, application system and storage device | |
| US11360854B2 (en) | Storage cluster configuration change method, storage cluster, and computer system | |
| EP0926608B1 (en) | Distributed persistent storage for intermittently connected clients | |
| CN111858097A (en) | Distributed database system and database access method | |
| WO2019154394A1 (en) | Distributed database cluster system, data synchronization method and storage medium | |
| JP6165729B2 (en) | Method and system for maintaining strong consistency of distributed replicated content in a client / server system | |
| US7546284B1 (en) | Virtual message persistence service | |
| US8954391B2 (en) | System and method for supporting transient partition consistency in a distributed data grid | |
| US20020131423A1 (en) | Method and apparatus for real-time parallel delivery of segments of a large payload file | |
| CN103345502B (en) | Transaction processing method and system of distributed type database | |
| US9881071B2 (en) | Transport layer abstraction for clustering implementation | |
| CN109639773B (en) | Dynamically constructed distributed data cluster control system and method thereof | |
| CN101706781A (en) | Method and system for centralized management of database caches | |
| Ramakrishnan | Cap and cloud data management | |
| CN110175089A (en) | A kind of dual-active disaster recovery and backup systems with read and write abruption function | |
| CN109344122A (en) | A kind of distributed meta-data management method and system for creating strategy in advance based on file | |
| CN104539583A (en) | Real-time database subscription system and method | |
| CN108566291A (en) | A kind of method of event handling, server and system | |
| CN103365987B (en) | Clustered database system and data processing method based on shared-disk framework | |
| CN110661841B (en) | Data consistency method for distributed service discovery cluster in micro-service architecture | |
| Ogunyadeka et al. | A multi-key transactions model for NoSQL cloud database systems | |
| CN109726211A (en) | A kind of distribution time series database | |
| CN105723365B (en) | Method for optimum indexing, main database node and subscriber database node | |
| CN107180034A (en) | The group system of MySQL database | |
| CN108418872A (en) | A kind of internet special train plateform system that the load balancing of easy extension multiple terminals is high |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190524 |