CN109359139A - Method of data synchronization, system, electronic equipment and computer readable storage medium - Google Patents
Method of data synchronization, system, electronic equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN109359139A CN109359139A CN201811244243.4A CN201811244243A CN109359139A CN 109359139 A CN109359139 A CN 109359139A CN 201811244243 A CN201811244243 A CN 201811244243A CN 109359139 A CN109359139 A CN 109359139A
- Authority
- CN
- China
- Prior art keywords
- data
- memory
- process flow
- module
- parsing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the present disclosure discloses a kind of method of data synchronization, system, electronic equipment and computer readable storage medium, which comprises the data in database are sent to the process flow of message storage；It consumes the data in the process flow of the message storage and is parsed, obtain parsing data；The parsing data are filtered according to default filtering rule, filter data is obtained, cross filter data storage into memory for described.The program can import data from relevant database toward non-relational database, and accomplish the real-time write-in of data, while can recall to data.
Description
Technical field
This disclosure relates to field of communication technology, and in particular to a kind of method of data synchronization, system, electronic equipment and computer
Readable storage medium storing program for executing.
Background technique
Currently, data are imported into non-relational database from relevant database, using in open source software Sqoop
Programming model MapReduce (mapping reduction) does the calculating of batch off-line data, and open source software Sqoop is very suitable to do off-line data
Operation task.
Due to, when carrying out Data Migration, there can be mass data, programming model MapReduce can execute the long period,
So that the real-time of data is poor.
Summary of the invention
The embodiment of the present disclosure provides a kind of method of data synchronization and system, electronic equipment and computer readable storage medium.
In a first aspect, providing a kind of method of data synchronization in the embodiment of the present disclosure.
Specifically, the method for data synchronization, comprising:
Data in database are sent to the process flow that distributed post subscribes to message；
It consumes the distributed post to subscribe to the data in the process flow of message and parsed, obtains parsing data；
And
The parsing data are filtered according to default filtering rule, filter data is obtained, the filter data of crossing is deposited
It stores up in memory.
With reference to first aspect, the disclosure is sent in the first implementation of first aspect by data in database
Before the process flow for subscribing to message to the distributed post, comprising: initial configuration information；Wherein, configuration information includes:
Company's configuration of database, distributed post subscribe to the configuration information of message, the parameter information of data processing, default filtering rule,
The structural information of table in memory.
With reference to first aspect with the first implementation of first aspect, the disclosure is in second of realization side of first aspect
In formula, the distributed post subscribes to the data that the data in the process flow of message are preset format.
With reference to first aspect, second of implementation of the first implementation of first aspect and first aspect, this public affairs
Be opened in the third implementation of first aspect, it is described by it is described cross filter data storage into memory after, further includes:
Store the function of the data in the process flow of consumption distribution formula distribution subscription message.
Second aspect provides a kind of data retrogressive method in the embodiment of the present disclosure.
Specifically, the data retrogressive method, comprising:
The parameter that backtracking tables of data is arranged obtains corresponding offset according to the parameter；
According to the corresponding offset, consumes the data that the distributed post is subscribed in the process flow of message and go forward side by side
Row parsing obtains parsing data；
The parsing data are filtered according to default filtering rule, filter data is obtained, the filter data of crossing is deposited
It stores up in memory.
The third aspect provides a kind of data synchronous system in the embodiment of the present disclosure.
Specifically, the data synchronous system, comprising:
Sending module is configured as the data in database being sent to the process flow that distributed post subscribes to message；
Parsing module, the data for being configured as consuming in the process flow of the distributed post subscription message are simultaneously solved
Analysis obtains parsing data；
Filtering module is configured as being filtered the parsing data according to default filtering rule, filter data is obtained；
Memory module is configured as crossing filter data storage into memory for described.
In conjunction with the third aspect, in the first implementation of the third aspect, the data synchronous system also wraps the disclosure
Include: initialization module is configured as initial configuration information；Wherein, configuration information includes: company's configuration of database, distribution
The configuration information of distribution subscription message, the parameter information of data processing, default filtering rule, in memory table structural information.
In conjunction with the first of the third aspect and the third aspect implementation, the disclosure is in second of realization side of the third aspect
In formula, the distributed post subscribes to the data that the data in the process flow of message are preset format.
In conjunction with the first implementation of the third aspect, the third aspect and second of implementation of the third aspect, this public affairs
It is opened in the third implementation of the third aspect, the memory module, is configured as storage consumption distribution formula distribution subscription and disappears
The function of data in the process flow of breath.
Fourth aspect provides a kind of data backtracking system in the embodiment of the present disclosure.
Specifically, the data backtracking system, comprising:
Setup module is configured as the parameter of setting backtracking tables of data；
Function obtains module, is configured as obtaining corresponding offset according to the parameter；
Parsing module, the data being configured as in the process flow of consumption distribution formula distribution subscription message are simultaneously parsed,
Obtain parsing data；
Filtering module is configured as being filtered the parsing data according to default filtering rule, filter data is obtained；
Memory module is configured as crossing filter data storage into memory for described.
5th aspect, the embodiment of the present disclosure provide a kind of electronic equipment, including memory and processor, the memory
The calculating of method of data synchronization in above-mentioned first and second aspect is executed for storing one or more support data synchronous system
Machine instruction, the processor is configured to for executing the computer instruction stored in the memory.The synchronous system of the data
System can also include communication interface, for data synchronous system and other equipment or communication.
6th aspect, the embodiment of the present disclosure provide a kind of computer readable storage medium, for storing data synchronous system
System computer instruction used, it includes be the synchronous system of data for executing method of data synchronization in above-mentioned first and second aspect
Computer instruction involved in uniting.
The technical solution that the embodiment of the present disclosure provides can include the following benefits:
Above-mentioned technical proposal, by sending the process flow that distributed post subscribes to message for data in database, and
It is parsed and is filtered, by the storage of filtered data to corresponding memory, realizes the real-time same of data in tables of data
Step processing.
It should be understood that above general description and following detailed description be only it is exemplary and explanatory, not
The disclosure can be limited.
Detailed description of the invention
In conjunction with attached drawing, by the detailed description of following non-limiting embodiment, the other feature of the disclosure, purpose and excellent
Point will be apparent.In the accompanying drawings:
Fig. 1 shows the flow chart of the method for data synchronization according to one embodiment of the disclosure；
Fig. 2 shows the flow charts according to the data retrogressive method of one embodiment of the disclosure；
Fig. 3 shows the structural block diagram of the data synchronous system according to one embodiment of the disclosure；
Fig. 4 shows the structural block diagram of the data backtracking system according to one embodiment of the disclosure；
Fig. 5 shows the structural block diagram of the electronic equipment according to one embodiment of the disclosure；
Fig. 6 is adapted for for realizing according to the data synchronization of one embodiment of the disclosure or the computer of data retrogressive method
The structural schematic diagram of system.
Specific embodiment
Hereinafter, the illustrative embodiments of the disclosure will be described in detail with reference to the attached drawings, so that those skilled in the art can
Easily realize them.In addition, for the sake of clarity, the portion unrelated with description illustrative embodiments is omitted in the accompanying drawings
Point.
In the disclosure, it should be appreciated that the term of " comprising " or " having " etc. is intended to refer to disclosed in this specification
Feature, number, step, behavior, the presence of component, part or combinations thereof, and be not intended to exclude other one or more features,
A possibility that number, step, behavior, component, part or combinations thereof exist or are added.
It also should be noted that in the absence of conflict, the feature in embodiment and embodiment in the disclosure
It can be combined with each other.The disclosure is described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.
The technical solution that the embodiment of the present disclosure provides passes through the binary log to Relational DBMS mySQL
It is sent to the process flow progress dissection process that distributed post subscribes to message Kafka after file BinLog parsing, passes through rule
The load filter method of engine, gets qualified data, and the data storage that will acquire is arrived corresponding with mySQL database
Table in.Wherein, regulation engine returns to filter result for being filtered to the data parsed according to customizing method.
Wherein, Kafka is a kind of distributed post message subscribing system of high-throughput, the middleware as message storage.
Fig. 1 shows the flow chart of the method for data synchronization according to one embodiment of the disclosure.As shown in Figure 1, the data
Synchronous method includes the following steps S110-S130:
In step s 110, data in database are sent to Kafka process flow；
In the step s 120, the data in Kafka process flow are consumed and are parsed, parsing data are obtained；
In step s 130, the parsing data are filtered according to default filtering rule, filter data are obtained, by institute
Filter data storage was stated into memory.
It is mentioned above, data in database processing mode in the prior art are as follows: insulted daily using Open-Source Tools Sqoop
Morning timing executes task, the batch data in database is imported from relevant database toward non-relational database, subsequent structure
Data warehouse is built, user can be complete using interactive structured query language (SQL, Structured Query Language)
Inquiry at data etc. operation, since data volume to be treated is larger, execute the time it is longer, cause the real-time of data compared with
Difference.
In view of drawbacks described above, in this embodiment, a kind of method of data synchronization is proposed, this method is by by database
Middle data are sent to Kafka process flow, and it is parsed and is filtered, by filtered data storage to corresponding storage
Device realizes the real-time synchronization processing of data in tables of data.
Wherein, data are sent to before Kafka process flow in database, need to solve binary log file BinLog
Analysis.
Wherein, the data in Kafka process flow are consumed and are parsed.The centre that Kafka system is stored as message
Part cannot disposably receive the data sent in all databases, therefore can only consume the data in Kafka and receive down again later
Batch of data.Wherein, parse data format can for Map<K, V>, facilitate the corresponding memory of deposit.It is stored in Kafka processing stream
The data of data in journey are all by DRC (Data Replication Center, data duplication centre) or Canal data
The data for the preset format (for example, JSON format) that library Incremental Log resolution system parses.Wherein, JSON
(JavaScript Object Notation, JavaScript object numbered musical notation).
Wherein, default filtering rule can be customized by function, and regulation engine is by loading customized function logarithm
According to being filtered, filter data is obtained.Wherein, crossing filter data is also Map<K, V>format.
It wherein, include: that will cross filter data storage to Kudu table or HBase into memory by the storage of filter data excessively
Table.
In an optional implementation of the present embodiment, by data in database be sent to Kafka process flow it
Before, comprising: initial configuration information；Wherein, configuration information includes: company's configuration of database, the configuration information of Kafka, data
The parameter information of processing, default filtering rule, in memory table structural information.Wherein, configuration information stores in the database.
Wherein, the parameter information of data processing is converted to the class for needing to use in data processing, by broadcasting class, makes
The metadata information of broadcast can be got in the task of execution by obtaining Excutor system.
In an optional implementation of the present embodiment, the filter data of crossing is stored into memory it described
Afterwards, further includes: the function of the data in storage consumption Kafka process flow.Wherein, the function is offset.Function
The function of offset is to be cited as reference system with specified, obtains new reference by given offset.
Fig. 2 shows the flow charts according to the data retrogressive method of one embodiment of the disclosure.As shown in Fig. 2, the data
Retrogressive method includes the following steps S210-S230:
In step S210, the parameter of setting backtracking tables of data, according to the parameter, obtain corresponding offset (for example,
Pass through function offset)；
In step S220, according to the corresponding offset, consumes the data in Kafka process flow and solved
Analysis obtains parsing data；
In step S230, the parsing data are filtered according to default filtering rule, filter data are obtained, by institute
Filter data storage was stated into memory.
In the present embodiment, the data for needing to inquire verification are recalled by the offset of preservation, guarantee the standard of data
True property and consistency.
Wherein, the parameter for recalling tables of data includes the title for the tables of data that need to be recalled, the period of backtracking, storage backtracking number
According to table type.The parameter of backtracking tables of data passes through external input.
Wherein, according to the parameter, corresponding offset is obtained, comprising: find according to the title for the tables of data that need to be recalled
Corresponding tables of data finds corresponding offset further according to the period of backtracking.Specifically, the parameter is parsed, according to the time
Section parses, and judgement is to recall according to day backtracking also according to hour, gets the configuration information of table to be recalled.
Wherein, the parameter information of data processing is converted to the class for needing to use in data processing, by class into
Row broadcast, so that Excutor system can get the metadata information of broadcast in the task of execution.
Wherein, according to the corresponding offset, the data in Kafka process flow is consumed and are parsed, are solved
Data are analysed, data in Kafka are particularly read by the API of Spark-Kafka, wherein the data are offset meanings
The data in fixed section.
Wherein, the data in Kafka process flow are consumed and are parsed.The centre that Kafka system is stored as message
Part cannot disposably receive the data sent in all databases, therefore can only consume the data in Kafka and receive down again later
Batch of data.Wherein, parse data format can for Map<K, V>, facilitate the corresponding memory of deposit.It is stored in Kafka processing stream
The data of data in journey are all by DRC (Data Replication Center, data duplication centre) or Canal data
The data for the JSON format that library Incremental Log resolution system parses.
Wherein, default filtering rule can be customized by function, and regulation engine is by loading customized function logarithm
According to being filtered, filter data is obtained.Wherein, crossing filter data is also Map<K, V>format.
It wherein, include: that will cross filter data storage to Kudu table or HBase into memory by the storage of filter data excessively
Table.
Following is disclosure system embodiment, can be used for executing embodiments of the present disclosure.
Fig. 3 shows the structural block diagram of the data synchronous system according to one embodiment of the disclosure, which can be by soft
Part, hardware or both are implemented in combination with as some or all of of electronic equipment.As shown in figure 3, the synchronous system of the data
System includes:
Sending module 302 is configured as data in database are sent to Kafka process flow；
Parsing module 303, the data being configured as in consumption Kafka process flow are simultaneously parsed, and parsing data are obtained；
Filtering module 304 is configured as being filtered the parsing data according to default filtering rule, obtains filtering number
According to；
Memory module 305 is configured as crossing filter data storage into memory for described.
It is mentioned above, data in database processing mode in the prior art are as follows: insulted daily using Open-Source Tools Sqoop
Morning timing executes task, the batch data in database is imported from relevant database toward non-relational database, subsequent structure
Data warehouse is built, user can be complete using interactive structured query language (SQL, Structured Query Language)
Inquiry at data etc. operation, since data volume to be treated is larger, execute the time it is longer, cause the real-time of data compared with
Difference.
In view of drawbacks described above, in this embodiment, a kind of data synchronous system is proposed, which will count in database
According to being sent to Kafka process flow, and it is parsed and is filtered, by filtered data storage to corresponding memory,
Realize the real-time synchronization processing of data in tables of data.
Wherein, data are sent to before Kafka process flow in database, need to solve binary log file BinLog
Analysis.
Wherein, the middleware that Kafka system is stored as message cannot disposably receive the number sent in all databases
According to, therefore the data in Kafka can only be consumed and receive next batch data again later.Wherein, parse data format can for Map <
K, V >, facilitate the corresponding memory of deposit.The data for being stored in the data in Kafka process flow are all by DRC (Data
Replication Center, data duplication centre) or the JSON that parses of Canal database Incremental Log resolution system
The data of format.
Wherein, default filtering rule can be customized by function, and regulation engine is by loading customized function logarithm
According to being filtered, filter data is obtained.Wherein, crossing filter data is also Map<K, V>format.
Wherein, memory module 305 are configured as to cross filter data storage to Kudu table or HBase table.
In an optional implementation of the present embodiment, the data synchronous system further include: initialization module 301,
It is configured as initial configuration information；Wherein, configuration information includes: company's configuration of database, the configuration information of Kafka, data
The parameter information of processing, default filtering rule, in memory table structural information.Wherein, configuration information stores in the database.
Wherein, the parameter information of data processing is converted to the class for needing to use in data processing, by broadcasting class, makes
The metadata information of broadcast can be got in the task of execution by obtaining Excutor system.
In an optional implementation of the present embodiment, the memory module 305 is configured as storage consumption Kafka
The function of data in process flow.Wherein, the function is offset.The function of function offset is to be cited as with specified
Reference system obtains new reference by given offset.
Fig. 4 shows the structural block diagram of the data backtracking system according to one embodiment of the disclosure, which can be by soft
Part, hardware or both are implemented in combination with as some or all of of electronic equipment.As shown in figure 4, data backtracking system
System includes:
Setup module 401 is configured as the parameter of setting backtracking tables of data；
Function obtains module 402, is configured as obtaining corresponding offset according to the parameter；
Parsing module 403, the data being configured as in consumption Kafka process flow are simultaneously parsed, and parsing data are obtained；
Filtering module 404 is configured as being filtered the parsing data according to default filtering rule, obtains filtering number
According to；
Memory module 405 is configured as crossing filter data storage into memory for described.
Specifically, the specific of data backtracking system limits the restriction that may refer to above for data retrogressive method,
This is repeated no more.
The disclosure also discloses a kind of electronic equipment, and Fig. 5 shows the knot of the electronic equipment according to one embodiment of the disclosure
Structure block diagram, as shown in figure 5, the electronic equipment 500 includes memory 501 and processor 502；Wherein,
The memory 501 is for storing one or more computer instruction, wherein one or more computer refers to
It enables and being executed by the processor 502 to realize any of the above-described method and step.
Fig. 6 is suitable for being used to realize the structural representation of the computer system of the method for data synchronization according to disclosure embodiment
Figure.
As shown in fig. 6, computer system 600 includes central processing unit (CPU) 601, it can be read-only according to being stored in
Program in memory (ROM) 602 or be loaded into the program in random access storage device (RAM) 603 from storage section 608 and
Execute the various processing in above embodiment.In RAM603, also it is stored with system 600 and operates required various program sum numbers
According to.CPU601, ROM602 and RAM603 are connected with each other by bus 604.Input/output (I/O) interface 605 is also connected to always
Line 604.
I/O interface 605 is connected to lower component: the importation 606 including keyboard, mouse etc.；It is penetrated including such as cathode
The output par, c 607 of spool (CRT), liquid crystal display (LCD) etc. and loudspeaker etc.；Storage section 608 including hard disk etc.；
And the communications portion 609 of the network interface card including LAN card, modem etc..Communications portion 609 via such as because
The network of spy's net executes communication process.Driver 610 is also connected to I/O interface 605 as needed.Detachable media 611, such as
Disk, CD, magneto-optic disk, semiconductor memory etc. are mounted on as needed on driver 610, in order to read from thereon
Computer program be mounted into storage section 608 as needed.
Particularly, according to embodiment of the present disclosure, method as described above may be implemented as computer software programs.
For example, embodiment of the present disclosure includes a kind of computer program product comprising be tangibly embodied in and its readable medium on
Computer program, the computer program includes program code for executing the method for data synchronization.In such reality
It applies in mode, which can be downloaded and installed from network by communications portion 609, and/or is situated between from detachable
Matter 611 is mounted.
Flow chart and block diagram in attached drawing illustrate system, method and computer according to the various embodiments of the disclosure
The architecture, function and operation in the cards of program product.In this regard, each box in course diagram or block diagram can be with
A part of a module, section or code is represented, a part of the module, section or code includes one or more
Executable instruction for implementing the specified logical function.It should also be noted that in some implementations as replacements, institute in box
The function of mark can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are practical
On can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it wants
It is noted that the combination of each box in block diagram and or flow chart and the box in block diagram and or flow chart, Ke Yiyong
The dedicated hardware based system of defined functions or operations is executed to realize, or can be referred to specialized hardware and computer
The combination of order is realized.
Being described in unit or module involved in disclosure embodiment can be realized by way of software, can also
It is realized in a manner of through hardware.Described unit or module also can be set in the processor, these units or module
Title do not constitute the restriction to the unit or module itself under certain conditions.
As on the other hand, the disclosure additionally provides a kind of computer readable storage medium, the computer-readable storage medium
Matter can be computer readable storage medium included in system described in above embodiment；It is also possible to individualism,
Without the computer readable storage medium in supplying equipment.Computer-readable recording medium storage has one or more than one journey
Sequence, described program is used to execute by one or more than one processor is described in disclosed method.
Above description is only the preferred embodiment of the disclosure and the explanation to institute's application technology principle.Those skilled in the art
Member is it should be appreciated that invention scope involved in the disclosure, however it is not limited to technology made of the specific combination of above-mentioned technical characteristic
Scheme, while should also cover in the case where not departing from the inventive concept, it is carried out by above-mentioned technical characteristic or its equivalent feature
Any combination and the other technical solutions formed.Such as features described above has similar function with (but being not limited to) disclosed in the disclosure
Can technical characteristic replaced mutually and the technical solution that is formed.
Claims (10)
1. a kind of method of data synchronization characterized by comprising
Data in database are sent to the process flow that distributed post subscribes to message；
It consumes the distributed post to subscribe to the data in Message Processing process and parsed, obtains parsing data；And
The parsing data are filtered according to default filtering rule, filter data is obtained, the filter data storage excessively is arrived
In memory.
2. the method according to claim 1, wherein the data in database are sent to the distributed hair
Cloth is subscribed to before the process flow of message, comprising:
Initial configuration information；
Wherein, configuration information includes: company's configuration of database, the configuration information of distributed post subscription message, data processing
Parameter information, default filtering rule, in memory table structural information.
3. the method according to claim 1, wherein the filter data of crossing is stored into memory it described
Afterwards, further includes:
The function for the data that the distributed post is subscribed in the process flow of message is consumed in storage.
4. a kind of data retrogressive method characterized by comprising
The parameter of backtracking tables of data is set, and corresponding offset is obtained according to the parameter；
According to the corresponding offset, data in the process flow of consumption distribution formula distribution subscription message are simultaneously parsed,
Obtain parsing data；And
The parsing data are filtered according to default filtering rule, filter data is obtained, the filter data storage excessively is arrived
In memory.
5. a kind of data synchronous system characterized by comprising
Sending module is configured as data in database being sent to the process flow that distributed post subscribes to message；
Parsing module, the data for being configured as consuming in the process flow of the distributed post subscription message are simultaneously parsed,
Obtain parsing data；
Filtering module is configured as being filtered the parsing data according to default filtering rule, filter data is obtained；
Memory module is configured as crossing filter data storage into memory for described.
6. system according to claim 5, which is characterized in that further include:
Initialization module is configured as initial configuration information；
Wherein, configuration information includes: company's configuration of database, the configuration information of distributed post subscription message, data processing
Parameter information, default filtering rule, in memory table structural information.
7. system according to claim 5, which is characterized in that the memory module is configured as described point of storage consumption
The function of data in the process flow of cloth distribution subscription message.
8. a kind of data backtracking system characterized by comprising
Setup module is configured as the parameter of setting backtracking tables of data；
Function obtains module, is configured as obtaining corresponding offset according to the parameter；
Parsing module, the data being configured as in the process flow of consumption distribution formula distribution subscription message are simultaneously parsed, are obtained
Parse data；
Filtering module is configured as being filtered the parsing data according to default filtering rule, filter data is obtained；
Memory module is configured as crossing filter data storage into memory for described.
9. a kind of electronic equipment, which is characterized in that including memory and processor；Wherein,
The memory is for storing one or more computer instruction, wherein one or more computer instruction is by institute
Processor is stated to execute to realize the described in any item method and steps of claim 1-4.
10. a kind of computer readable storage medium, is stored thereon with computer instruction, which is characterized in that the computer instruction quilt
Claim 1-4 described in any item method and steps are realized when processor executes.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811244243.4A CN109359139A (en) | 2018-10-24 | 2018-10-24 | Method of data synchronization, system, electronic equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811244243.4A CN109359139A (en) | 2018-10-24 | 2018-10-24 | Method of data synchronization, system, electronic equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109359139A true CN109359139A (en) | 2019-02-19 |
Family
ID=65346611
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811244243.4A Pending CN109359139A (en) | 2018-10-24 | 2018-10-24 | Method of data synchronization, system, electronic equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109359139A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110515944A (en) * | 2019-08-16 | 2019-11-29 | 出门问问(苏州)信息科技有限公司 | Date storage method, storage medium and electronic equipment based on distributed data base |
| CN110968592A (en) * | 2019-12-06 | 2020-04-07 | 深圳前海环融联易信息科技服务有限公司 | Metadata acquisition method and device, computer equipment and computer-readable storage medium |
| CN111666344A (en) * | 2020-06-19 | 2020-09-15 | 中信银行股份有限公司 | Heterogeneous data synchronization method and device |
| CN111797158A (en) * | 2019-04-08 | 2020-10-20 | 北京沃东天骏信息技术有限公司 | Data synchronization system, method and computer-readable storage medium |
| CN112052295A (en) * | 2020-08-06 | 2020-12-08 | 中信银行股份有限公司 | Data synchronization method and device, electronic equipment and readable storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106713470A (en) * | 2016-12-29 | 2017-05-24 | 北京奇艺世纪科技有限公司 | Distributed cache updating method and cache updating system |
| CN107122497A (en) * | 2017-05-25 | 2017-09-01 | 北京微影时代科技有限公司 | Data processing method, device, electronic equipment and computer-readable recording medium |
| CN107609008A (en) * | 2017-07-26 | 2018-01-19 | 郑州云海信息技术有限公司 | A kind of data importing device and method from relevant database to Kafka based on Apache Sqoop |
| CN107885881A (en) * | 2017-11-29 | 2018-04-06 | 顺丰科技有限公司 | Business datum real-time report, acquisition methods, device, equipment and its storage medium |
| CN108365985A (en) * | 2018-02-07 | 2018-08-03 | 深圳壹账通智能科技有限公司 | A kind of cluster management method, device, terminal device and storage medium |
-
2018
- 2018-10-24 CN CN201811244243.4A patent/CN109359139A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106713470A (en) * | 2016-12-29 | 2017-05-24 | 北京奇艺世纪科技有限公司 | Distributed cache updating method and cache updating system |
| CN107122497A (en) * | 2017-05-25 | 2017-09-01 | 北京微影时代科技有限公司 | Data processing method, device, electronic equipment and computer-readable recording medium |
| CN107609008A (en) * | 2017-07-26 | 2018-01-19 | 郑州云海信息技术有限公司 | A kind of data importing device and method from relevant database to Kafka based on Apache Sqoop |
| CN107885881A (en) * | 2017-11-29 | 2018-04-06 | 顺丰科技有限公司 | Business datum real-time report, acquisition methods, device, equipment and its storage medium |
| CN108365985A (en) * | 2018-02-07 | 2018-08-03 | 深圳壹账通智能科技有限公司 | A kind of cluster management method, device, terminal device and storage medium |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111797158A (en) * | 2019-04-08 | 2020-10-20 | 北京沃东天骏信息技术有限公司 | Data synchronization system, method and computer-readable storage medium |
| CN111797158B (en) * | 2019-04-08 | 2024-04-05 | 北京沃东天骏信息技术有限公司 | Data synchronization system, method and computer readable storage medium |
| CN110515944A (en) * | 2019-08-16 | 2019-11-29 | 出门问问(苏州)信息科技有限公司 | Date storage method, storage medium and electronic equipment based on distributed data base |
| CN110968592A (en) * | 2019-12-06 | 2020-04-07 | 深圳前海环融联易信息科技服务有限公司 | Metadata acquisition method and device, computer equipment and computer-readable storage medium |
| CN110968592B (en) * | 2019-12-06 | 2023-11-21 | 深圳前海环融联易信息科技服务有限公司 | Metadata acquisition method, metadata acquisition device, computer equipment and computer readable storage medium |
| CN111666344A (en) * | 2020-06-19 | 2020-09-15 | 中信银行股份有限公司 | Heterogeneous data synchronization method and device |
| CN112052295A (en) * | 2020-08-06 | 2020-12-08 | 中信银行股份有限公司 | Data synchronization method and device, electronic equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109359139A (en) | Method of data synchronization, system, electronic equipment and computer readable storage medium | |
| US9262716B2 (en) | Content response prediction | |
| US9495436B2 (en) | Apparatus and method for ingesting and augmenting data | |
| CN106067080B (en) | Configurable workflow capabilities are provided | |
| US9767126B2 (en) | Storing and processing data organized as flexible records | |
| CN110019267A (en) | A kind of metadata updates method, apparatus, system, electronic equipment and storage medium | |
| CN107273122A (en) | Based on decoupling mechanism can iteration set up operation system method and its terminal | |
| US9396448B2 (en) | Distributed and open schema interactions management system and method | |
| CN108171528A (en) | A kind of attribution method and attribution system | |
| CN109101309A (en) | For updating user interface method and device | |
| CN108200070A (en) | A kind of method and device for generating list | |
| CN110888974A (en) | Data visualization method and device | |
| CN106202438A (en) | The method and system of storage associated data | |
| CN106156060B (en) | Tag control system and terminal, label application method and label method for sorting | |
| CN105335466A (en) | Audio data retrieval method and apparatus | |
| CN108345658A (en) | Algorithm calculates decomposing process, server and the storage medium of track | |
| CN114860727A (en) | Zipper watch updating method and device | |
| CN105955961B (en) | Reservation information treating method and apparatus | |
| CN106716400B (en) | A kind of partition management method and device of tables of data | |
| Usuelli | R machine learning essentials | |
| US9411555B2 (en) | Systems and methods for creating context sensitive graph topologies based on multidimensional context information | |
| US11238077B2 (en) | Auto derivation of summary data using machine learning | |
| CN107615789A (en) | The notice orientation participated in based on downstream user | |
| CN110147381A (en) | Information processing method, system and recording medium | |
| WO2015028895A1 (en) | A system and method for managing partner feed index |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190219 |