CN113206757B - Method and electronic equipment for configuring full data and incremental data by stream type synchronous network management - Google Patents
Method and electronic equipment for configuring full data and incremental data by stream type synchronous network management Download PDFInfo
- Publication number
- CN113206757B CN113206757B CN202110447453.9A CN202110447453A CN113206757B CN 113206757 B CN113206757 B CN 113206757B CN 202110447453 A CN202110447453 A CN 202110447453A CN 113206757 B CN113206757 B CN 113206757B
- Authority
- CN
- China
- Prior art keywords
- data
- full
- incremental
- time
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/085—Retrieval of network configuration; Tracking network configuration history
- H04L41/0853—Retrieval of network configuration; Tracking network configuration history by actively collecting configuration information or by backing up configuration information
- H04L41/0856—Retrieval of network configuration; Tracking network configuration history by actively collecting configuration information or by backing up configuration information by backing up or archiving configuration information
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0893—Assignment of logical groups to network elements
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method for configuring full data and incremental data by a stream type synchronous network management system, which comprises the following steps: s1: the incremental data processing task starts to receive and process the incremental message; s2: monitoring whether full data needs to be synchronously processed or not in the incremental message processing process, if so, turning to the step S4, and if not, turning to the step S3; s3: sequentially warehousing the incremental messages; s4: if it is monitored that the full data needs to be processed in the incremental message processing process, immediately pausing and blocking the incremental data processing task; s5: starting a full data synchronous processing task; s6: after the full data synchronous processing is finished, the full data is put into a warehouse; s7: and after the full data is put into the database, the incremental data processing task is unblocked, and the process returns to the step S1. The invention can realize the historical zipper record of the network configuration data, provide the network configuration at any time for inquiry, improve the historical performance and the significance of alarm data, and provide the historical performance and the significance for other systems or application display or analysis. The invention also provides corresponding electronic equipment.
Description
Technical Field
The invention belongs to the technical field of network data management, and particularly relates to a method for configuring full data and incremental data by a stream-type synchronous network manager and electronic equipment.
Background
As networks grow larger, the data generated by the networks also grow toward a vast amount. However, the capabilities of the network management system such as data storage and data provision for the devices and the management and control system have highlighted the shortcuts. With the emergence of applications such as network artificial intelligence, network digital insight analysis, network digital simulation and the like, higher and more rigorous technical requirements are provided for aspects such as the volume, the data quality, the data storage, the data integrity and the like of network big data, which are difficult to solve by the existing network management system, particularly the aspects of data quality, data integrity and consistency. Under the background, a method and a system capable of expanding the data volume of the expanded network are needed to solve a series of problems that the network management data is not comprehensive enough at present. Based on the basis, breakthrough in aspects such as network management intelligent operation and maintenance is realized.
The network management configuration data (service configuration, Topo configuration and the like) only has current snapshot data and no change history. If some configurations of the network manager are deleted at a certain time, the historical performance or the service related to the historical alarm before the time does not exist or is not matched, so that the value of the historical performance data or the historical alarm data is low.
In order to store and record the historical configuration change record, a configuration data historical pull list is needed, so the network management configuration change record needs to be obtained. However, the network manager initializes the configuration in a configuration file import mode at some time, so that the whole amount (stock) data configured by the network manager needs to be synchronized to the zipper table. The data and processing logic of both change log and full configuration are relatively independent, so the data processing sequence cannot be guaranteed. In addition, because the network configuration may be relatively large, the time for generating the full configuration is relatively long, which is a continuous generation process and needs asynchronous processing in the background. During the asynchronous generation of the full configuration, the configuration of the network may change, so that various complex scenes of the full configuration data and the incremental change record data need to be processed synchronously.
Disclosure of Invention
Aiming at the defects or the improvement requirements of the prior art, the invention provides a scheme for configuring the full data and the incremental data by the streaming synchronous network management, and the consistency of the historical record of the network management configuration change and the real-time operation change of the network management is ensured.
To achieve the above object, according to an aspect of the present invention, there is provided a method for configuring full data and incremental data by a streaming synchronization network manager, including:
s1: the incremental data processing task starts to receive and process the incremental message;
s2: monitoring whether full data needs to be synchronously processed or not in the incremental message processing process, if so, turning to the step S4, and if not, turning to the step S3;
s3: sequentially warehousing the incremental messages;
s4: if it is monitored that the full data needs to be processed in the incremental message processing process, immediately pausing and blocking the incremental data processing task;
s5: starting a full data synchronous processing task;
s6: after the full data synchronous processing is finished, the full data is put into a warehouse;
s7: and after the full data is put into the database, the incremental data processing task is unblocked, and the process returns to the step S1.
In an embodiment of the present invention, the webmaster logs for storing the full data and the incremental data follow a uniform format constraint, and each log needs to have the real event occurrence time of the log and also needs to have the data operation type of the log.
In one embodiment of the invention, the data operation types include add, update, and delete.
In an embodiment of the present invention, the webmaster log generation adopts a multithreading mode, and the log of the full data and the log of the incremental data are processed by independent threads respectively.
In one embodiment of the invention, a start marker is added at the beginning of the full data sync, an end marker M2 is added at the end of the full data sync, and a full start marker M3 is also added to the delta data.
In one embodiment of the invention, a mark is punched into a log of incremental data and time is recorded when full-scale synchronization starts, a start mark is punched into the log of full-scale data and time is recorded, an end mark is also punched into the end of the log of full-scale data and time is recorded when output of the log of full-scale synchronization ends, and the start mark and the end mark need to have the same time stamp in order to ensure the integrity of the full-scale data each time.
In an embodiment of the present invention, the full data synchronization processing task includes:
s11: reading the full data from the full queue;
s12: judging whether the read data is overtime, if not, turning to the step S13; otherwise go to step S14;
s13: judging whether the read data is a full end mark, if not, returning to the step S11 to continue reading the data; otherwise go to step S15;
s14: if the time is out when the whole data is read for processing, the read data is indicated to be dirty data, and the task is directly discarded and ended;
s15: if the reading is the end mark, loading all the data read this time to a disk, and then putting the full data into a warehouse in batch;
s16: and recording the last full synchronization time, and finishing the full data synchronization processing task.
In an embodiment of the present invention, the incremental data processing task includes:
s21: starting an incremental data processing task and reading a latest incremental data;
s22: judging whether the piece of data is a full-scale starting mark or not; if yes, go to step S23, if not continue to execute step S24;
s23: if the mark is the full-volume start mark, the incremental data processing task is blocked, and the processing flow of the full-volume processing task is started;
s24: if the total quantity task is not the total quantity starting mark, acquiring the last total quantity synchronization time, and recording and storing the time after the total quantity task is processed;
s25: comparing the current data time with the last full synchronization time; if the current data time is greater than the last full synchronization time, go to step S26; otherwise, the step S27 of incremental processing and full synchronous repeating data task is carried out;
s26: putting the data into a database;
s27: and repeating data task processing by increment processing and full synchronization.
In one embodiment of the present invention, the incremental processing and full synchronization of duplicate data tasks includes:
s31: judging whether the operation type of the data is newly increased or not; if the new addition is made, jumping to step S32; otherwise, the next condition judgment is carried out to step S35;
s32: judging whether the database has the data; if yes, go to step S33; otherwise, jumping to step S34;
s33: invalid data is directly discarded;
s34: directly warehousing incremental data;
s35: judging whether the operation type of the data is updating, if so, jumping to the step S36; otherwise, the next condition judgment is carried out to step S37;
s36: judging whether the current record modification time is less than the latest record time of the database; if the current record modification time is less than the latest record time of the database, jumping to step S33; otherwise, jumping to step S34;
s37: judging whether the operation type of the data is deletion, if so, directly executing the step S34 to delete the data; the negative jumps to step S33 to discard the data.
According to another aspect of the present invention, there is also provided an electronic apparatus including:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the method for configuring the full data and the incremental data by the streaming synchronous network management system.
Generally, compared with the prior art, the technical scheme of the invention has the following beneficial effects:
(1) by the method, the historical zipper record of the network configuration data can be realized, and the network configuration at any moment can be inquired, so that the actual significance of the data such as historical performance and alarm is improved, and the data is provided for other systems or application display or analysis;
(2) the invention provides a scheme for the consistency of log data synchronization data content of the full configuration data and the incremental data processed by the network manager, and the result generated when the network manager processes the incremental data and the full data simultaneously can be real and effective through the scheme;
(3) the invention provides a backtracking scheme of network management historical snapshot data, which can enable a network manager to backtrack network configuration snapshots at any historical moment;
(4) the invention can improve the integrity of network management data, provide network configuration topological structure and service configuration at any time, and based on the characteristics, can provide more and richer service capability, expand other services and the like.
Drawings
FIG. 1 is a business data model to be solved in an embodiment of the present invention;
FIG. 2 is a model diagram of full and incremental synchronization results in an embodiment of the present invention;
FIG. 3 is a diagram of another model of full and incremental synchronization results in accordance with an embodiment of the present invention;
fig. 4 is a schematic flow chart of a method for configuring full data and incremental data by a streaming synchronization network manager in an embodiment of the present invention;
FIG. 5 is a flowchart illustrating a process for synchronizing full data according to an embodiment of the present invention;
FIG. 6 is a flowchart of an incremental data processing task according to an embodiment of the present invention;
FIG. 7 is a flowchart illustrating incremental processing and full synchronization of duplicate data tasks in an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
The invention mainly explains the synchronization of the full baseline data and the incremental change data based on the network management configuration, and ensures the consistency of the historical record of the network management configuration change and the real-time operation change of the network management based on a data consistency algorithm.
Fig. 1 shows a business data model to be solved by the present invention, and in fig. 1, it is assumed that full data synchronization occurs at 10:06 minutes and continues until 10:08 minutes ends, and 4 pieces of configuration data (as indicated by the filled circles in the vertical lines in the above figure) need to be synchronized. Incremental data continues to be generated starting at 10:00 and is still generated and consumed during the time period of full data synchronization. Under the service model, the integrity and final data consistency of full data and incremental data need to be guaranteed, and the processing logic of the full data and the incremental data in the time of 10: 06-10: 08 is mainly needed to be solved.
The first situation is that the problem that the data after the incremental data is processed to 10:06 minutes (for example, the increment is processed to 10:09 minutes) is needed to be solved, the full data of 10: 06-10: 08 minutes is processed, the post-processed full data is caused to flush the incremental data after 10:06 minutes, and data inconsistency is caused.
The second situation is the situation that data delay of increment due to network or other reasons is needed to be solved, for example, the increment is processed until 10:02 time-sharing, and then the full synchronous data request is received. After the full data is processed in advance, the problems of faults and inconsistency of the data occur.
The third situation is that the problem of final consistency of data results is solved when incremental data is processing partial data in a full synchronization time interval, for example, when an incremental processing reaches 10: 07-point data, a full synchronization request is received, the incremental data has already processed 10: 06-point data, but the 10: 07-point data is not processed yet.
As shown in fig. 2, the effect model after the synchronization process is shown, and after the synchronization process in fig. 2, at 10: 06-10: the 08-point data may result in partial data in the full data being synchronized into the result set, for example, 3 of 4 full data are synchronized (indicated by filled circles in the figure). The incremental data for this time period is also partially synchronized into the result set, such as one of two pieces of incremental data is synchronized (indicated by the filled circles in the figure). The subsequent incremental data are still unaffected (the horizontal and vertical cross bars in the figure fill the circles).
In addition to the possible outcome models described above, there are other outcome models, such as where the full amount of data is invalid data, where the incremental data is contained in the full amount of data, and so forth.
As shown in fig. 3, a start marker M1 is added at the beginning of the full data sync, an end marker M2 (circles are hatched in the figure) is added at the end of the full data sync, and a full start marker M3 (circles are hatched in the figure) is also added to the incremental data.
By adding the flag, the incremental data processing task can be prevented from being excessive when processing the incremental data. When processing the full amount of data, it is possible to know whether or not the data processing is finished. Therefore, the consistency of the processing results of the incremental data and the full data is ensured.
Detailed description of functional implementation:
network management configuration log generation
Firstly, the network management log needs to follow the uniform format constraint, the log needs to have the real event occurrence time of the record every day, and also needs to have the data operation type of the log, and the current main data operation types are respectively added, updated and deleted. Other content of the log data is, among other things, other information of the log.
Secondly, the network management logs adopt a multi-thread mode, and the logs of the full data and the logs of the incremental data are processed by independent threads respectively. However, in order to ensure the sequential consumption processing of the full data and the incremental data, it is necessary to record a time by punching a mark into the log of the incremental data when the full synchronization starts, and to also punch a start mark into the log of the full data and record the time by punching the mark. When the output of the full-volume synchronous log is finished, an end mark is also punched into the log end of the full-volume data, and the time is recorded. And the start marker and the end marker need to have the same time stamp (version marker) in order to guarantee the integrity of the full amount of data at a time.
Full volume tag blocks incremental processing and opens full volume processing tasks
When the incremental data is processed, a streaming processing mode is adopted, and once the full-volume start mark in the data stream content is found, the incremental data is paused to be processed, and the full-volume processing task is started. And the full data processing task starts to acquire data from the full queue and process the data until all the full data are processed, and informs the incremental data processing task to continue processing the incremental data. And the incremental data processing task resumes the processing flow after receiving the notice of the completion of the continuous full processing.
In the process of pulling the full amount of data, the data pulling may be overtime due to problems such as abnormality, and in such a case, incomplete data needs to be discarded. In addition, when the pulling of the full data is finished, whether the pulled data belongs to the same batch of data is judged, whether the timestamps of the full start marker and the full end marker are matched can be judged, and if the data does not belong to the same batch, the batch of data also needs to be discarded, because dirty data may be mixed in the data.
Full-sync end resume delta processing
And after receiving the full-volume end notification, releasing the previous blockage of the incremental data processing task and continuously consuming the incremental data. When the incremental data is consumed again, whether the incremental data has a full-volume start mark or not is monitored (observed) at the moment, and once the full-volume start mark is found, the full-volume processing flow is repeated.
Incremental processing and full-sync time-coincident sections
In resuming the process of the incremental data processing task, since the incremental task is blocked for a period of time, new incremental data may be generated but not consumed during the blocked period of time, while the full amount of data at this period of time may already include the unconsumed incremental data. Thus, during the period of time that the incremental task has just recovered, there may be portions of data overlap between incremental consumption and full consumption. The data is processed to ensure that no data exception is caused even if the data has overlapped parts.
The logic for processing the duplicate data is as follows:
1. firstly, caching the last full synchronization ending timestamp in a memory;
2. acquiring a piece of incremental flow data, comparing the time stamp of the piece of data with the time stamp of the end of the last full-scale synchronization, and if the time stamp of the current data is greater than the time stamp of the end of the last full-scale synchronization, directly processing the piece of incremental data. If the current data timestamp is less than the last full-synchronization finishing timestamp, the current data operation type needs to be continuously judged;
3. and judging the operation type of the current incremental data again when the current data timestamp is smaller than the last full-synchronization ending timestamp, which indicates that the partial data may be overlapped. In the case of a delete operation, it can be performed directly, since the delete operation is idempotent and the result is the same even if it is performed multiple times at the database level. If the operation is a new addition operation, whether the data record exists in the database or not needs to be judged, if not, the new addition can be directly executed, and if the same record exists, the execution of the data is abandoned. If the current data is the updating operation, the operation time of the current data needs to be compared with the last updating time of the existing record of the database, if the updating time of the current data is greater than the latest time of the existing record, the updating operation of the current data needs to be executed to the database layer, otherwise, the data is directly discarded;
4. continue to acquire the next piece of delta stream data and repeat step 2.
Fig. 4 is a general flowchart of data processing logic in the embodiment of the present invention, and the detailed steps include:
s1: the incremental data processing task starts to receive and process the incremental message;
s2: monitoring whether full data needs to be synchronously processed or not in the incremental message processing process, if so, turning to the step S4, and if not, turning to the step S3;
s3: sequentially warehousing the incremental messages;
s4: if it is monitored that the full data needs to be processed in the incremental message processing process, immediately pausing and blocking the incremental data processing task;
s5: starting a full data synchronous processing task;
s6: after the full data synchronous processing is finished, the full data is put into a warehouse;
s7: and after the full data is put into the database, the incremental data processing task is unblocked, and the process returns to the step S1.
Task flow for synchronous processing of full data
Fig. 5 is a flowchart of a task of full data synchronization processing according to an embodiment of the present invention, which includes the following detailed steps:
s11: reading the full data from the full queue;
s12: judging whether the read data is overtime, if not, turning to the step S13; otherwise go to step S14;
s13: judging whether the read data is a full end mark, if not, returning to the step S11 to continue reading the data; otherwise go to step S15;
s14: if the time is out when the whole data is read for processing, the read data is indicated to be dirty data, and the task is directly discarded and ended;
s15: if the reading is the end mark, loading all the data read this time to a disk, and then putting the full data into a warehouse in batch;
s16: and recording the last full synchronization time, and finishing the full data synchronization processing task.
Incremental data processing task flow
Fig. 6 is a flowchart illustrating an incremental data processing task according to an embodiment of the present invention, where the detailed steps include:
s21: starting an incremental data processing task and reading a latest incremental data;
when the previous full data synchronous processing task is performed, the incremental data processing task is blocked, and the previous old data can be backlogged in a message queue; therefore, after the incremental data processing task is started, the data is received from the blocked position, the old data is received first, and then the new data is received, and the data reading process in the incremental data processing task is first-in first-out.
S22: judging whether the piece of data is a full-scale starting mark or not; if yes, go to step S23, if not continue to execute step S24;
s23: if the mark is the full-volume start mark, the incremental data processing task is blocked, and the processing flow of the full-volume processing task is started;
s24: if the total quantity task is not the total quantity starting mark, acquiring the last total quantity synchronization time, and recording and storing the time after the total quantity task is processed;
s25: comparing the current data time with the last full synchronization time; if the current data time is greater than the last full synchronization time, go to step S26; otherwise, entering an increment processing and full synchronization repeating data task S27;
s26: putting the data into a database;
s27: and repeating data task processing by increment processing and full synchronization.
Incremental processing and full synchronization repeating data task flow
Fig. 7 is a task diagram illustrating incremental processing and full synchronization of duplicate data according to an embodiment of the present invention, which includes the following detailed steps:
s31: judging whether the operation type of the data is newly increased or not; if the new addition is made, jumping to step S32; otherwise, the next condition judgment is carried out to step S35;
s32: judging whether the database has the data; if yes, go to step S33; otherwise, jumping to step S34;
s33: invalid data is directly discarded;
s34: directly warehousing incremental data;
s35: judging whether the operation type of the data is updating, if so, jumping to the step S36; otherwise, the next condition judgment is carried out to step S37;
s36: judging whether the current record modification time is less than the latest record time of the database; if the current record modification time is less than the latest record time of the database, jumping to step S33; otherwise, jumping to step S34;
s37: judging whether the operation type of the data is deletion, if so, directly executing the step S34 to delete the data; the negative jumps to step S33 to discard the data.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (8)
1. A method for configuring full data and incremental data for a streaming synchronous network manager is characterized by comprising the following steps:
s1: the incremental data processing task starts to receive and process the incremental message;
s2: monitoring whether full data needs to be synchronously processed or not in the incremental message processing process, if so, turning to the step S4, and if not, turning to the step S3;
s3: sequentially warehousing the incremental messages;
s4: if it is monitored that the full data needs to be processed in the incremental message processing process, immediately pausing and blocking the incremental data processing task;
s5: starting a full data synchronous processing task;
s6: after the full data synchronous processing is finished, the full data is put into a warehouse;
s7: after the full data is put into the database, the incremental data processing task is unblocked, and the process returns to S1; wherein the incremental data processing task comprises:
s21: starting an incremental data processing task and reading a latest incremental data;
s22: judging whether the piece of data is a full-scale starting mark or not; if yes, go to step S23, if not continue to execute step S24;
s23: if the mark is the full-volume start mark, the incremental data processing task is blocked, and the processing flow of the full-volume processing task is started;
s24: if the total quantity task is not the total quantity starting mark, acquiring the last total quantity synchronization time, and recording and storing the time after the total quantity task is processed;
s25: comparing the current data time with the last full synchronization time; if the current data time is greater than the last full synchronization time, go to step S26; otherwise, the step S27 of incremental processing and full synchronous repeating data task is carried out;
s26: putting the data into a database;
s27: performing incremental processing and full synchronous repeated data task processing; wherein the incremental processing and full synchronization duplicate data tasks comprise:
s31: judging whether the operation type of the data is newly increased or not; if the new addition is made, jumping to step S32; otherwise, the next condition judgment is carried out to step S35;
s32: judging whether the database has the data; if yes, go to step S33; otherwise, jumping to step S34;
s33: invalid data is directly discarded;
s34: directly warehousing incremental data;
s35: judging whether the operation type of the data is updating, if so, jumping to the step S36; otherwise, the next condition judgment is carried out to step S37;
s36: judging whether the current record modification time is less than the latest record time of the database; if the current record modification time is less than the latest record time of the database, jumping to step S33; otherwise, jumping to step S34;
s37: judging whether the operation type of the data is deletion, if so, directly executing the step S34 to delete the data; the negative jumps to step S33 to discard the data.
2. The method of claim 1, wherein the webmaster logs for storing the full data and the incremental data follow a uniform format constraint, and each log needs to have a real event occurrence time of the log and also needs to have a data operation type of the log.
3. The method of claim 2, wherein the data operation types include add, update, and delete.
4. The method of claim 2, wherein the webmaster log generation adopts a multi-thread mode, and the log of the full data and the log of the incremental data are processed by independent threads respectively.
5. The method for configuring full data and incremental data by a streaming synchronization network manager according to claim 1 or 2, wherein a start marker is added at the beginning of the full data synchronization, an end marker M2 is added at the end of the full data synchronization, and a full start marker M3 is also added in the incremental data.
6. The method as claimed in claim 5, wherein a mark is typed into the log of the incremental data and records time when the full synchronization starts, a start mark is typed into the log of the full data and records time of the mark, an end mark is typed into the log end of the full data and records time when the output of the log of the full synchronization ends, and the start mark and the end mark need to have the same time stamp in order to ensure the integrity of the full data each time.
7. The method for configuring the full data and the incremental data by the streaming synchronization network manager according to claim 1 or 2, wherein the full data synchronization processing task comprises:
s11: reading the full data from the full queue;
s12: judging whether the read data is overtime, if not, turning to the step S13; otherwise go to step S14;
s13: judging whether the read data is a full end mark, if not, returning to the step S11 to continue reading the data; otherwise go to step S15;
s14: if the time is out when the whole data is read for processing, the read data is indicated to be dirty data, and the task is directly discarded and ended;
s15: if the reading is the end mark, loading all the data read this time to a disk, and then putting the full data into a warehouse in batch;
s16: and recording the last full synchronization time, and finishing the full data synchronization processing task.
8. An electronic device, comprising:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110447453.9A CN113206757B (en) | 2021-04-25 | 2021-04-25 | Method and electronic equipment for configuring full data and incremental data by stream type synchronous network management |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110447453.9A CN113206757B (en) | 2021-04-25 | 2021-04-25 | Method and electronic equipment for configuring full data and incremental data by stream type synchronous network management |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113206757A CN113206757A (en) | 2021-08-03 |
| CN113206757B true CN113206757B (en) | 2022-04-19 |
Family
ID=77028331
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110447453.9A Active CN113206757B (en) | 2021-04-25 | 2021-04-25 | Method and electronic equipment for configuring full data and incremental data by stream type synchronous network management |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113206757B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113656511B (en) * | 2021-10-20 | 2022-02-18 | 天津南大通用数据技术股份有限公司 | Heterogeneous database increment synchronization method and system based on source database non-outage |
| CN115114344B (en) * | 2021-11-05 | 2023-06-23 | 腾讯科技（深圳）有限公司 | Transaction processing method, device, computing equipment and storage medium |
| CN113824651B (en) * | 2021-11-25 | 2022-02-22 | 上海金仕达软件科技有限公司 | Market data caching method and device, storage medium and electronic equipment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102163308A (en) * | 2011-04-18 | 2011-08-24 | 中国科学院计算技术研究所 | Flow service data uploading system and method |
| CN103905397A (en) * | 2012-12-28 | 2014-07-02 | 腾讯科技（深圳）有限公司 | Data synchronization method and system |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102752372A (en) * | 2012-06-18 | 2012-10-24 | 天津神舟通用数据技术有限公司 | File based database synchronization method |
| CN103685585B (en) * | 2012-09-07 | 2016-11-23 | 中国科学院计算机网络信息中心 | A kind of highly reliable DNS data update method and system |
| CN104881494B (en) * | 2015-06-12 | 2019-02-19 | 北京奇虎科技有限公司 | The methods, devices and systems synchronous with Redis server progress data |
| CN107527091B (en) * | 2016-10-14 | 2021-05-25 | 腾讯科技（北京）有限公司 | Data processing method and device |
| CN107704550B (en) * | 2017-09-26 | 2021-04-30 | 深圳前海微众银行股份有限公司 | File migration method and device and computer readable storage medium |
| CN109871394B (en) * | 2019-01-17 | 2022-11-11 | 苏宁易购集团股份有限公司 | Full-distributed high-concurrency calculation method and device |
| CN110866158B (en) * | 2019-11-14 | 2021-01-26 | 北京沃东天骏信息技术有限公司 | Information updating method, device, system, storage medium and electronic equipment |
-
2021
- 2021-04-25 CN CN202110447453.9A patent/CN113206757B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102163308A (en) * | 2011-04-18 | 2011-08-24 | 中国科学院计算技术研究所 | Flow service data uploading system and method |
| CN103905397A (en) * | 2012-12-28 | 2014-07-02 | 腾讯科技（深圳）有限公司 | Data synchronization method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113206757A (en) | 2021-08-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113206757B (en) | Method and electronic equipment for configuring full data and incremental data by stream type synchronous network management | |
| CN104679772B (en) | Method, apparatus, equipment and the system of file are deleted in Distributed Data Warehouse | |
| CN108563502B (en) | Task scheduling method and device | |
| CN112019369A (en) | Dynamic configuration management method and system under micro-service framework | |
| CN102968486B (en) | A kind of highly reliable file synchronisation method based on change journal | |
| CN111104445A (en) | Data synchronization method, device and equipment | |
| CN108804523A (en) | Method of data synchronization, system and computer readable storage medium | |
| CN104750738B (en) | A kind of method for updating the data information, back end, management node and system | |
| CN103581231A (en) | UGC primary standby data synchronization method and system thereof | |
| CN106453297A (en) | Master and slave time delay detection method, device and system | |
| CN111177254B (en) | Method and device for data synchronization between heterogeneous relational databases | |
| CN107040576A (en) | Information-pushing method and device, communication system | |
| CN111064789A (en) | Data migration method and system | |
| CN112069195A (en) | Database-based message transmission method and device, electronic equipment and storage medium | |
| CN109657000B (en) | Real-time data synchronization method and device of rail transit comprehensive monitoring system | |
| CN110866068B (en) | Advertisement data storage method and device based on HDFS | |
| CN113348450A (en) | Vehicle-mounted data storage method and system | |
| CN114297216B (en) | Data synchronization method and device, computer storage medium and electronic equipment | |
| CN115002507A (en) | Video data updating method, device, equipment and readable storage medium | |
| CN112000492B (en) | Public number user tag management system and method | |
| CN112860804B (en) | Fuse quota data synchronization method and device | |
| CN117539534A (en) | Version synchronization method and device for database fingerprint model | |
| CN117708243A (en) | Multisource heterogeneous data synchronization method based on Flink CDC | |
| CN116701540A (en) | Data synchronization method, device, equipment and medium | |
| CN116881365A (en) | Method, device, equipment and storage medium for field synchronization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |