CN115630120A - Data synchronization method, device, equipment and storage medium - Google Patents

Abstract

The invention relates to a data synchronization method, a device, equipment and a storage medium based on processing node state monitoring, wherein the method comprises the following steps: receiving a synchronous request of a data demand party; responding to the data synchronization request, and detecting whether the data to be synchronized is synchronized to a CSS temporary table; if the data to be synchronized is synchronized to the CSS temporary table, detecting whether the data to be synchronized is extracted; if the data extraction is finished, detecting whether the Saturn file transmission task is successfully scheduled; if the scheduling is successful, detecting whether the data is unloaded; if the number is unloaded, detecting whether the data is synchronized to a CHANNEL temporary table; if the data is synchronized to the CHANNEL temporary table, detecting whether the data is synchronized to the UM system; and if the data is synchronized to the UM system, completing data synchronization. The data synchronization is assisted by introducing the state monitoring of the processing nodes, so that the real-time performance of the data synchronization is ensured, the testing efficiency is improved, and the technical problems of poor real-time performance and low testing efficiency of the existing data synchronization are solved.

Description

Data synchronization method, device, equipment and storage medium

Technical Field

The application relates to the field of monitoring, in particular to a technology for monitoring and correcting nodes through software, and particularly discloses a data synchronization method, a device, equipment and a storage medium based on processing node state monitoring.

Background

In the current test environment, nodes are scattered due to environmental barriers, and the created account data cannot be automatically synchronized to other required application systems. Especially, the network in the wireless environment has the characteristics of limited bandwidth resources, unstable link state, large communication delay and the like, and the load of message diffusion is increased along with the increase of the number of sent messages; and the data synchronization link is long, the number of applications is large, the problem that each synchronization node application responsible person needs to be found manually to advance data synchronization in the existing process is solved, and the real-time performance and the testing efficiency of data are greatly influenced.

Therefore, how to solve the technical problems of poor real-time performance and low test efficiency of the existing data synchronization becomes a technical problem to be solved urgently at present.

Disclosure of Invention

The invention mainly aims to provide a data synchronization method, a data synchronization device, data synchronization equipment and a data synchronization storage medium based on processing node state monitoring, and aims to solve the technical problems of poor data synchronization real-time performance and low test efficiency in the prior art.

In order to achieve the above object, the present invention provides a data synchronization method based on processing node status monitoring, wherein the method includes: receiving a synchronous request of a data demand party; responding to the data synchronization request, and detecting whether the data to be synchronized is synchronized to a CSS temporary table; if the data to be synchronized is synchronized to the CSS temporary table, detecting whether the data to be synchronized is extracted; if the data extraction is finished, detecting whether the Saturn file transmission task is successfully scheduled or not; if the scheduling is successful, detecting whether the data is unloaded; if the data is unloaded, detecting whether the data is synchronized to a CHANNEL temporary table; if the data is synchronized to the CHANNEL temporary table, detecting whether the data is synchronized to the UM system; and if the data are synchronized to the UM system, completing data synchronization.

In one embodiment, after the step of detecting whether the data to be synchronized is synchronized to the CSS temporary table, the method further includes: if the data are not synchronized to the CSS temporary table, scheduling a CSS synchronization trigger to synchronize the data to the CSS temporary table; and if the first preset times of continuous scheduling are not synchronized to the CSS temporary table, generating first early warning information and sending the first early warning information to the affiliate.

In one embodiment, after the step of detecting whether the data to be synchronized has completed data extraction, the method further includes: if the data extraction is not finished, logging in a DS platform to schedule a corresponding data extraction task; and if the data extraction is not finished after the second preset times of continuous scheduling, generating second early warning information and sending the second early warning information to the affiliate.

In one embodiment, after the step of detecting whether the Saturn file transmission task is scheduled successfully, the method further includes: if the file transmission task is not scheduled successfully, logging in the Saturn platform to schedule the corresponding file transmission task; and if the Saturn file transmission task is not completed for the third preset times after continuous scheduling, generating third early warning information and sending the third early warning information to the major personnel.

In one embodiment, after the step of detecting whether the data is unloaded, the method further comprises: if the unloading is not finished, logging in an ODS platform to schedule a corresponding unloading task; and if the fourth preset times of continuous scheduling are not unloaded successfully, generating fourth early warning information and sending the fourth early warning information to the major personnel.

In one embodiment, after the step of detecting whether the data has been synchronized to the CHANNEL temporary table, the method further comprises: if not, scheduling the CHANNEL synchronization trigger to synchronize data to the CHANNEL temporary table; and if the fifth preset frequency of continuous scheduling is not synchronized to the CHANNEL temporary table, generating fifth early warning information and sending the fifth early warning information to the main staff.

In one embodiment, after the step of detecting whether the data has been synchronized to the UM system, the method further comprises: if the synchronization is not synchronized to the UM system, logging in the OGG platform to schedule a corresponding synchronization task; and if the sixth preset times of continuous scheduling are not synchronized to the UM system, generating sixth early warning information and sending the sixth early warning information to the major personnel.

In order to achieve the above object, the present invention provides a data synchronization apparatus, wherein the apparatus includes: the receiving module is used for receiving a synchronous request of a data demand party; and the processing node monitoring module is used for responding to the data synchronization request, detecting whether the data to be synchronized is synchronized to a CSS temporary table, if so, detecting whether the data to be synchronized completes data extraction, if so, detecting whether a Saturn file transmission task is successfully scheduled, if so, detecting whether the data is unloaded, if so, detecting whether the data is synchronized to a CHANNEL temporary table, and if so, detecting whether the data is synchronized to a UM system.

In order to achieve the above object, the present invention provides an electronic device, wherein the electronic device includes: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the method for data synchronization based on monitoring of status of processing nodes in the above method embodiments.

In order to achieve the above object, the present invention provides a computer-readable storage medium storing a computer program, where the computer program, when executed by a processor, implements the data synchronization method based on monitoring of the state of the processing node in the above method embodiments.

Compared with the prior art, the embodiment of the invention has the following beneficial effects:

the embodiment of the invention receives a synchronous request of a data demand party; responding to the data synchronization request, and detecting whether the data to be synchronized is synchronized to a CSS temporary table; if the data to be synchronized is synchronized to the CSS temporary table, detecting whether the data to be synchronized is extracted; if the data extraction is finished, detecting whether the Saturn file transmission task is successfully scheduled; if the scheduling is successful, detecting whether the data is unloaded; if the number is unloaded, detecting whether the data is synchronized to a CHANNEL temporary table; if the data is synchronized to the CHANNEL temporary table, detecting whether the data is synchronized to the UM system; and if the data are synchronized to the UM system, completing data synchronization. The data synchronization is assisted by introducing the state monitoring of the processing nodes, so that the real-time performance of the data synchronization is ensured, the testing efficiency is improved, and the technical problems of poor real-time performance and low testing efficiency of the existing data synchronization are solved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.

Fig. 1 is a schematic diagram of an application environment of a data synchronization method based on processing node status monitoring according to an embodiment of the present invention;

fig. 2 is a flowchart of a data synchronization method based on processing node status monitoring according to a second embodiment of the present invention;

fig. 3 is a flowchart of a data synchronization method based on monitoring of a state of a processing node according to a third embodiment of the present invention;

fig. 4 is a flowchart of a data synchronization method based on monitoring of a state of a processing node according to a fourth embodiment of the present invention;

fig. 5 is a flowchart of a data synchronization method based on monitoring of a state of a processing node according to a fifth embodiment of the present invention;

fig. 6 is a flowchart of a data synchronization method based on processing node status monitoring according to a sixth embodiment of the present invention;

fig. 7 is a flowchart of a data synchronization method based on processing node status monitoring according to a seventh embodiment of the present invention;

fig. 8 is a flowchart of a data synchronization method based on processing node status monitoring according to an eighth embodiment of the present invention;

fig. 9 is a schematic structural diagram of a data synchronization apparatus according to a ninth embodiment of the present invention;

fig. 10 is a schematic structural diagram of a computer device according to a tenth embodiment of the present invention.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to" determining "or" in response to detecting ". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".

Furthermore, in the description of the present specification and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing a relative importance or importance.

Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present invention. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless expressly specified otherwise.

It should be understood that, the sequence numbers of the steps in the following embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

In order to explain the technical means of the present invention, the following description will be given by way of specific examples.

The data synchronization method based on processing node state monitoring provided by the embodiment of the invention can be applied to the application environment shown in fig. 1, wherein a client communicates with a server. The client includes, but is not limited to, a palm top computer, a desktop computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cloud terminal device, a Personal Digital Assistant (PDA), and other computer devices. The server can be implemented by an independent server or a server cluster composed of a plurality of servers.

Referring to fig. 2, which is a schematic flowchart of a data synchronization method based on processing node state monitoring according to a second embodiment of the present invention, the data synchronization method based on processing node state monitoring may be applied to the server in fig. 1. As shown in fig. 2, the data synchronization method based on the monitoring of the state of the processing node may include the following steps:

step S201, receiving a synchronization request from a data consumer.

Typically, each business system has a corresponding trading platform, such as a bank's website, stock exchange software, etc. As shown in the embodiment of fig. 1, the server of each service system is configured to process various service data on the transaction platform of the service system, and can count and store data corresponding to each account according to the service data. The data requiring party can directly send data synchronization to the server side, requests to directly acquire data from the server side of the target business system, and the server side receives the data synchronization request and sends the data to the data requiring party.

In practical application, the server of the service system may also store the data in a third-party database (such as a cloud database), and in this case, the data demanding party may send data synchronization to the server first, request the server to acquire the data, receive the data synchronization request, retrieve the data from the third-party database according to the request, and then send the retrieved data to the data demanding party.

Step S202, in response to the data synchronization request, detects whether the data to be synchronized has been synchronized to a Common Search Space (CSS) temporary table.

Step S203, if the data to be synchronized has been synchronized to the CSS temporary table, detecting whether the data to be synchronized has completed data extraction.

In particular, the extraction of data may be performed by listening, timing, or waiting for a user instruction. Wherein, the monitoring mode is as follows: when the data source inputs new data, the data extraction tool extracts the data when detecting the generation of the new data; the timing mode comprises the following steps: the data extraction tool periodically extracts newly generated data; mode of waiting for user instruction: data is uploaded manually by a user.

Step S204, if the data extraction is finished, whether the Saturn file transmission task is scheduled successfully is detected.

Specifically, the scheduling platform is responsible for scheduling the audited data and monitoring the flow direction of the audited data, and for the embodiment of the invention, the scheduling platform can be realized through a task scheduling system Saturn, specifically, each service logic is packaged into a task and issued to the public platform, then the configuration of the task and resources, the control and management of the task, the monitoring of the task execution state, the display and the alarm of an abnormal task and the like are performed, and the traditional Linux Cron/Spring Batch Job mode is replaced, so that the global unified configuration and unified monitoring, the high availability of the task and the fragment concurrent processing are realized.

Step S205, if the scheduling is successful, detecting whether the data is unloaded.

Specifically, unloading number: the data unloading party (data provider) exports and writes the data from the database of the local node into an external file according to a standard data exchange mode, and pushes the data to the data demand party by using a data transmission tool.

Before a data demander uses a data unloading tool to unload data in a database to a target data table, the data demander needs to firstly establish the requirement of the target data table to form a corresponding data unloading requirement file, the data unloading tool analyzes the data unloading requirement file of a data provider to obtain the basic information of the target data table, and the data unloading requirement file is used for recording the basic information and relevant requirements of the target data table. The basic information comprises a table name, a number unloading date, a number unloading mode, a multi-entity identifier, an additional condition and the like.

In step S206, if the number is unloaded, it is checked whether the data is synchronized to the CHANNEL temporary table.

Specifically, the data to be synchronized acquired in the database a is sent to the database B through a certain mechanism, and the data is loaded into the database B through the SQL specification, where the loading process may include: starting a transaction Txa; inserting a channel _ id identifier of a database A into a transaction; inserting service data into the transaction; clearing the channel _ id identification of the database A; transaction Txa is committed to database B.

In step S207, if the CHANNEL temporary table is synchronized, it is checked whether the data is synchronized to the UM system.

Specifically, the most common technical means of data integration is to use ETL, OGG, DBlink, webservice, etc. to complete the extraction, conversion, and loading processes of data from the business system to the data warehouse.

In step S208, if the data is synchronized to the UM system, the data synchronization is completed.

Furthermore, the embodiment of the present invention may use the state monitoring software of the processing node to monitor the state of the processing node, where the state monitoring software of the processing node is installed in the server; the server may be a terminal, such as a mobile phone or a tablet computer, and the server may also be a server or a service cluster formed by multiple servers.

Specifically, the method for monitoring the state of the processing node includes:

step 1, acquiring processing state information of a processing node.

The processing state information is used for representing the processing states of the processing nodes, and the processing states comprise normal states and stagnation states, and the stagnation states indicate that processing errors occur in the processing nodes. For example, when the processing node is in a normal state, the processing node may generate the processing state information representing the normal state and send the processing state information representing the dead state to the server side installed with the state monitoring software.

And 2, if the processing state information represents that the processing node is in a stagnation state, generating a target control instruction according to a preset processing strategy.

The processing policy is configured in advance, and represents which operation the processing node needs to execute, and specifically, the processing policy may include an exception skipping policy and a delay retry policy; the abnormal skipping strategy indicates that the processing node can skip part of data in the operation data and then process the skipped operation data again.

Step 3, sending a target control instruction to a processing node; the target control instructions are for controlling the processing nodes to perform target data processing operations.

The target data processing operation refers to the operation of the processing node on the operation data based on the target control instruction.

And 4, acquiring the operation state information of the processing node.

The operation state information is used for representing the result of the self-healing attempt of the processing node, and comprises operation success and operation failure; for example, when the processing node successfully executes the target data processing operation, the processing node may generate operation state information representing that the operation is successful and send the operation state information to the state monitoring software, and when the processing node fails to execute the target data processing operation, the processing node may generate operation state information representing that the operation is failed and send the operation state information to the state monitoring software.

Step 5, if the operation state information represents that the target data processing operation fails, outputting early warning information according to the processing node;

if the operation state information is generated and sent by the processing node, the state monitoring software can execute the subsequent steps after receiving the operation state information representing the successful operation; and if the state monitoring software receives the operation state information representing the operation failure, the subsequent steps are not executed.

The early warning information can be output in various ways, such as by telephone, mail, short message and the like, it can be understood that the data forms of the early warning information corresponding to different output ways are different, including the forms of voice, text, pictures, video and the like, and the early warning information is mainly used for reminding a manager to perform manual repair;

the output of the early warning information is based on the processing node, so that a manager can determine the processing node according to the early warning information, namely can know which processing node needs to be manually repaired currently; specifically, related information capable of representing the identity of the processing node may be directly added to the warning information, for example, the real-time node position where the processing node is currently located in the entire processing system, so that the manager may directly locate the processing node according to the real-time node position.

Referring to fig. 3, which is a schematic flowchart of a data synchronization method based on processing node status monitoring according to a third embodiment of the present invention, after step S202, the method further includes:

step S301: if not, a CSS synchronization trigger is scheduled to synchronize data to the CSS temporary table.

Step S302: and if the first preset times of continuous scheduling are not synchronized to the CSS temporary table, generating first early warning information and sending the first early warning information to the affiliate.

The first preset times can be 3-5 times, and the first early warning information is sent out in a mode of displaying an interface or sending an email or notifying a short message according to a predefined notification strategy.

Referring to fig. 4, which is a schematic flowchart of a data synchronization method based on processing node status monitoring according to a fourth embodiment of the present invention, after step S203, the method further includes:

step S401: if the data extraction is not finished, logging in the DS platform to schedule the corresponding data extraction task.

Step S402: and if the data extraction is not finished after the second preset times of continuous scheduling, generating second early warning information and sending the second early warning information to the affiliate.

The second preset times can be 3-5 times, and the second early warning information is sent out in a mode of displaying an interface or sending an email or notifying a short message according to a predefined notification strategy.

Referring to fig. 5, which is a schematic flowchart of a data synchronization method based on monitoring of a state of a processing node according to a fifth embodiment of the present invention, after step S204, the method further includes:

step S501: if the file transmission task is not scheduled successfully, logging in the Saturn platform to schedule the corresponding file transmission task.

Step S502: and if the Saturn file transmission task is not completed for the third preset times after continuous scheduling, generating third early warning information and sending the third early warning information to the contact personnel.

And sending the third early warning information in a predefined notification strategy through a display interface or a mail sending or short message notification mode, wherein the third preset times can be 3-5 times.

Referring to fig. 6, which is a schematic flowchart of a data synchronization method based on processing node status monitoring according to a sixth embodiment of the present invention, after step S205, the method further includes:

step S601: if the unloading is not finished, logging in the ODS platform to schedule the corresponding unloading task.

The method comprises the steps that a synchronization mechanism can be controlled by setting the scheduling priority of an ODS platform, when the generation of a corresponding task instance or the execution of a task is found before the ODS platform is scheduled, synchronization of an execution table structure is postponed, dependency of the task instance is established, and the synchronization of the execution table structure is carried out after the execution of the task instance is finished; when the corresponding task instance is found to be generated in the scheduling process of the ODS platform, the task instance is suspended, and the task instance is triggered to continue executing after the ODS platform is scheduled to be finished; and if the generation of the corresponding task instance is not found before and after the scheduling process of the ODS platform, the ODS is normally executed.

Step S602: and if the fourth preset times of continuous scheduling are not unloaded successfully, generating fourth early warning information and sending the fourth early warning information to the major personnel.

And sending the fourth early warning information in a predefined notification strategy through a display interface or a mail sending or short message notification mode, wherein the fourth preset times can be 3-5 times.

Referring to fig. 7, which is a schematic flowchart of a data synchronization method based on monitoring of a state of a processing node according to a seventh embodiment of the present invention, after step S206, the method further includes:

step S701: if not synchronized to the CHANNEL temporary table, a CHANNEL synchronization trigger is scheduled to synchronize data to the CHANNEL temporary table.

Step S702: and if the fifth preset frequency of continuous scheduling is not synchronized to the CHANNEL temporary table, generating fifth early warning information and sending the fifth early warning information to the main staff.

And sending the fifth early warning information in a predefined notification strategy through a display interface or a mail sending or short message notification mode, wherein the fifth preset times can be 3-5 times.

Referring to fig. 8, which is a schematic flowchart of a data synchronization method based on processing node status monitoring according to an eighth embodiment of the present invention, after step S207, the method further includes:

step S801: and if the synchronization is not carried out to the UM system, logging in the OGG platform to schedule the corresponding synchronization task.

Step S802: and if the sixth preset times of continuous scheduling are not synchronized to the UM system, generating sixth early warning information and sending the sixth early warning information to the major personnel.

And sending the sixth early warning information in a predefined notification strategy through a display interface or a mail sending or short message notification mode, wherein the sixth preset times can be 3-5 times.

Fig. 9 is a block diagram illustrating a structure of a data synchronization apparatus according to a ninth embodiment of the present invention, where the data synchronization apparatus is applied to a client, in accordance with the data synchronization method based on monitoring of a state of a processing node according to the foregoing embodiment. For convenience of explanation, only portions related to the embodiments of the present invention are shown.

Referring to fig. 9, the data synchronization device 90 includes:

a receiving module 91, configured to receive a synchronization request of a data consumer;

and the processing node monitoring module 92 is configured to respond to the data synchronization request, detect whether data to be synchronized is synchronized to the CSS temporary table, detect whether data extraction of the data to be synchronized is completed when the data to be synchronized is synchronized to the CSS temporary table, detect whether a Saturn file transmission task is successfully scheduled when the data extraction is completed, detect whether data is unloaded when the scheduling is successful, detect whether data is synchronized to the CHANNEL temporary table when the data is unloaded, and detect whether data is synchronized to the UM system when the data is synchronized to the CHANNEL temporary table.

The processing node monitoring module 92 is further configured to schedule the CSS synchronization trigger to synchronize data to the CSS temporary table when the CSS temporary table is not synchronized; and when the first preset times of continuous scheduling are not synchronized to a CSS temporary table, generating first early warning information and sending the first early warning information to a contact person.

The processing node monitoring module 92 is further configured to log in the DS platform to schedule a corresponding data extraction task when data extraction is not completed; and when the data extraction is not completed for a second preset number of continuous scheduling, generating second early warning information and sending the second early warning information to the personnel in the line.

The processing node monitoring module 92 is further configured to log in the Saturn platform to schedule the corresponding file transmission task when the scheduling is not successful; and when the Saturn file transmission task is not completed for the third preset times after continuous scheduling, generating third early warning information and sending the third early warning information to the main staff.

The processing node monitoring module 92 is further configured to log in the ODS platform to schedule a corresponding number unloading task when the number unloading is not completed; and when the fourth preset times of continuous scheduling are not unloaded successfully, generating fourth early warning information and sending the fourth early warning information to the main staff.

Processing node monitoring module 92, further configured to schedule a CHANNEL synchronization trigger to synchronize data to the CHANNEL temporary table when not synchronized to the CHANNEL temporary table; and when the fifth preset frequency is not synchronized to the CHANNEL temporary table yet, generating fifth early warning information and sending the fifth early warning information to the main staff.

The processing node monitoring module 92 is further configured to log in the OGG platform to schedule the corresponding synchronization task when the synchronization is not synchronized to the UM system; and when the sixth preset times of continuous scheduling are not synchronized to the UM system, generating sixth early warning information and sending the sixth early warning information to the line personnel.

It should be noted that, because the contents of information interaction, execution process, and the like between the above units are based on the same concept as the method embodiment of the present invention, specific functions and technical effects thereof may be specifically referred to a part of the method embodiment, and details thereof are not described herein.

Fig. 10 is a schematic structural diagram of a computer device according to a tenth embodiment of the present invention. As shown in fig. 10, the computer apparatus of this embodiment includes: at least one processor (only one shown in fig. 10), a memory, and a computer program stored in the memory and executable on the at least one processor, the processor when executing the computer program implementing the steps in any of the various item recommendation method embodiments described above.

The computer device may include, but is not limited to, a processor, a memory. Those skilled in the art will appreciate that fig. 10 is merely exemplary of a computing device and is not intended to limit the computing device, which may include more or fewer components than those shown, or some of the components may be combined, or different components may be included, such as a network interface, a display screen, and input devices, etc.

The Processor may be a CPU, or other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory includes readable storage media, internal memory, etc., wherein the internal memory may be the internal memory of the computer device, and the internal memory provides an environment for the operating system and the execution of the computer-readable instructions in the readable storage media. The readable storage medium may be a hard disk of the computer device, and in other embodiments may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, provided on the computer device. Further, the memory may also include both internal and external storage units of the computer device. The memory is used for storing an operating system, application programs, a BootLoader (BootLoader), data, and other programs, such as program codes of a computer program, and the like. The memory may also be used to temporarily store data that has been output or is to be output.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules, so as to perform all or part of the functions described above. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of the target application functional unit. In addition, specific names of the functional units and modules are only used for distinguishing one functional unit from another, and are not used for limiting the protection scope of the present invention. For the specific working processes of the units and modules in the above-mentioned apparatus, reference may be made to the corresponding processes in the foregoing method embodiments, which are not described herein again. The integrated unit, if implemented in the form of the target application functional unit and sold or used as a separate product, may be stored in a computer-readable storage medium. Based on such understanding, all or part of the flow of the method of the above embodiments may be implemented by a computer program, which may be stored in a computer readable storage medium and used for instructing relevant hardware, and when the computer program is executed by a processor, the steps of the above method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code, recording medium, computer Memory, read-Only Memory (ROM), random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, and the target application distribution medium. Such as a usb-drive, a removable hard drive, a magnetic or optical disk, etc. In some jurisdictions, computer-readable media may not be an electrical carrier signal or a telecommunications signal in accordance with legislative and proprietary practices.

The present invention may also be implemented by a computer program product, which when executed on a computer device, enables the computer device to implement all or part of the processes in the method according to the above embodiments.

In the above embodiments, the description of each embodiment has its own emphasis, and reference may be made to the related description of other embodiments for parts that are not described or recited in any embodiment.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of both computer-described target application programs and electronic hardware. Whether these functions are performed in hardware or the target application depends on the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

In the embodiments provided by the present invention, it should be understood that the disclosed apparatus/computer device and method may be implemented in other ways. For example, the above-described apparatus/computer device embodiments are merely illustrative, and for example, a module or a unit may be divided into only one logical function, and may be implemented in other ways, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

The data synchronization method, apparatus, computer device and storage medium based on processing node status monitoring provided by the present invention are introduced in detail, and specific examples are applied herein to explain the principle and implementation of the present invention, and the above examples are only used to illustrate the technical solution of the present invention, but not to limit the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein.

Claims (10)

1. A data synchronization method based on processing node state monitoring is characterized by comprising the following steps:

receiving a synchronous request of a data demand party;

responding to the data synchronization request, and detecting whether the data to be synchronized is synchronized to a CSS temporary table;

if the data to be synchronized is synchronized to the CSS temporary table, detecting whether the data to be synchronized is extracted;

if the data extraction is finished, detecting whether the Saturn file transmission task is successfully scheduled or not;

if the scheduling is successful, detecting whether the data is unloaded;

if the number is unloaded, detecting whether the data is synchronized to a CHANNEL temporary table;

if the data is synchronized to the CHANNEL temporary table, detecting whether the data is synchronized to the UM system;

and if the data are synchronized to the UM system, completing data synchronization.

2. The processing node status monitoring based data synchronization method of claim 1, wherein after the step of detecting whether the data to be synchronized has been synchronized to the CSS temporary table, the method further comprises:

if the data are not synchronized to the CSS temporary table, scheduling a CSS synchronization trigger to synchronize the data to the CSS temporary table;

and if the first preset times of continuous scheduling are not synchronized to the CSS temporary table, generating first early warning information and sending the first early warning information to the affiliate.

3. The processing node status monitoring based data synchronization method of claim 1, wherein after the step of detecting whether the data to be synchronized has completed data extraction, the method further comprises:

if the data extraction is not finished, logging in a DS platform to schedule a corresponding data extraction task;

and if the data extraction is not finished after the second preset times of continuous scheduling, generating second early warning information and sending the second early warning information to the affiliate.

4. The processing node status monitoring-based data synchronization method as claimed in claim 1, wherein after the step of detecting whether the Saturn file transmission task is scheduled successfully, the method further comprises:

if the file transmission task is not successfully scheduled, logging in the Saturn platform to schedule the corresponding file transmission task;

and if the Saturn file transmission task is not completed for the third preset times after continuous scheduling, generating third early warning information and sending the third early warning information to the major personnel.

5. The processing node status monitoring based data synchronization method of claim 1, wherein after said step of detecting whether data has been offloaded, the method further comprises:

if the unloading is not finished, logging in an ODS platform to schedule a corresponding unloading task;

and if the fourth preset times of continuous scheduling are not unloaded successfully, generating fourth early warning information and sending the fourth early warning information to the major personnel.

6. The method for data synchronization based on status monitoring of processing nodes as claimed in claim 1, wherein after said step of detecting whether data has been synchronized to CHANNEL temporary table, said method further comprises:

if not, scheduling a CHANNEL synchronization trigger to synchronize data to the CHANNEL temporary table;

and if the fifth preset times of continuous scheduling are not synchronized to the CHANNEL temporary table, generating fifth early warning information and sending the fifth early warning information to the contact personnel.

7. The processing node status monitoring based data synchronization method as claimed in claim 1, wherein after said step of detecting whether data has been synchronized to a UM system, said method further comprises:

if the synchronization is not carried out to the UM system, logging in an OGG platform to schedule a corresponding synchronization task;

and if the sixth preset times of continuous scheduling are not synchronized to the UM system, generating sixth early warning information and sending the sixth early warning information to the major personnel.

8. A data synchronization apparatus, the apparatus comprising:

the receiving module is used for receiving a synchronous request of a data demand party;

and the processing node monitoring module is used for responding to the data synchronization request, detecting whether the data to be synchronized is synchronized to a CSS temporary table, if so, detecting whether the data to be synchronized completes data extraction, if so, detecting whether a Saturn file transmission task is successfully scheduled, if so, detecting whether the data is unloaded, if so, detecting whether the data is synchronized to a CHANNEL temporary table, and if so, detecting whether the data is synchronized to a UM system.

9. An electronic device, characterized in that the electronic device comprises:

at least one processor; and (c) a second step of,

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform a method of data synchronization based on monitoring of status of processing nodes as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, carries out a method for data synchronization based on monitoring of the status of processing nodes according to any one of claims 1 to 7.

Images