CN110032569B - Method, device, equipment and system for checking changed data - Google Patents
Method, device, equipment and system for checking changed data Download PDFInfo
- Publication number
- CN110032569B CN110032569B CN201910044207.1A CN201910044207A CN110032569B CN 110032569 B CN110032569 B CN 110032569B CN 201910044207 A CN201910044207 A CN 201910044207A CN 110032569 B CN110032569 B CN 110032569B
- Authority
- CN
- China
- Prior art keywords
- data
- source
- task
- change data
- target data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Abstract
In the embodiment of the present disclosure, when receiving a data inspection request sent by a source data end, the data inspection request includes source change data sent by the source data end to a target data end, so that it can determine whether an inspection task indicating a task execution client to check, according to the source change data, whether the actually obtained change data in the target data end is valid, and send the inspection task to the task execution client configured in the target data end indicated by the system identifier, so that the task execution client checks, according to the source change data, whether the actually obtained change data in the target data end is valid, thereby implementing an inspection process in a data change process, implementing data inspection, improving inspection efficiency, and reducing labor cost.
Description
Technical Field
The present disclosure relates to the field of data processing, and in particular, to a method, apparatus, device, and system for checking changed data.
Background
With the advent of the internet age, the internet has been widely used in people's daily study, work and life. Various daily transactions of people can be processed through the network. Thus, the user can be provided with services through the service application system. At present, the source data terminal triggers the target data terminal to change data, which may be caused by the change of data in the memory, or may be caused by the trigger of an external system. For example, after a transaction occurs, the business application system may send the transaction data in the memory to the corresponding database for data modification, where in this example, the business application system may be used as a source data end and the database may be used as a target data end. For another example, the configuration issuing end sends the configuration data to a database corresponding to the service application system, and the service application system loads the configuration data from the database to the memory so as to use the configuration data to newly add or update the service. In this example, the configuration issuing side may be used as a source data side, and the service application system may be used as a target data side.
In practical applications, to ensure validity of the change data, it is often necessary to check whether the change data actually obtained at the target data terminal is desired change data. Currently, manual inspection is required. Taking the case of checking the change data in the memory, assuming that 1000 machines are on line, the inspector needs to log on the 1000 machines, and judging whether the actually obtained change data in the memory is the expected change data or not by means of a check log, so that 100% of the change data can be ensured to be correct. Since the machines are very many, the inspection workload is huge and the time is long.
Disclosure of Invention
In order to overcome the problems in the related art, the present specification provides a method, apparatus, device and system for checking modified data.
According to a first aspect of embodiments of the present specification, there is provided a change data inspection system, the system including a task publishing server and a task execution client configured in a target data side;
the task release server is used for:
receiving a data checking request sent by a source data end, wherein the data checking request comprises source change data sent to a target data end by the source data end and a system identifier of the target data end;
Determining an inspection task, and sending the inspection task to a task execution client configured in a target data end indicated by the system identifier, wherein the inspection task indicates whether the task execution client verifies whether the actually obtained change data in the target data end is valid or not according to the source change data;
the task execution client is used for executing the checking task.
According to a second aspect of embodiments of the present specification, there is provided a change data checking method, the method being applied to a task publishing server, the method including:
receiving a data checking request sent by a source data end, wherein the data checking request comprises source change data sent to a target data end by the source data end and a system identifier of the target data end;
and determining an inspection task, and sending the inspection task to a task execution client configured in the target data end indicated by the system identifier, wherein the inspection task indicates whether the task execution client verifies whether the actually obtained change data in the target data end is valid or not according to the source change data.
In one embodiment, the verifying whether the change data actually obtained in the target data end is valid according to the source change data includes:
Checking whether actually obtained change data in a target data end is matched with the source change data or not; or alternatively, the first and second heat exchangers may be,
checking whether actually obtained change data in the target data end is matched with data obtained by converting the source change data by using a conversion strategy, wherein the conversion strategy comprises conversion operation performed on the source change data from the source data end to the target data end.
In one embodiment, the actually obtained change data is obtained from a log file of the target data end, where the log file is generated by adopting a mode of printing a log when the data in the memory of the target data end is changed.
In one embodiment, the checking task carries a sending time of source change data sent by the active data end to the target data end, and the log file is generated after the sending time.
In one embodiment, the source data end sends source change data to a database corresponding to the target data end, the target data end loads the source change data from the database to the memory, and the data inspection request is a data inspection request sent to the task release server end by the source data end at intervals of preset delay time after the source change data is sent to the target data end; the preset delay time is determined based on the interval time of the target data end for loading data from the database to the memory.
In one embodiment, the source data end sends the source change data to a database corresponding to the target data end, the target data end loads the source change data from the database to the memory, the inspection task carries a delay processing time, the delay processing time is an interval time between a receiving time of the inspection task received by the task execution client and a starting time of executing the inspection task, and the delay processing time is determined based on an interval time of the target data end loading the data from the database to the memory.
According to a third aspect of embodiments of the present specification, there is provided a change data inspection method applied to a task execution client provided in a target data side, the method including:
receiving an inspection task sent by a task release server;
executing the checking task to check whether the actually obtained change data in the target data end is valid or not according to the source change data, wherein the source change data is the source change data sent to the target data end by the source data end;
and feeding back the checking result to the task issuing server.
In one embodiment, the verifying whether the change data actually obtained in the target data end is valid according to the source change data includes:
Checking whether actually obtained change data in a target data end is matched with the source change data or not; or alternatively, the first and second heat exchangers may be,
checking whether actually obtained change data in the target data end is matched with data obtained by converting the source change data by using a conversion strategy, wherein the conversion strategy comprises conversion operation performed on the source change data from the source data end to the target data end.
According to a fourth aspect of embodiments of the present disclosure, there is provided a change data checking device, the device being provided at a task issuing server, the device including:
a request receiving module, configured to: receiving a data checking request sent by a source data end, wherein the data checking request comprises source change data sent to a target data end by the source data end and a system identifier of the target data end;
a task determination module for: determining an inspection task, wherein the inspection task instructs a task execution client to check whether the actually obtained change data in the target data end is valid according to the source change data;
the task issuing module is used for: and sending the checking task to a task execution client configured in the target data end indicated by the system identifier.
In one embodiment, the inspection task includes:
checking whether actually obtained change data in a target data end is matched with the source change data or not; or alternatively, the first and second heat exchangers may be,
checking whether actually obtained change data in the target data end is matched with data obtained by converting the source change data by using a conversion strategy, wherein the conversion strategy comprises conversion operation performed on the source change data from the source data end to the target data end.
In one embodiment, the actually obtained change data is obtained from a log file of the target data end, where the log file is generated by adopting a mode of printing a log when the data in the memory of the target data end is changed.
In one embodiment, the checking task carries a sending time of source change data sent by the active data end to the target data end, and the log file is generated after the sending time.
In one embodiment, the source data end sends source change data to a database corresponding to the target data end, the target data end loads the source change data from the database to the memory, and the data inspection request is a data inspection request sent to the task release server end by the source data end at intervals of preset delay time after the source change data is sent to the target data end; the preset delay time is determined based on the interval time of the target data end for loading data from the database to the memory.
In one embodiment, the source data end sends the source change data to a database corresponding to the target data end, the target data end loads the source change data from the database to the memory, the inspection task carries a delay processing time, the delay processing time is an interval time between a receiving time of the inspection task received by the task execution client and a starting time of executing the inspection task, and the delay processing time is determined based on an interval time of the target data end loading the data from the database to the memory.
According to a fifth aspect of embodiments of the present specification, there is provided a change data inspection apparatus provided to a task execution client in a target data side, the apparatus including:
a task receiving module, configured to: receiving an inspection task sent by a task release server;
the task execution module is used for: executing the checking task to check whether the actually obtained change data in the target data end is valid or not according to the source change data, wherein the source change data is the source change data sent to the target data end by the source data end;
the result feedback module is used for: and feeding back the checking result to the task issuing server.
In one embodiment, the task execution module is specifically configured to:
checking whether actually obtained change data in a target data end is matched with the source change data or not; or alternatively, the first and second heat exchangers may be,
checking whether actually obtained change data in the target data end is matched with data obtained by converting the source change data by using a conversion strategy, wherein the conversion strategy comprises conversion operation performed on the source change data from the source data end to the target data end.
According to a sixth aspect of embodiments of the present specification, there is provided a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any one of the preceding claims when executing the program.
The technical scheme provided by the embodiment of the specification can comprise the following beneficial effects:
when receiving a data checking request sent by a source data end, the embodiment of the specification can determine whether the checking task indicating the task execution client end checks the actually obtained change data in the target data end according to the source change data because the data checking request comprises the source change data sent by the source data end to the target data end, and send the checking task to the task execution client end configured in the target data end indicated by the system identifier, so that the task execution client end checks whether the actually obtained change data in the target data end is effective according to the source change data, thereby realizing the introduction of a checking flow in a data changing flow, realizing the change, simultaneously performing the data checking, improving the checking efficiency and simultaneously reducing the labor cost.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and together with the description, serve to explain the principles of the specification.
Fig. 1 is a schematic diagram of a change data inspection system according to an exemplary embodiment of the present disclosure.
Fig. 2 is a flow chart of a change data inspection method according to an exemplary embodiment of the present description.
Fig. 3 is a flow chart of another method of change data inspection shown in this specification according to an exemplary embodiment.
Fig. 4 is a flow chart of another method of change data inspection shown in the present specification according to an exemplary embodiment.
Fig. 5 is a hardware configuration diagram of a computer device in which the modified data checking apparatus of the present specification is located.
Fig. 6 is a block diagram of a change data checking apparatus according to an exemplary embodiment of the present specification.
Fig. 7 is a block diagram of another change data checking apparatus according to an exemplary embodiment of the present specification.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present specification. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present description as detailed in the accompanying claims.
The terminology used in the description presented herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the description. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.
It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, the first information may also be referred to as second information, and similarly, the second information may also be referred to as first information, without departing from the scope of the present description. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "responsive to a determination", depending on the context.
At present, a source data terminal often triggers a target data terminal to change data. For example, a memory change in a business application system requires an external system trigger, which may be a configuration management class platform. The data change in the memory of the business application system is triggered by the external systems. Since the data sent by the source data terminal may cause the data in the target data terminal to be changed, the sent data may be referred to as changed data. The source change data can be sent to a Database (DB) corresponding to the target data end by the source data end, and the target data end loads the source change data from the database to the memory; the source change data may also be directly sent from the source data end to the target data end. In the whole link, it is necessary to check whether the actually obtained change data at the target data terminal accords with the expectation according to the source change data. At present, a non-real-time inspection scheme is adopted, and after the memory data of a business application system is changed, the changed information is generally printed on a server log. Because of the huge amount of logs in the service application system, logs on the server can only be synchronized to an ODPS (offline log storage system) for storage in an asynchronous mode. The configuration management type platform initiates change data to the service application system, then acquires a log through the docking ODPS manually, and then checks whether the actually obtained change data in the target data end is expected change data or not based on the log. However, since the machines are very many, the inspection work is huge and it takes a long time. And the sampling inspection mode is adopted, so that the risk of missed inspection can occur.
In view of this, the present embodiment provides an automatic inspection scheme.
As shown in fig. 1, a schematic diagram of a modification data inspection system according to an exemplary embodiment of the present disclosure is shown. The system comprises a task issuing server 12 and a task execution client 14 configured in a target data end;
the task release server 12 is configured to:
receiving a data checking request sent by a source data end, wherein the data checking request comprises source change data sent to a target data end by the source data end and a system identifier of the target data end;
and determining an inspection task, and sending the inspection task to a task execution client configured in the target data end indicated by the system identifier, wherein the inspection task indicates whether the task execution client verifies whether the actually obtained change data in the target data end is valid or not according to the source change data.
The task execution client 14 is for executing the inspection task.
Accordingly, the present disclosure further describes a change data checking scheme from the perspective of the task issuing server, as shown in fig. 2, which is a flowchart of a change data checking method according to an exemplary embodiment of the present disclosure. The method is applied to the task release server, and comprises the following steps:
In step 202, a data inspection request sent by a source data end is received, wherein the data inspection request includes source change data sent by the source data end to a target data end and a system identifier of the target data end;
in step 204, a checking task is determined, and the checking task is sent to a task execution client configured in the target data end indicated by the system identifier, where the checking task indicates that the task execution client verifies whether the actually obtained change data in the target data end is valid according to the source change data.
Accordingly, the present specification also describes a change data checking scheme from the perspective of the task execution client, as shown in fig. 3, which is a flowchart of another change data checking method according to an exemplary embodiment of the present specification. The method is applied to a task execution client arranged in a business application system, and comprises the following steps:
in step 302, receiving an inspection task sent by a task issuing server;
in step 304, the checking task is performed to check whether the actually obtained change data in the target data end is valid according to the source change data, where the source change data is the source change data sent by the source data end to the target data end.
In step 306, the checking result is fed back to the task publishing server.
For economy of description, the present specification will be described with reference to fig. 1, 2 and 3, wherein related technologies in different drawings are the same, and description thereof will not be repeated.
In practical applications, the change data actually obtained by the target data end may not be consistent with the change data sent by the source data end, so the change data sent by the source data end is called source change data to be distinguished from the change data actually obtained by the target data end. In order to ensure the validity of the change data, it is often necessary to check whether the change data actually obtained in the target data terminal is desired change data.
In order to realize automatic inspection, the embodiment is provided with a task execution client side at the target data side, and a task release server side in butt joint with the task execution client side is provided. The task issuing server is used for issuing the checking task, and the task execution client is used for executing the checking task.
The source data end is an end that triggers the target data end to generate data change, and may also be referred to as a data change trigger end, or a data source end. The source change data can be sent to a database corresponding to the target data end by the source data end, and the target data end loads the source change data from the database to the memory; or the source data end can directly send the data to the target data end.
For example, the data change in the memory of the service application system may be triggered by an external system, and the source change data sent by the source data end is data that may cause the data in the memory of the target data end to change. For example, the source data end may be a configuration publishing end, and the target data end may be a service application system. The configuration issuing terminal can issue the source change data to a database corresponding to the service application system so that the service application system can read the source change data from the corresponding database. The source change data may be configuration data of a service or configuration parameters of a system, for example, may be service configuration data required for a newly added service, service configuration data required for changing a service, system operation parameters required for changing a system operation state, and the like. The service configuration data required for the new/changed service may be a service rule. For example, assuming that the business scenario is code-scanning payment, the business rules (payment instrument) may be: balance payment, bank card payment, balance bank payment, no payment. For the system operation parameters, for example, the system operation parameters can be consumed by the system by sending the system operation parameters to the service application system, so that the service logic when the system is operated is dynamically changed.
As another example, data in the database may be changed due to changes in the business data in the memory of the business application system. For example, after a transaction occurs, the business application system sends the transaction data in the memory to the corresponding database to change the data, such as changing the daily credit, the monthly credit, etc.
The source data end may send a data inspection request to the task publishing server. The data check request may be a data check request sent to the task publishing server by the source data end simultaneously or after the source data end sends the source change data to the target data end. The data check request comprises source change data sent to a target data end by a source data end and a system identifier of the target data end. The target data end is a system that needs to be checked, and therefore, may also be referred to as a target data end to be checked. The system identifier may be an identifier capable of uniquely identifying the target data end, so that the task publishing server end can find the target data end to be checked according to the system identifier. In one example, the target data end may be a service application system and the system identification may be an IP address of the service application system. Since the source data end may send the source change data to a plurality of service application systems, the system identifier carried in the data inspection request may be an IP set (IP address set).
After the task issuing server receives the data check request, it can determine a check task for checking whether the change data actually obtained in the target data end is valid (validated) according to the source change data. The task publishing server may be used to maintain inspection tasks.
In one embodiment, the mapping relationship between the source change data and the checking task may be pre-stored in the task release server. In order to reduce the number of mappings, source change data that can be inspected using the same inspection task may be classified into one type to store mappings of each type of source change data and inspection task. And the checking task is used for checking whether the actually obtained change data in the target data end is valid or not according to the source change data. The configured inspection tasks are different for different types of source change data.
In another embodiment, the inspection task may also be a task that is generated after the source change data is obtained. For example, after the task issuing server obtains the data inspection request including the source change data and the system identifier, an inspection command script, such as a script command, is constructed by the source change data: execution (data). The script command is a command for checking whether the change data actually obtained in the target data side is valid.
In some application scenarios, the change data expected to be obtained by the target data end is the same as the source change data sent by the source data end, so the checking task may be to check whether the change data actually obtained in the target data end matches the source change data. In some cases, the change data expected to be obtained by the target data end may be the data transformed by the source data end. For example, in the data change link, the source data end may provide source change data to a plurality of service application systems, where different service application systems are based on different service operation requirements, and the required data is data obtained by converting the source change data. Corresponding auxiliary operation systems (or called intermediate systems) are configured for different business application systems, and the source data end sends source change data to a database of the business application system by calling the auxiliary operation systems. Specifically, the source data end sends the source change data to the auxiliary operation system, the auxiliary operation system converts the source change data, the converted data is sent to the database of the service application system, and the service application system can load the converted data into the cache from the database. It can be seen that the change data expected to be obtained by the target data end is different from the source change data sent by the source data end.
For example, source change data—operational non-technical classmate configuration data:
conditions are as follows: business scenario: external transactions
Sales product code: on-the-surface payment
The actions are as follows: flower-like Chinese character 'Bei', balance and balance treasures
The actual obtained change data:
expression: bizability=xxx & salep =on-the-fly payment
actionexcompression: # payalis=flower code # { payalis=balance code }
It is to be understood that the operation of converting the source change data may refer to the related art, and will not be described in detail herein.
While the present application introduces the inspection process simultaneously in the data change process. Taking an example, the a system sends data a to the database of the B system by calling the auxiliary operation system, the data is expressed as a ' in the database, the B system loads the database a ' of the B system, and the memory of the B system is expressed as a ', and the a ' needs to be checked by using a to judge whether the a ' is valid or not. a' is valid and may include: whether a 'exists and whether a' corresponds to a.
In view of this, it is checked whether the actually obtained change data in the target data side matches the data after the source change data is converted by the conversion policy. Wherein the conversion policy includes a conversion operation performed on the source change data during a period from the source data side to the target data side. For example, the intermediate system corresponding to the target data end performs a conversion operation on the source change data.
In one example, the conversion operation may be triggered to execute by the task publishing server. For example, the task issuing server may directly convert the source change data according to the conversion policy, or may obtain the data inspection request at the task issuing server, then convert the source change data by calling the intermediate system, then generate the inspection task based on the converted data, and issue the inspection task to the database of the corresponding service application system.
In another example, the conversion operation may be triggered and executed by the task execution client, for example, the task execution client further includes a conversion policy, and the task execution client directly converts the source change data according to the conversion policy, and then obtains the data from the log file and compares the converted data to determine whether to effectively load the data from the database into the memory. For another example, the task execution client converts the source change data by calling the intermediate system, acquires the data from the log file, compares the data with the converted data, and judges whether the data is effectively loaded into the memory from the database.
It can be seen that whether the obtained change data is valid or not can be judged by directly matching the obtained change data with the source change data, or whether the obtained change data is valid or not can be judged by matching the actually obtained change data with the data obtained by converting the source change data by using the conversion strategy, and the configuration is specifically performed according to the requirement.
In the scenario that the source change data is sent to the target data end by the source data end through the database corresponding to the target data end, the checking task may further include a task of checking whether the change data stored in the database is consistent with the source change data.
In particular, embodiments of the present disclosure enable automated inspection of altered memory data. For example, the target data end may be a service application system, and since the change data in the memory cannot be directly obtained, in order to be able to check the change data in the memory of the service application system, the log file of the service application system may be used to implement automatic checking. Correspondingly, the actually obtained change data is obtained from a log file of the target data end, wherein the log file is generated in a printing log mode when the data in the memory of the target data end is changed.
It can be seen that the actually obtained change data is checked by the checking task by acquiring the actually obtained change data from the log file.
In practical application, because the log files of the service application system are often generated according to time periods, and the log files generated in each time period may have larger data volume, in order to improve the checking efficiency, the checking task carries the sending time of the source change data sent by the active data end to the target data end, and the log files are generated after the sending time.
Therefore, only from the log file generated by the service application system after the sending time, whether the data corresponding to the source change data exists or not is judged, so that the checking efficiency can be improved.
After the task issuing server obtains the change task, the check task may be sent to the task execution client configured in the target data end indicated by the system identifier. The task execution client provided in the target data terminal may execute a check task to check whether the change data actually obtained in the target data terminal is valid according to the source change data, and may feed back the check result to the task issuing server terminal.
In an application scenario that the source data end sends source change data to a database corresponding to the target data end, the target data end loads the source change data from the database to the memory, and a certain interval time is required for loading the data from the database to the buffer area by the target data end, so that a situation that an inspection task is executed without being loaded to the change data due to the interval time is avoided, and correspondingly, delay time is also set.
In one embodiment, the data inspection request is a data inspection request that is sent to the task publishing server by the source data end at intervals of a preset delay time after the source data end sends the source change data to the target data end; the preset delay time is determined based on the interval time of the target data end for loading data from the database to the memory. For example, assume that the target data side performs a load operation once every interval t1. The preset delay time may be equal to t1 or may even be greater than t1.
Therefore, the embodiment achieves the aim of delaying execution of the checking task by delaying transmission of the data checking request, thereby avoiding the situation that the target data end is not loaded to the changed data but the checking task is executed, and improving the checking accuracy.
In another embodiment, the inspection task carries a delay processing time, where the delay processing time is an interval time between a receiving time of the inspection task received by the task execution client and a starting time of executing the inspection task, that is, after the task execution client receives the inspection task, the task execution client delays the delay processing time and then executes the inspection task. The delay processing time is determined based on the interval time of the target data end for loading data from the database to the memory. For example, assume that the target data side performs a load operation once every interval t1. Then the delay processing time may be equal to t1 or even greater than t1.
Therefore, in the embodiment, the task execution client delays execution of the inspection task according to the delay processing time, so that the situation that the target data end is not loaded to the changed data but the inspection task is executed is avoided, and the inspection accuracy is improved.
It can be understood that the execution time of the inspection task may also be determined by other means, so as to avoid a situation that the target data end is not loaded to the changed data but performs the inspection task, which is not described in detail herein.
The various technical features of the above embodiments may be arbitrarily combined as long as there is no conflict or contradiction between the features, but are not described in detail, and therefore, the arbitrary combination of the various technical features of the above embodiments is also within the scope of the disclosure of the present specification.
The following is an illustration of one of the combinations.
As shown in fig. 4, there is a flowchart of another modification data checking method according to an exemplary embodiment of the present specification. The embodiment is described by checking memory change data, wherein the target data end is a service application system, and the method comprises the following steps:
in step 402, a task publishing server receives a data check request sent by a source data end, where the data check request includes source change data sent by the source data end to a service application system, a system identifier of the target data end, and a sending time of the source data end to send the source change data to the service application system;
in step 404, the task publishing server determines a checking task for checking whether the change data actually obtained in the service application system is valid according to the source change data.
The inspection task may also carry a delay processing time, where the delay processing time is determined based on an interval time between loading data from the database to the memory by the target data terminal.
In step 406, the task publishing server sends the check task to a task execution client configured in the business application system indicated by the system identifier.
In step 408, the task execution client receives the inspection task and executes the inspection task after the delay processing time is spaced. Specifically, from the log file generated after the sending time, whether there is data matched with the source change data or data matched with the data converted by the conversion strategy is judged, so as to judge whether the change data actually obtained in the memory of the service application system is effective or not.
Accordingly, a specific example will be described. In this embodiment, the source data end may be referred to as an external system (OUTAPP), and the task execution client in the service application system to be changed may also be referred to as a BIZAPP. When the external system needs to send the change data to the memory of the business application system, the change data and the business application system (server) to be changed are determined, and the task release server is triggered to initiate the checking task. For example, the external system may send a data check request carrying the change data and the system identifier to the task publishing server. Because the data checking request has the current change data and the IP set of the service application system to be changed. After the task issuing server takes the data and the IP set, constructing a checking command script through the data, and forming a script command: execution (data). An IP set is a collection of IPs that tasks distribute to specific servers. The task execution client (client) of the server receives the script command: after execution (data), a series of checking operations such as specific collection data, checking data, statistical results and the like can be executed. It can be seen that the inspection process is introduced simultaneously in one change process, and the inspection is immediately and synchronously performed after the change.
Corresponding to the foregoing embodiments of the change data checking method, the present specification also provides embodiments of the change data checking apparatus and an electronic device to which the change data checking apparatus is applied.
The embodiment of the present specification modification data inspection apparatus may be applied to a computer device. The apparatus embodiments may be implemented by software, or may be implemented by hardware or a combination of hardware and software. Taking software implementation as an example, the device in a logic sense is formed by reading corresponding computer program instructions in a nonvolatile memory into a memory by a processor of a computer device where the device is located for operation. In terms of hardware, as shown in fig. 5, a hardware configuration diagram of a computer device where the change data checking device is located in the present specification is shown, and in addition to the processor 510, the network interface 520, the memory 530, and the nonvolatile memory 540 shown in fig. 5, the computer device where the change data checking device 531 is located in the embodiment generally includes other hardware according to the actual function of the device, which is not described herein again.
As shown in fig. 6, a block diagram of a change data checking device according to an exemplary embodiment of the present disclosure, where the device is disposed at a task issuing server, the device includes:
A request receiving module 62 for: receiving a data checking request sent by a source data end, wherein the data checking request comprises source change data sent to a target data end by the source data end and a system identifier of the target data end;
a task determination module 64 for: determining an inspection task, wherein the inspection task instructs a task execution client to check whether the actually obtained change data in the target data end is valid according to the source change data;
a task issuing module 66 for: and sending the checking task to a task execution client configured in the target data end indicated by the system identifier.
In one embodiment, the inspection task includes:
checking whether actually obtained change data in a target data end is matched with the source change data or not; or alternatively, the first and second heat exchangers may be,
checking whether actually obtained change data in the target data end is matched with data obtained by converting the source change data by using a conversion strategy, wherein the conversion strategy comprises conversion operation performed on the source change data from the source data end to the target data end.
In one embodiment, the actually obtained change data is obtained from a log file of the target data end, where the log file is generated by adopting a mode of printing a log when the data in the memory of the target data end is changed.
In one embodiment, the checking task carries a sending time of source change data sent by the active data end to the target data end, and the log file is generated after the sending time.
In one embodiment, the source data end sends source change data to a database corresponding to the target data end, the target data end loads the source change data from the database to the memory, and the data inspection request is a data inspection request sent to the task release server end by the source data end at intervals of preset delay time after the source change data is sent to the target data end; the preset delay time is determined based on the interval time of the target data end for loading data from the database to the memory.
In one embodiment, the source data end sends the source change data to a database corresponding to the target data end, the target data end loads the source change data from the database to the memory, the inspection task carries a delay processing time, the delay processing time is an interval time between a receiving time of the inspection task received by the task execution client and a starting time of executing the inspection task, and the delay processing time is determined based on an interval time of the target data end loading the data from the database to the memory.
As shown in fig. 7, there is shown a block diagram of another modification data checking apparatus according to an exemplary embodiment of the present specification, the apparatus being provided at a task execution client in a target data side, the apparatus including:
a task receiving module 72 for: receiving an inspection task sent by a task release server;
a task execution module 74 for: executing the checking task to check whether the actually obtained change data in the target data end is valid or not according to the source change data, wherein the source change data is the source change data sent to the target data end by the source data end;
a result feedback module 76 for: and feeding back the checking result to the task issuing server.
In one embodiment, the task execution module 74 is specifically configured to:
checking whether actually obtained change data in a target data end is matched with the source change data or not; or alternatively, the first and second heat exchangers may be,
checking whether actually obtained change data in the target data end is matched with data obtained by converting the source change data by using a conversion strategy, wherein the conversion strategy comprises conversion operation performed on the source change data from the source data end to the target data end.
For the device embodiments, reference is made to the description of the method embodiments for the relevant points, since they essentially correspond to the method embodiments. The apparatus embodiments described above are merely illustrative, wherein the modules illustrated as separate components may or may not be physically separate, and the components shown as modules may or may not be physical, i.e., may be located in one place, or may be distributed over a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purposes of the present description. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
Accordingly, the embodiments of the present disclosure further provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements any one of the above modified data checking methods when executing the program.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the apparatus embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments in part.
Accordingly, embodiments of the present disclosure also provide a computer storage medium having program instructions stored therein, the program instructions comprising:
receiving a data checking request sent by a source data end, wherein the data checking request comprises source change data sent to a target data end by the source data end and a system identifier of the target data end;
and determining an inspection task, and sending the inspection task to a task execution client configured in the target data end indicated by the system identifier, wherein the inspection task indicates whether the task execution client verifies whether the actually obtained change data in the target data end is valid or not according to the source change data.
Accordingly, embodiments of the present disclosure also provide another computer storage medium, in which program instructions are stored, the program instructions including:
receiving an inspection task sent by a task release server;
and executing the checking task to check whether the actually obtained change data in the target data end is valid or not according to the source change data, and feeding back a checking result to the task issuing server end, wherein the source change data is the source change data sent to the target data end by the source data end.
Embodiments of the present description may take the form of a computer program product embodied on one or more storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having program code embodied therein. Computer-usable storage media include both permanent and non-permanent, removable and non-removable media, and information storage may be implemented by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to: phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, may be used to store information that may be accessed by the computing device.
Other embodiments of the present description will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This specification is intended to cover any variations, uses, or adaptations of the specification following, in general, the principles of the specification and including such departures from the present disclosure as come within known or customary practice within the art to which the specification pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the specification being indicated by the following claims.
It is to be understood that the present description is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present description is limited only by the appended claims.
The foregoing description of the preferred embodiments is provided for the purpose of illustration only, and is not intended to limit the scope of the disclosure, since any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the disclosure are intended to be included within the scope of the disclosure.
Claims (14)
1. The system comprises a task release server and a task execution client configured in a target data end;
The task release server is used for:
receiving a data checking request sent by a source data end, wherein the data checking request comprises source change data sent to a target data end by the source data end and a system identifier of the target data end;
determining an inspection task, and sending the inspection task to a task execution client configured in a target data end indicated by the system identifier, wherein the inspection task indicates whether the task execution client verifies whether the actually obtained change data in the target data end is valid or not according to the source change data;
the verifying whether the actually obtained change data in the target data end is valid according to the source change data includes:
checking whether actually obtained change data in a target data end is matched with the source change data or not; or alternatively, the first and second heat exchangers may be,
checking whether actually obtained change data in a target data end is matched with data obtained by converting the source change data by using a conversion strategy, wherein the conversion strategy comprises conversion operation performed on the source change data from the source data end to the target data end;
the task execution client is used for executing the checking task.
2. A method for checking change data, the method being applied to a task release server, the method comprising:
Receiving a data checking request sent by a source data end, wherein the data checking request comprises source change data sent to a target data end by the source data end and a system identifier of the target data end;
determining an inspection task, and sending the inspection task to a task execution client configured in a target data end indicated by the system identifier, wherein the inspection task indicates whether the task execution client verifies whether the actually obtained change data in the target data end is valid or not according to the source change data;
the verifying whether the actually obtained change data in the target data end is valid according to the source change data includes:
checking whether actually obtained change data in a target data end is matched with the source change data or not; or alternatively, the first and second heat exchangers may be,
checking whether actually obtained change data in the target data end is matched with data obtained by converting the source change data by using a conversion strategy, wherein the conversion strategy comprises conversion operation performed on the source change data from the source data end to the target data end.
3. The method according to claim 2, wherein the actually obtained change data is obtained from a log file of the target data end, and the log file is generated by adopting a printing log mode when the data in the memory of the target data end is changed.
4. A method according to claim 3, wherein the checking task carries a sending time of source change data sent by the active data end to the target data end, and the log file is generated after the sending time.
5. The method according to any one of claims 2 to 4, wherein the source data end sends source change data to a database corresponding to a target data end, the target data end loads the source change data from the database to a memory, and the data inspection request is a data inspection request that the source data end sends the source change data to the target data end at intervals of a preset delay time and then sends the source change data to the task issuing server; the preset delay time is determined based on the interval time of the target data end for loading data from the database to the memory.
6. The method according to any one of claims 2 to 4, wherein the source data end sends source change data to a database corresponding to a target data end, the target data end loads the source change data from the database to the memory, the inspection task carries a delay processing time, the delay processing time is an interval time between a receiving time of the inspection task received by the task execution client and a starting time of executing the inspection task, and the delay processing time is determined based on an interval time of loading data from the database to the memory by the target data end.
7. A change data inspection method applied to a task execution client provided in a target data terminal, the method comprising:
receiving an inspection task sent by a task release server;
executing the checking task to check whether the actually obtained change data in the target data end is valid or not according to the source change data, wherein the source change data is the source change data sent to the target data end by the source data end; the verifying whether the actually obtained change data in the target data end is valid according to the source change data comprises the following steps:
checking whether actually obtained change data in a target data end is matched with the source change data or not; or alternatively, the first and second heat exchangers may be,
checking whether actually obtained change data in a target data end is matched with data obtained by converting the source change data by using a conversion strategy, wherein the conversion strategy comprises conversion operation performed on the source change data from the source data end to the target data end;
and feeding back the checking result to the task issuing server.
8. A change data inspection device, the device being provided at a task distribution server, the device comprising:
a request receiving module, configured to: receiving a data checking request sent by a source data end, wherein the data checking request comprises source change data sent to a target data end by the source data end and a system identifier of the target data end;
A task determination module for: determining an inspection task, wherein the inspection task instructs a task execution client to check whether the actually obtained change data in the target data end is valid according to the source change data;
the inspection task includes:
checking whether actually obtained change data in a target data end is matched with the source change data or not; or alternatively, the first and second heat exchangers may be,
checking whether actually obtained change data in a target data end is matched with data obtained by converting the source change data by using a conversion strategy, wherein the conversion strategy comprises conversion operation performed on the source change data from the source data end to the target data end;
the task issuing module is used for: and sending the checking task to a task execution client configured in the target data end indicated by the system identifier.
9. The apparatus of claim 8, wherein the actually obtained change data is obtained from a log file of the target data side, and the log file is generated by adopting a printing log mode when the data in the memory of the target data side is changed.
10. The apparatus of claim 9, wherein the checking task carries a sending time of source change data sent by the active data end to the target data end, and the log file is a log file generated after the sending time.
11. The apparatus according to any one of claims 8 to 10, wherein the source data end sends source change data to a database corresponding to a target data end, the target data end loads the source change data from the database to a memory, and the data inspection request is a data inspection request that the source data end sends the source change data to the target data end at intervals of a preset delay time after sending the source change data to the task issuing server; the preset delay time is determined based on the interval time of the target data end for loading data from the database to the memory.
12. The apparatus according to any one of claims 8 to 10, wherein the source data terminal sends source change data to a database corresponding to a target data terminal, the target data terminal loads the source change data from the database to the memory, the inspection task carries a delay processing time, the delay processing time is an interval time between a receiving time of the inspection task received by the task execution client and a starting time of executing the inspection task, and the delay processing time is determined based on an interval time of loading data from the database to the memory by the target data terminal.
13. A change data inspection apparatus provided at a task execution client in a target data terminal, the apparatus comprising:
A task receiving module, configured to: receiving an inspection task sent by a task release server;
the task execution module is used for: executing the checking task to check whether the actually obtained change data in the target data end is valid or not according to the source change data, wherein the source change data is the source change data sent to the target data end by the source data end;
the task execution module is specifically configured to:
checking whether actually obtained change data in a target data end is matched with the source change data or not; or alternatively, the first and second heat exchangers may be,
checking whether actually obtained change data in a target data end is matched with data obtained by converting the source change data by using a conversion strategy, wherein the conversion strategy comprises conversion operation performed on the source change data from the source data end to the target data end;
the result feedback module is used for: and feeding back the checking result to the task issuing server.
14. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any of claims 2 to 7 when the program is executed by the processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910044207.1A CN110032569B (en) | 2019-01-17 | 2019-01-17 | Method, device, equipment and system for checking changed data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910044207.1A CN110032569B (en) | 2019-01-17 | 2019-01-17 | Method, device, equipment and system for checking changed data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110032569A CN110032569A (en) | 2019-07-19 |
| CN110032569B true CN110032569B (en) | 2023-06-27 |
Family
ID=67235575
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910044207.1A Active CN110032569B (en) | 2019-01-17 | 2019-01-17 | Method, device, equipment and system for checking changed data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110032569B (en) |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2531537A (en) * | 2014-10-21 | 2016-04-27 | Ibm | Database Management system and method of operation |
| CN106294357B (en) * | 2015-05-14 | 2019-07-09 | 阿里巴巴集团控股有限公司 | Data processing method and stream calculation system |
| CN106095571B (en) * | 2016-06-07 | 2019-06-04 | 中国建设银行股份有限公司 | More RAC group systems, data access method and device |
| KR101904786B1 (en) * | 2017-03-06 | 2018-10-08 | 주식회사 데이터스트림즈 | Apparatus and method for replicating changed data in a source database management system to a target database management system in real time |
-
2019
- 2019-01-17 CN CN201910044207.1A patent/CN110032569B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110032569A (en) | 2019-07-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111177005B (en) | Service application testing method, device, server and storage medium | |
| CN111723313A (en) | Page jump processing method, device, equipment and storage medium | |
| CN107276970B (en) | Unbinding and binding method and device | |
| CN111460504B (en) | Service processing method, device, node equipment and storage medium | |
| CN109447820B (en) | Data processing method, device, computer equipment and storage medium | |
| AU2016373251A1 (en) | Method for performing inter-system service operation, service platform, and target system | |
| WO2023207146A1 (en) | Service simulation method and apparatus for esop system, and device and storage medium | |
| WO2019033741A1 (en) | Investment commodity resource processing method, device, storage medium and computer apparatus | |
| CN110955724A (en) | Data processing method and device based on block chain, node equipment and storage medium | |
| CN106952085B (en) | Method and device for data storage and service processing | |
| CN112866421A (en) | Intelligent contract operation method and device based on distributed cache and NSQ | |
| US9537940B2 (en) | Exchange of information between processing servers | |
| JPWO2016129028A1 (en) | Business linkage system and business linkage method | |
| CN112685391B (en) | Service data migration method and device, computer equipment and storage medium | |
| CN110032569B (en) | Method, device, equipment and system for checking changed data | |
| CN115826997A (en) | Version release method and device, electronic equipment and storage medium | |
| GB2582509A (en) | Error handling | |
| CN113393252A (en) | Online work order management method, device, equipment and storage medium | |
| CN113744050A (en) | Network loan approval method, device and system | |
| CN113485942A (en) | Automatic testing method and device based on independent module | |
| CN113282347A (en) | Plug-in operation method, device, equipment and storage medium | |
| KR101737575B1 (en) | Method and device for verifying data based on sql sentences generated automatically | |
| CN112613967B (en) | Business transaction data processing method and device, computer equipment and storage medium | |
| CN113849403A (en) | YAML configuration-based flow testing method, device and equipment | |
| CN114911522A (en) | Software version release system, method, device and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200923 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Innovative advanced technology Co.,Ltd. Address before: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant before: Advanced innovation technology Co.,Ltd. Effective date of registration: 20200923 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Advanced innovation technology Co.,Ltd. Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands Applicant before: Alibaba Group Holding Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |