CN113743842A - Multi-source data difference warning method and device - Google Patents
Multi-source data difference warning method and device Download PDFInfo
- Publication number
- CN113743842A CN113743842A CN202010468676.9A CN202010468676A CN113743842A CN 113743842 A CN113743842 A CN 113743842A CN 202010468676 A CN202010468676 A CN 202010468676A CN 113743842 A CN113743842 A CN 113743842A
- Authority
- CN
- China
- Prior art keywords
- source data
- comparison
- change
- data
- change message
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 230000008859 change Effects 0.000 claims abstract description 217
- 238000004590 computer program Methods 0.000 claims description 9
- 238000005516 engineering process Methods 0.000 claims description 6
- 238000010586 diagram Methods 0.000 description 16
- 238000012545 processing Methods 0.000 description 9
- 238000006243 chemical reaction Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 230000002045 lasting effect Effects 0.000 description 6
- 230000001960 triggered effect Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 239000003607 modifier Substances 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000006641 stabilisation Effects 0.000 description 1
- 238000011105 stabilization Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
- G06Q10/087—Inventory or stock management, e.g. order filling, procurement or balancing against orders
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Economics (AREA)
- Physics & Mathematics (AREA)
- Tourism & Hospitality (AREA)
- Development Economics (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Strategic Management (AREA)
- Entrepreneurship & Innovation (AREA)
- Human Resources & Organizations (AREA)
- General Business, Economics & Management (AREA)
- Finance (AREA)
- Accounting & Taxation (AREA)
- Computer Security & Cryptography (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a multi-source data difference warning method and device, and relates to the technical field of computers. One embodiment of the method comprises: receiving change messages sent by at least two source data changers, and storing each change message to a cache; wherein, the change message carries data change item and change time; according to a preset comparison rule, performing data difference comparison on each change message; and judging whether the comparison result has continuous data difference within a first preset time period, and if so, sending alarm information. The implementation method can solve the technical problem of occupying a large number of resources in the background.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a multi-source data difference warning method and device.
Background
In a large-scale software architecture, the same piece of data may not be consistent across different systems. For example, after an order is placed by an order system, inventory deduction is needed, and data difference may exist between the order system and the inventory system; for example, if the calling inventory is overtime, the user is returned to fail to place an order, and the actual inventory system is deducted; for example, due to system bugs, data inconsistency between systems is also caused, which is not the same as sales inventory and warehouse inventory, resulting in over-selling and failure to deliver goods.
For the problem of data inconsistency between systems, the existing solution is: after receiving the complaint of the user, the complaint is solved by inquiring the system log; or the background system continuously compares the total data among the systems, and intervenes after differences are found.
In the process of implementing the invention, the inventor finds that at least the following problems exist in the prior art:
influence the user experience; comparing the full amount of data may occupy a large amount of resources in the background, resulting in system performance being affected.
Disclosure of Invention
In view of this, embodiments of the present invention provide a multi-source data difference warning method and apparatus, so as to solve the technical problem of occupying a large amount of resources in a background.
To achieve the above object, according to an aspect of an embodiment of the present invention, there is provided a multi-source data difference warning method, including:
receiving change messages sent by at least two source data changers; wherein, the change message carries data change item and change time;
according to a preset comparison rule, performing data difference comparison on each change message;
and judging whether the comparison result has continuous data difference within a first preset time period, and if so, sending alarm information.
Optionally, receiving change messages sent by at least two source data changers, and storing each change message in a cache, includes:
receiving change messages sent by at least two source data changers;
converting each change message into a Map format through a Java reflection technology;
and storing each change message converted into the Map format into a cache.
Optionally, the change message further carries an item identifier and a receipt identifier;
according to a preset comparison rule, performing data difference comparison on each change message, including:
judging whether a second change message sent by a second source data changer and identical to the article identifier and the receipt identifier of the first change message exists in any first change message sent by a first source data changer;
if so, continuously judging whether the data change item carried by the first change message is the same as the data change item carried by the second change message; if not, marking the first change message and the second change message as difference; if so, marking the first change message and the second change message as the same;
if not, marking the first change message as a difference.
Optionally, the determining whether there is a data difference lasting within a first preset time period in the comparison result includes:
and judging whether the marks of the first change message are continuously different in a first preset time period.
Optionally, the method further comprises:
comparing the data of a plurality of source data changers every second preset time period;
if the data difference exists, generating a comparison task, writing the comparison task into a comparison task table, and configuring the stable times of the comparison task as 1; wherein the comparison task comprises a difference value between the item identification and the source data of each source data changer.
Optionally, the method further comprises:
and comparing each comparison task in the comparison task table every other third preset time period, and if the source data of each source data changer is continuously different, sending alarm information.
Optionally, comparing each comparison task in the comparison task table, and if the source data of each source data changer is continuously different, sending an alarm message, including:
for any comparison task in the comparison task table, judging whether the source data of each source data changer are the same or not based on the article identification of the comparison task;
if yes, deleting the comparison task from the comparison task table;
if not, continuously judging whether the current difference value between the source data of each source data changer is the same as the difference value of the comparison task; if yes, adding 1 to the stable times of the comparison task; if not, updating the current difference value into the comparison task, and configuring the stable times of the comparison task as 1;
and if the stable times of the comparison task exceed a preset time threshold, sending alarm information.
In addition, according to another aspect of the embodiments of the present invention, there is provided a multi-source data difference warning apparatus including:
the receiving module is used for receiving change messages sent by at least two source data changers; wherein, the change message carries data change item and change time;
the comparison module is used for comparing the data difference of each change message according to a preset comparison rule;
and the alarm module is used for judging whether the comparison result has continuous data difference within a first preset time period, and if so, sending alarm information.
Optionally, the receiving module is further configured to:
receiving change messages sent by at least two source data changers;
converting each change message into a Map format through a Java reflection technology;
and storing each change message converted into the Map format into a cache.
Optionally, the change message further carries an item identifier and a receipt identifier;
the comparison module is further configured to:
judging whether a second change message sent by a second source data changer and identical to the article identifier and the receipt identifier of the first change message exists in any first change message sent by a first source data changer;
if so, continuously judging whether the data change item carried by the first change message is the same as the data change item carried by the second change message; if not, marking the first change message and the second change message as difference; if so, marking the first change message and the second change message as the same;
if not, marking the first change message as a difference.
Optionally, the alarm module is further configured to:
and judging whether the marks of the first change message are continuously different in a first preset time period.
Optionally, the system further comprises a full quantity comparison module, configured to:
comparing the data of a plurality of source data changers every second preset time period;
if the data difference exists, generating a comparison task, writing the comparison task into a comparison task table, and configuring the stable times of the comparison task as 1; wherein the comparison task comprises a difference value between the item identification and the source data of each source data changer.
Optionally, the full-scale comparison module is further configured to:
and comparing each comparison task in the comparison task table every other third preset time period, and if the source data of each source data changer is continuously different, sending alarm information.
Optionally, the full-scale comparison module is further configured to:
for any comparison task in the comparison task table, judging whether the source data of each source data changer are the same or not based on the article identification of the comparison task;
if yes, deleting the comparison task from the comparison task table;
if not, continuously judging whether the current difference value between the source data of each source data changer is the same as the difference value of the comparison task; if yes, adding 1 to the stable times of the comparison task; if not, updating the current difference value into the comparison task, and configuring the stable times of the comparison task as 1;
and if the stable times of the comparison task exceed a preset time threshold, sending alarm information.
According to another aspect of the embodiments of the present invention, there is also provided an electronic device, including:
one or more processors;
a storage device for storing one or more programs,
when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any of the embodiments described above.
According to another aspect of the embodiments of the present invention, there is also provided a computer readable medium, on which a computer program is stored, which when executed by a processor implements the method of any of the above embodiments.
One embodiment of the above invention has the following advantages or benefits: because the technical means of receiving the change messages sent by at least two source data changers, comparing the data difference of each change message and sending out the alarm information if the continuous data difference exists is adopted, the technical problem that a large amount of background resources are occupied in the prior art is solved. The embodiment of the invention triggers the comparison of the difference data of the multi-party change message by receiving the change message sent by the source data changer with data change, if the difference is different, the follow-up is not needed, and only the current difference state needs to be recorded; because the data change is inevitably carried out again when the difference state is changed into the non-difference state, a new change message is triggered, and whether the difference exists can be judged by analyzing continuous local difference results, so that the alarm is carried out. Therefore, the embodiment of the invention monitors the data change condition of each source data changer in real time, does not influence the user experience, and also obviously reduces the occupied background resources.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of a main flow of a multi-source data difference warning method according to an embodiment of the present invention;
FIG. 2 is a schematic view of a main flow of a multi-source data difference warning method according to a referential embodiment of the present invention;
FIG. 3 is a flow chart illustrating a process for sending a change message according to one exemplary embodiment of the present invention;
FIG. 4 is a flow diagram of a conversion process according to one referenced embodiment of the present invention;
FIG. 5 is a schematic flow chart of data difference comparison according to a reference embodiment of the present invention;
FIG. 6 is a schematic flow chart of result analysis according to a reference embodiment of the present invention;
FIG. 7 is a schematic view of a main flow of a multi-source data difference warning method according to another referential embodiment of the present invention;
FIG. 8 is a diagram showing a main flow of a multi-source data difference warning method according to another referential embodiment of the present invention;
FIG. 9 is a schematic diagram of the main modules of a multi-source data difference warning device according to an embodiment of the present invention;
FIG. 10 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
fig. 11 is a schematic structural diagram of a computer system suitable for implementing a terminal device or a server according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
FIG. 1 is a schematic diagram of a main flow of a multi-source data difference warning method according to an embodiment of the present invention. As an embodiment of the present invention, as shown in fig. 1, the multi-source data difference warning method may include:
step 101, receiving change messages sent by at least two source data changers.
In embodiments of the present invention, the source data changer may be an order system, an inventory system, or the like. When the data stored by the source data changer is changed, a change message is sent, the alarm system of the embodiment of the invention receives the change message sent by the source data changer, and then the change message is stored in the cache. Wherein, the change message carries data change item and change time. Optionally, the change message further carries an item identifier and a document identifier, so as to perform data comparison on the data change item in the change message in the subsequent step. Optionally, the change message further carries an identifier of the source data changer, so as to perform data comparison on change messages sent by different source data changers in subsequent steps.
Alternatively, in another embodiment of the invention, the sending of the change messages may be implemented by a message middleware MQ (message notification system). For example, log change (changeLog) records of the order database and the inventory database are respectively subscribed, and when data is changed, the changeLog change is sent out by MQ messages, so that the alarm system executing the embodiment of the invention receives the MQ messages.
Optionally, step 101 may comprise: receiving change messages sent by at least two source data changers; converting each change message into a Map format through a Java reflection technology; and storing each change message converted into the Map format into a cache. In order to facilitate data comparison in the subsequent steps, the change message needs to be converted into standard data (i.e., data in Map format) that can be processed by the system, and then the converted change message needs to be stored in a cache (e.g., Redis).
And 102, comparing the data difference of each change message according to a preset comparison rule.
The comparison rules may be preconfigured, such as: whether the data change of the first source data changer is the same as the data change of the second source data changer, or whether the sum of the data change of the first source data changer and the data change of the second source data changer is equal to the data change of the third source data changer, and so on. According to the method and the device, data difference comparison is carried out on all the change messages stored in the cache in real time, so that the data difference can be found timely.
Optionally, step 102 may comprise:
judging whether a second change message sent by a second source data changer and identical to the article identifier and the receipt identifier of the first change message exists in any first change message sent by a first source data changer;
if so, continuously judging whether the data change item carried by the first change message is the same as the data change item carried by the second change message; if not, marking the first change message and the second change message as difference; if so, marking the first change message and the second change message as the same;
if not, marking the first change message as a difference.
After the data difference comparison is completed, the comparison result (the marking result of each change message) may be stored in the comparison database. Optionally, if the first change message and the second change message are both marked as the same, indicating that the data change item has no difference between the two source data changers, the first change message and the second change message in the cache may be deleted.
And 103, judging whether the comparison result has data difference lasting in the first preset time period, and if so, sending alarm information.
And judging whether a change message which is always different in a preset time period exists according to the comparison result obtained in the step 102, if so, indicating that only one source data changer has data change and other source data changers may not have data change or the changed data is inconsistent, and therefore, alarming is required to be sent out to prompt operation and maintenance personnel.
Optionally, the determining whether there is a data difference lasting within a first preset time period in the comparison result includes: and judging whether the marks of the first change message are continuously different in a first preset time period. If the marks of the first change messages are different all the time within the preset time period, the second change messages sent by other source data changers which are the same as the first change messages are not shown in the cache all the time, and the other source data changers are possibly not changed or the changed data are inconsistent.
The embodiment of the invention discovers continuous difference by analyzing the comparison result and then sends out the alarm information. If the difference is not continuous, the difference is initially obtained, and the subsequent difference is not obtained, the delay of the data difference structure is indicated, no difference exists, and no alarm information is sent. Therefore, the data difference comparison is only carried out on the change messages in the embodiment of the application, and the data stored by each changer does not need to be compared in a full amount, so that the occupied background resources are obviously reduced.
According to the various embodiments described above, it can be seen that the technical means of the embodiments of the present invention, by receiving the change messages sent by at least two source data changers, comparing the data differences of the change messages, and if there is a continuous data difference, sending an alarm message, solves the technical problem of occupying a large amount of background resources in the prior art. The embodiment of the invention triggers the comparison of the difference data of the multi-party change message by receiving the change message sent by the source data changer with data change, if the difference is different, the follow-up is not needed, and only the current difference state needs to be recorded; because the data change is inevitably carried out again when the difference state is changed into the non-difference state, a new change message is triggered, and whether the difference exists can be judged by analyzing continuous local difference results, so that the alarm is carried out. Therefore, the embodiment of the invention monitors the data change condition of each source data changer in real time, does not influence the user experience, and also obviously reduces the occupied background resources.
FIG. 2 is a diagram illustrating a main flow of a multi-source data difference warning method according to a referential embodiment of the present invention. As another embodiment of the present invention, as shown in fig. 2, the multi-source data difference warning method may include:
taking an order system (a first source data changer) and an inventory system (a second source data changer) as examples, first, the inventory middleware and the order middleware respectively subscribe the changeLog records of the order database and the inventory database, and when data is changed, the corresponding Middleware (MQ) sends out the changeLog change by MQ messages, as shown in fig. 3. It should be noted that if message middleware is not used, when data stored in the service system (order system, inventory system) is changed, the change message may also be sent directly by the service system. In order to send the change message, message middleware may be used or not be used, which is not limited in this embodiment of the present invention. Whether the message middleware or the service system is adopted, the change message is sent to the conversion processing module, and the conversion processing module carries out unified processing on the change message.
As shown in fig. 4, the conversion processing module receives change messages sent by the middleware corresponding to each system, and through the Java reflection technology, the inventory converter converts the change messages of the inventory system into a Map format, and the order converter converts the change messages of the order system into the Map format, and then stores each change message converted into the Map format to Redis. After the format conversion and storage of the change message are completed, the conversion processing module sends a comparison message to a downstream comparison executor.
As shown in fig. 5, after receiving the comparison message sent by the conversion processing module, the comparison executor performs data difference comparison on each change message stored in the Redis based on a preset comparison rule, marks a state (difference or same) for each change message, and then records a marking result of each change message in the comparison database.
As shown in fig. 6, the result analysis module obtains difference records, i.e. change messages marked as differences, from the comparison database, and performs result analysis on the difference records. And judging whether the data difference lasting in the first preset time period exists or not, if so, sending out alarm information, otherwise, judging that the data difference is normal and not sending out the alarm information.
In addition, in a referential embodiment of the present invention, the detailed implementation contents of the multi-source data difference warning method are already described in detail in the above multi-source data difference warning method, so that the repeated contents are not described again.
FIG. 7 is a diagram showing a main flow of a multi-source data difference warning method according to another referential embodiment of the present invention. As another embodiment of the present invention, as shown in fig. 7, the multi-source data difference warning method may include:
step 701, receiving change messages sent by at least two source data changers, and storing each change message to a cache; wherein, the change message carries data change item and change time.
And 702, comparing the data difference of each change message according to a preset comparison rule.
Step 703, determining whether there is a data difference lasting within a first preset time period in the comparison result, and if yes, sending an alarm message.
Steps 701-703 are similar to steps 101-103 and will not be described again.
And 704, comparing the data of the plurality of source data changers every second preset time period.
Data detail difference can be found through steps 701-703, but data total number difference cannot be found, so that the data stored by each source data changer is compared in full quantity every second preset time period. Optionally, the data stored by each source data modifier is obtained by calling a query interface, so as to perform full-scale comparison.
In addition to triggering the full-data comparison by setting a second preset time period, the full-data comparison may also be triggered by message middleware, and the full-data comparison is triggered once the data stored by the source data modifier changes. However, avoiding too frequent a comparison (e.g., frequent changes in data for some hot-pin items) may limit the frequency of comparison.
Step 705, if there is a data difference, a comparison task is generated, the comparison task is written into a comparison task table, and the number of times of stabilization of the comparison task is configured to be 1.
And if the data stored by the source data changers are different, generating a comparison task, wherein the comparison task comprises a difference value between the article identifier and the source data of each source data changer, and the difference value is a relative value. And then, writing the comparison task into a comparison task table, and configuring the stable times of the comparison task as 1.
Step 706, comparing each comparison task in the comparison task table every third preset time period, and if the source data of each source data changer is continuously different, sending an alarm message.
And comparing each comparison task in the comparison tasks in sequence every third preset time period through the timing task, and if the source data among the source data changers are different continuously, sending out alarm information.
The embodiment of the invention can not only carry out difference alarm on detailed data, but also carry out difference alarm on total data, thereby ensuring that the data change condition of each source data changer is accurately monitored, not influencing user experience, and also obviously reducing occupied background resources.
In addition, in another embodiment of the present invention, the detailed implementation of the multi-source data difference warning method is described in detail in the above-mentioned multi-source data difference warning method, so that the repeated description is not repeated here.
FIG. 8 is a diagram showing a main flow of a multi-source data difference warning method according to still another referential embodiment of the present invention. As still another embodiment of the present invention, as shown in fig. 7, steps 704-706 may specifically include the following steps:
and carrying out full-quantity comparison on the data stored by each source data changer every a second preset time period. Optionally, the data stored by each source data modifier is obtained by calling a query interface, so as to perform full-scale comparison.
And if the data stored by the source data changers are different, generating a comparison task, wherein the comparison task comprises a difference value between the article identifier and the source data of each source data changer, and the difference value is a relative value.
And writing the comparison task into a comparison task table, and configuring the stable times of the comparison task as 1.
And comparing each comparison task in the comparison tasks in sequence every third preset time period through the timing task.
Optionally, for any comparison task in the comparison task table, based on the article identifier of the comparison task, determining whether the source data of each source data changer is the same;
if yes (no difference), deleting the comparison task from the comparison task table;
if not (difference), continuously judging whether the current difference value between the source data of each source data changer is the same as the difference value of the comparison task; if yes (difference, and the difference is the same as the last time), adding 1 to the stable times of the comparison task; if not (the difference is different from the last time), updating the current difference value into the comparison task, and configuring the stable times of the comparison task as 1;
and if the stable times of the comparison task exceed a preset time threshold, sending alarm information.
In addition, in another reference embodiment of the present invention, the detailed implementation of the multi-source data difference warning method is described in detail in the above-mentioned multi-source data difference warning method, so that the repeated content will not be described again.
FIG. 9 is a schematic diagram of the main modules of a multi-source data difference warning device according to an embodiment of the present invention, and as shown in FIG. 9, the multi-source data difference warning device 900 includes a receiving module 901, a comparing module 902 and a warning module 903; the receiving module 901 is configured to receive change messages sent by at least two source data changers; wherein, the change message carries data change item and change time; the comparison module 902 is configured to perform data difference comparison on each change message according to a preset comparison rule; the alarm module 903 is configured to determine whether a data difference lasting within a first preset time period exists in the comparison result, and if yes, send an alarm message.
Optionally, the receiving module 901 is further configured to:
receiving change messages sent by at least two source data changers;
converting each change message into a Map format through a Java reflection technology;
and storing each change message converted into the Map format into a cache.
Optionally, the change message further carries an item identifier and a receipt identifier;
the comparison module 902 is further configured to:
judging whether a second change message sent by a second source data changer and identical to the article identifier and the receipt identifier of the first change message exists in any first change message sent by a first source data changer;
if so, continuously judging whether the data change item carried by the first change message is the same as the data change item carried by the second change message; if not, marking the first change message and the second change message as difference; if so, marking the first change message and the second change message as the same;
if not, marking the first change message as a difference.
Optionally, the alarm module 903 is further configured to:
and judging whether the marks of the first change message are continuously different in a first preset time period.
Optionally, the system further comprises a full quantity comparison module, configured to:
comparing the data of a plurality of source data changers every second preset time period;
if the data difference exists, generating a comparison task, writing the comparison task into a comparison task table, and configuring the stable times of the comparison task as 1; wherein the comparison task comprises a difference value between the item identification and the source data of each source data changer.
Optionally, the full-scale comparison module is further configured to:
and comparing each comparison task in the comparison task table every other third preset time period, and if the source data of each source data changer is continuously different, sending alarm information.
Optionally, the full-scale comparison module is further configured to:
for any comparison task in the comparison task table, judging whether the source data of each source data changer are the same or not based on the article identification of the comparison task;
if yes, deleting the comparison task from the comparison task table;
if not, continuously judging whether the current difference value between the source data of each source data changer is the same as the difference value of the comparison task; if yes, adding 1 to the stable times of the comparison task; if not, updating the current difference value into the comparison task, and configuring the stable times of the comparison task as 1;
and if the stable times of the comparison task exceed a preset time threshold, sending alarm information.
According to the various embodiments described above, it can be seen that the technical means of the embodiments of the present invention, by receiving the change messages sent by at least two source data changers, comparing the data differences of the change messages, and if there is a continuous data difference, sending an alarm message, solves the technical problem of occupying a large amount of background resources in the prior art. The embodiment of the invention triggers the comparison of the difference data of the multi-party change message by receiving the change message sent by the source data changer with data change, if the difference is different, the follow-up is not needed, and only the current difference state needs to be recorded; because the data change is inevitably carried out again when the difference state is changed into the non-difference state, a new change message is triggered, and whether the difference exists can be judged by analyzing continuous local difference results, so that the alarm is carried out. Therefore, the embodiment of the invention monitors the data change condition of each source data changer in real time, does not influence the user experience, and also obviously reduces the occupied background resources.
It should be noted that, in the implementation contents of the multi-source data difference warning apparatus of the present invention, the details of the above multi-source data difference warning method have been described in detail, and therefore, the repeated contents are not described again.
FIG. 10 illustrates an exemplary system architecture 1000 to which the multi-source data variance alerting method or the multi-source data variance alerting apparatus of embodiments of the present invention may be applied.
As shown in fig. 10, the system architecture 1000 may include source data changers 1001, 1002, a network 1004, and a server 1003. The network 1004 is used to provide a medium for communication links between the source data changers 1001, 1002 and the server 1003. Network 1004 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
A user may use source data changers 1001, 1002 to interact with a server 1003 over a network 1004 to receive or send messages or the like. The source data changers 1001, 1002 may be business systems such as inventory systems, order systems, and the like.
The server 1003 may be a server providing various services, the multi-source data difference warning method provided by the embodiment of the present invention is generally executed by the server 1003, and accordingly, the multi-source data difference warning apparatus is generally disposed in the server 1003.
It should be understood that the number of source data changers, networks, and servers in fig. 10 is merely illustrative. There may be any number of source data changers, networks, and servers, as desired for an implementation.
Referring now to FIG. 11, shown is a block diagram of a computer system 1100 suitable for use with a terminal device implementing an embodiment of the present invention. The terminal device shown in fig. 11 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 11, the computer system 1100 includes a Central Processing Unit (CPU)1101, which can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)1102 or a program loaded from a storage section 1108 into a Random Access Memory (RAM) 1103. In the RAM1103, various programs and data necessary for the operation of the system 1100 are also stored. The CPU 1101, ROM 1102, and RAM1103 are connected to each other by a bus 1104. An input/output (I/O) interface 1105 is also connected to bus 1104.
The following components are connected to the I/O interface 1105: an input portion 1106 including a keyboard, mouse, and the like; an output portion 1107 including a signal output unit such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and a speaker; a storage section 1108 including a hard disk and the like; and a communication section 1109 including a network interface card such as a LAN card, a modem, or the like. The communication section 1109 performs communication processing via a network such as the internet. A driver 1110 is also connected to the I/O interface 1105 as necessary. A removable medium 1111 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 1110 as necessary, so that a computer program read out therefrom is mounted into the storage section 1108 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication portion 1109 and/or installed from the removable medium 1111. The above-described functions defined in the system of the present invention are executed when the computer program is executed by a Central Processing Unit (CPU) 1101.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer programs according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes a receiving module, a comparing module, and an alerting module, where the names of the modules do not in some cases constitute a limitation on the module itself.
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, implement the method of: receiving change messages sent by at least two source data changers, and storing each change message to a cache; wherein, the change message carries data change item and change time; according to a preset comparison rule, performing data difference comparison on each change message; and judging whether the comparison result has continuous data difference within a first preset time period, and if so, sending alarm information.
According to the technical scheme of the embodiment of the invention, the technical means that the change messages sent by at least two source data changers are received, the data difference comparison is carried out on each change message, and if the continuous data difference exists, the alarm information is sent out is adopted, so that the technical problem that a large amount of background resources are occupied in the prior art is solved. The embodiment of the invention triggers the comparison of the difference data of the multi-party change message by receiving the change message sent by the source data changer with data change, if the difference is different, the follow-up is not needed, and only the current difference state needs to be recorded; because the data change is inevitably carried out again when the difference state is changed into the non-difference state, a new change message is triggered, and whether the difference exists can be judged by analyzing continuous local difference results, so that the alarm is carried out. Therefore, the embodiment of the invention monitors the data change condition of each source data changer in real time, does not influence the user experience, and also obviously reduces the occupied background resources.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A multi-source data difference alarm method is characterized by comprising the following steps:
receiving change messages sent by at least two source data changers; wherein, the change message carries data change item and change time;
according to a preset comparison rule, performing data difference comparison on each change message;
and judging whether the comparison result has continuous data difference within a first preset time period, and if so, sending alarm information.
2. The method of claim 1, wherein receiving change messages sent by at least two source data changers comprises:
receiving change messages sent by at least two source data changers;
converting each change message into a Map format through a Java reflection technology;
and storing each change message converted into the Map format into a cache.
3. The method of claim 1, wherein the change message further carries an item identifier and a document identifier;
according to a preset comparison rule, performing data difference comparison on each change message, including:
judging whether a second change message sent by a second source data changer and identical to the article identifier and the receipt identifier of the first change message exists in any first change message sent by a first source data changer;
if so, continuously judging whether the data change item carried by the first change message is the same as the data change item carried by the second change message; if not, marking the first change message and the second change message as difference; if so, marking the first change message and the second change message as the same;
if not, marking the first change message as a difference.
4. The method of claim 3, wherein determining whether there is a data difference in the comparison result that persists within a first preset time period comprises:
and judging whether the marks of the first change message are continuously different in a first preset time period.
5. The method of claim 1, further comprising:
comparing the data of a plurality of source data changers every second preset time period;
if the data difference exists, generating a comparison task, writing the comparison task into a comparison task table, and configuring the stable times of the comparison task as 1; wherein the comparison task comprises a difference value between the item identification and the source data of each source data changer.
6. The method of claim 5, further comprising:
and comparing each comparison task in the comparison task table every other third preset time period, and if the source data of each source data changer is continuously different, sending alarm information.
7. The method according to claim 5, wherein comparing each comparison task in the comparison task table, and if the source data of each source data changer is continuously different, sending an alarm message includes:
for any comparison task in the comparison task table, judging whether the source data of each source data changer are the same or not based on the article identification of the comparison task;
if yes, deleting the comparison task from the comparison task table;
if not, continuously judging whether the current difference value between the source data of each source data changer is the same as the difference value of the comparison task; if yes, adding 1 to the stable times of the comparison task; if not, updating the current difference value into the comparison task, and configuring the stable times of the comparison task as 1;
and if the stable times of the comparison task exceed a preset time threshold, sending alarm information.
8. A multi-source data difference warning device, comprising:
the receiving module is used for receiving change messages sent by at least two source data changers; wherein, the change message carries data change item and change time;
the comparison module is used for comparing the data difference of each change message according to a preset comparison rule;
and the alarm module is used for judging whether the comparison result has continuous data difference within a first preset time period, and if so, sending alarm information.
9. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
the one or more programs, when executed by the one or more processors, implement the method of any of claims 1-7.
10. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010468676.9A CN113743842A (en) | 2020-05-28 | 2020-05-28 | Multi-source data difference warning method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010468676.9A CN113743842A (en) | 2020-05-28 | 2020-05-28 | Multi-source data difference warning method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113743842A true CN113743842A (en) | 2021-12-03 |
Family
ID=78724211
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010468676.9A Pending CN113743842A (en) | 2020-05-28 | 2020-05-28 | Multi-source data difference warning method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113743842A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7035866B1 (en) * | 2001-10-09 | 2006-04-25 | Microsoft Corporation | System and method providing diffgram format |
| CN105243001A (en) * | 2014-07-07 | 2016-01-13 | 阿里巴巴集团控股有限公司 | Abnormal alarm method and apparatus for business object |
| CN105512853A (en) * | 2016-02-02 | 2016-04-20 | 成都市泰牛科技股份有限公司 | System and method based on commodity purchase and put-in-storage difference management |
| CN106056331A (en) * | 2016-05-27 | 2016-10-26 | 乐视控股（北京）有限公司 | Commodity inventory early warning method and early warning system |
| CN106447246A (en) * | 2015-08-06 | 2017-02-22 | 阿里巴巴集团控股有限公司 | Inventory data account checking method and inventory data account checking device |
| CN111126970A (en) * | 2020-01-02 | 2020-05-08 | 兰州嘉文电子科技有限公司 | Operation management system and method based on data consistency |
-
2020
- 2020-05-28 CN CN202010468676.9A patent/CN113743842A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7035866B1 (en) * | 2001-10-09 | 2006-04-25 | Microsoft Corporation | System and method providing diffgram format |
| CN105243001A (en) * | 2014-07-07 | 2016-01-13 | 阿里巴巴集团控股有限公司 | Abnormal alarm method and apparatus for business object |
| CN106447246A (en) * | 2015-08-06 | 2017-02-22 | 阿里巴巴集团控股有限公司 | Inventory data account checking method and inventory data account checking device |
| CN105512853A (en) * | 2016-02-02 | 2016-04-20 | 成都市泰牛科技股份有限公司 | System and method based on commodity purchase and put-in-storage difference management |
| CN106056331A (en) * | 2016-05-27 | 2016-10-26 | 乐视控股（北京）有限公司 | Commodity inventory early warning method and early warning system |
| CN111126970A (en) * | 2020-01-02 | 2020-05-08 | 兰州嘉文电子科技有限公司 | Operation management system and method based on data consistency |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107491488B (en) | Page data acquisition method and device | |
| CN111339186B (en) | Workflow engine data synchronization method, device, medium and electronic equipment | |
| CN110807067B (en) | Data synchronization method, device and equipment for relational database and data warehouse | |
| CN109257200B (en) | Method and device for monitoring big data platform | |
| CN111190888A (en) | Method and device for managing graph database cluster | |
| CN110532322B (en) | Operation and maintenance interaction method, system, computer readable storage medium and equipment | |
| US20200286014A1 (en) | Information updating method and device | |
| CN110727560A (en) | Cloud service alarm method and device | |
| CN111427701A (en) | Workflow engine system and business processing method | |
| CN113495820A (en) | Method and device for collecting and processing abnormal information and abnormal monitoring system | |
| CN116450622B (en) | Method, apparatus, device and computer readable medium for data warehouse entry | |
| CN111858621B (en) | Method, apparatus, device and computer readable medium for monitoring business process | |
| CN116737662A (en) | Method, device, electronic equipment and storage medium for processing business data | |
| CN113743842A (en) | Multi-source data difference warning method and device | |
| CN111813765B (en) | Method, device, electronic equipment and computer readable medium for processing abnormal data | |
| CN113076343A (en) | Data query method, device, equipment and storage medium | |
| CN113760698A (en) | Method and device for converting test case file data | |
| CN112559001A (en) | Method and device for updating application | |
| CN112214500A (en) | Data comparison method and device, electronic equipment and storage medium | |
| CN113779018A (en) | Data processing method and device | |
| CN113760695A (en) | Method and device for positioning problem code | |
| CN113722193A (en) | Method and device for detecting page abnormity | |
| CN113779048A (en) | Data processing method and device | |
| CN112749204A (en) | Method and device for reading data | |
| CN112783677A (en) | Method and device for monitoring service abnormity |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |