CN110188116B - Processing method and device for checking script, storage medium and electronic device - Google Patents
Processing method and device for checking script, storage medium and electronic device Download PDFInfo
- Publication number
- CN110188116B CN110188116B CN201910285898.4A CN201910285898A CN110188116B CN 110188116 B CN110188116 B CN 110188116B CN 201910285898 A CN201910285898 A CN 201910285898A CN 110188116 B CN110188116 B CN 110188116B
- Authority
- CN
- China
- Prior art keywords
- script
- data
- health score
- checking
- adjusting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000013515 script Methods 0.000 title claims abstract description 287
- 238000003672 processing method Methods 0.000 title abstract description 8
- 230000036541 health Effects 0.000 claims abstract description 176
- 238000000034 method Methods 0.000 claims abstract description 33
- 238000012545 processing Methods 0.000 claims description 35
- 230000002159 abnormal effect Effects 0.000 claims description 24
- 238000004364 calculation method Methods 0.000 claims description 20
- 238000004590 computer program Methods 0.000 claims description 18
- 238000013524 data verification Methods 0.000 claims description 11
- 238000012795 verification Methods 0.000 claims description 11
- 238000004458 analytical method Methods 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 5
- 230000008569 process Effects 0.000 description 8
- 230000005540 biological transmission Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000012546 transfer Methods 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000004140 cleaning Methods 0.000 description 1
- 230000009849 deactivation Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1734—Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention provides a processing method and device for checking a script, a storage medium and an electronic device, wherein the method comprises the following steps: determining a data checking script of a data circulation platform; calculating a health score of the data reconciliation script, wherein the health score is used for representing the reconciliation efficiency of the data reconciliation script; and when the health score is lower than a preset value, downloading the data checking script from the data circulation platform. The invention solves the technical problem of high false alarm rate of the data checking script in the related technology, reduces the noise and false alarm rate of the data circulation platform and saves system resources.
Description
Technical Field
The present invention relates to the field of computers, and in particular, to a method and an apparatus for processing a verification script, a storage medium, and an electronic apparatus.
Background
In the related technology, with the rapid development of the internet technology, the processing efficiency of various services is continuously improved, and great convenience is brought to the work and life of people. In the process of business processing, data circulation is usually involved. In order to avoid the user resource loss caused by abnormal data flow, a check operation is usually performed during the data flow. For example, in a bank, a third-party payment platform, or various platforms relating to article transactions, it is generally necessary to perform a checking operation for business processing such as account checking, account certificate checking, inter-system checking, and checking of each node between systems, thereby ensuring the accuracy of business flow or the like.
In the related art, the verification operation is usually implemented by compiling a corresponding verification script by a developer. However, in the related art, situations that the compiled check script is wrong or the compiled check script is not suitable for a new service scene often occur, so that a large amount of check error alarm information occurs, and the check error false alarm rate is greatly improved. Therefore, in the related art, it is very easy to cause the situation that the real error event in the checking process is covered, and the checking platform needs to maintain a large number of error checking rules and a large number of error alarm messages generated by the checking rules, thereby further causing the waste of platform resources and increasing the maintenance cost of the platform.
In view of the above problems in the related art, no effective solution has been found at present.
Disclosure of Invention
The embodiment of the invention provides a processing method and device for checking a script, a storage medium and an electronic device.
According to an embodiment of the present invention, there is provided a processing method of a collation script, including: determining a data checking script of a data circulation platform; calculating a health score of the data reconciliation script, wherein the health score is used for representing the reconciliation efficiency of the data reconciliation script; and when the health score is lower than a preset value, downloading the data checking script from the data circulation platform.
Optionally, calculating the health score of the data checking script includes: counting the execution failure amount of the data checking script in the period time; and adjusting the health score of the data checking script according to the execution failure amount.
Optionally, adjusting the health score of the data checking script according to the execution failure amount includes: judging whether the execution failure amount is larger than a first threshold value; when the execution failure amount is larger than the first threshold value, reducing the health score of the data checking script; when the execution failure amount is less than or equal to the first threshold, increasing or maintaining a health score of the data collation script.
Optionally, calculating the health score of the data checking script includes: counting the execution failure rate of the data checking script in the period time; and adjusting the health score of the data checking script according to the execution failure rate.
Optionally, adjusting the health score of the data checking script according to the execution failure rate includes: judging whether the execution failure rate is greater than a second threshold value; when the execution failure rate is greater than the second threshold, reducing a health score of the data checking script; when the execution failure rate is less than or equal to the second threshold, increasing or maintaining the health score of the data reconciliation script.
Optionally, calculating the health score of the data checking script includes: calculating the average time consumption of the data checking script for executing the checking task; and adjusting the health score of the data check script according to the average consumed time.
Optionally, adjusting the health score of the data verification script according to the average elapsed time includes: judging whether the average consumed time is larger than a third threshold value; when the average elapsed time is greater than the third threshold, reducing the health score of the data verification script; increasing or maintaining the health score of the data reconciliation script when the average elapsed time is less than or equal to the third threshold.
Optionally, calculating the health score of the data checking script includes: analyzing the rule integrity of the data checking script, wherein the rule integrity is used for representing the similarity of the data checking script and a preset standard script; and adjusting the health score of the data checking script according to the rule integrity.
Optionally, calculating the health score of the data checking script includes: counting the abnormal processing efficiency of the data checking script, wherein the abnormal processing efficiency is the feedback rate or the feedback duration of the abnormal data generated by the data checking script; and adjusting the health score of the data checking script according to the abnormal processing failure.
Optionally, calculating the health score of the data checking script includes: acquiring the signal-to-noise ratio of the data checking script to execute the checking task; and adjusting the health score of the data checking script according to the signal-to-noise ratio.
Optionally, calculating the health score of the data checking script includes: before the data checking script is on-line, adjusting the health score of the data checking script by adopting a first strategy; when the data checking script executes the checking task on line, adjusting the health score of the data checking script by adopting a second strategy; and when the data checking script does not execute the checking task on line, adjusting the health score of the data checking script by adopting a third strategy.
According to another embodiment of the present invention, there is provided a processing apparatus for collating a script, including: the determining module is used for determining a data checking script of the data circulation platform; the calculation module is used for calculating the health score of the data verification script, wherein the health score is used for representing the verification efficiency of the data verification script; and the offline module is used for offline the data checking script on the data circulation platform when the health score is lower than a preset value.
Optionally, the calculation module includes: a first statistical unit for counting an execution failure amount of the data collation script within a cycle time; and the first adjusting unit is used for adjusting the health score of the data checking script according to the execution failure amount.
Optionally, the calculation module includes: the second statistical unit is used for counting the execution failure rate of the data checking script in the period time; and the second adjusting unit is used for adjusting the health score of the data checking script according to the execution failure rate.
Optionally, the calculation module includes: a calculating unit, configured to calculate an average consumed time for the data collation script to perform the collation task; and the third adjusting unit is used for adjusting the health score of the data checking script according to the average consumed time.
Optionally, the calculation module includes: the analysis unit is used for analyzing the rule integrity of the data checking script, wherein the rule integrity is used for representing the similarity of the data checking script and a preset standard script; and the fourth adjusting unit is used for adjusting the health score of the data checking script according to the rule integrity.
Optionally, the calculation module includes: a third statistical unit, configured to count an exception handling efficiency of the data checking script, where the exception handling efficiency is a feedback rate or a feedback duration of exception data generated by the data checking script; and the fifth adjusting unit is used for adjusting the health score of the data check script according to the abnormal processing failure.
Optionally, calculating the health score of the data checking script includes: the acquisition unit is used for acquiring the signal to noise ratio of the data checking script to execute the checking task; and the sixth adjusting unit is used for adjusting the health score of the data check script according to the signal-to-noise ratio.
Optionally, the calculation module includes: a seventh adjusting unit, configured to adjust a health score of the data checking script by using a first policy before the data checking script is online; the eighth adjusting unit is used for adjusting the health score of the data checking script by adopting a second strategy when the data checking script executes the checking task on line; and when the data checking script does not execute the checking task on line, adjusting the health score of the data checking script by adopting a third strategy.
According to a further embodiment of the present invention, there is also provided a storage medium having a computer program stored therein, wherein the computer program is arranged to perform the steps of any of the above method embodiments when executed.
According to yet another embodiment of the present invention, there is also provided an electronic device, including a memory in which a computer program is stored and a processor configured to execute the computer program to perform the steps in any of the above method embodiments.
According to the invention, the health score of the data checking script is calculated, and when the health score is lower than a preset value, the data checking script is downloaded from the data circulation platform, and the health score of the data checking script is adjusted and the script with low downloading efficiency is downloaded, so that the technical problem of high false alarm rate of the data checking script in the related technology is solved, the noise and the false alarm rate of the data circulation platform are reduced, and the system resources are saved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
FIG. 1 is a block diagram of a processing server for reconciling scripts in accordance with an embodiment of the present invention;
FIG. 2 is a flow diagram of a method of processing a reconciliation script according to an embodiment of the present invention;
FIG. 3 is a flow chart of adjusting a health score according to an embodiment of the present invention;
fig. 4 is a block diagram of a processing apparatus for checking a script according to an embodiment of the present invention.
Detailed Description
The invention will be described in detail hereinafter with reference to the accompanying drawings in conjunction with embodiments. It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
Example 1
The method provided by the first embodiment of the present application may be executed in a mobile terminal, a computer terminal, a server, or a similar computing device. Taking the example of running on a server, fig. 1 is a block diagram of a processing server for checking a script according to an embodiment of the present invention. As shown in fig. 1, the server 10 may include one or more (only one shown in fig. 1) processors 102 (the processors 102 may include, but are not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, and optionally may also include a transmission device 106 for communication functions and an input-output device 108. It will be understood by those skilled in the art that the structure shown in fig. 1 is only an illustration, and is not intended to limit the structure of the server. For example, the server 10 may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.
The memory 104 may be used to store a computer program, for example, a software program and a module of application software, such as a computer program corresponding to a processing method of checking a script in the embodiment of the present invention, and the processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, so as to implement the above-mentioned method. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 104 may further include memory located remotely from processor 102, which may be connected to server 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 106 is used for receiving or transmitting data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the server 10. In one example, the transmission device 106 includes a Network adapter (NIC), which can be connected to other Network devices through a base station so as to communicate with the internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.
In this embodiment, a processing method for checking a script is provided, and is applied to a server side, fig. 2 is a flowchart of a processing method for checking a script according to an embodiment of the present invention, and as shown in fig. 2, the flowchart includes the following steps:
step S202, determining a data checking script of the data circulation platform;
in the embodiment, the data transfer platform is an online platform for data transfer or data transfer, and when data is transferred, to ensure the security or accuracy of the data, the data to be transferred needs to be checked, including account and account checking, account and certificate checking, account and password checking, checking between systems, and checking peer of each node between systems, so as to execute a checking operation by using a data checking script running on the data transfer platform;
step S204, calculating a health score of the data checking script, wherein the health score is used for representing the checking efficiency of the data checking script; the health score of the embodiment is an evaluation value of the checking efficiency of the data checking script, and the health score of each data checking script in the data circulation platform is updated in real time by evaluating the failure rate, the feedback rate and other coefficients of the statistical data checking script;
and step S206, when the health score is lower than a preset value, offline data checking scripts are executed on the data circulation platform.
Through the steps, the health score of the data checking script is calculated, the script is checked on the offline data of the data circulation platform when the health score is lower than a preset value, the health score of the script is checked through adjusting the data, the script with low offline efficiency is offline, the technical problem that the false alarm rate of the script is high in the data checking script in the related technology is solved, the noise and the false alarm rate of the data circulation platform are reduced, and system resources are saved.
Optionally, when the health score is higher than or equal to the preset value, the data checking script continues to run on the data streaming platform, and the priority can be further set according to the health scores of the various data checking scripts, and the data checking script is selected according to the priority sequence.
In this embodiment, when calculating the health score of the data collation script, the method includes a plurality of calculation policies, each calculation policy may be operated individually or may be operated in combination, and when operating in combination, the health score of the same data collation script is calculated sequentially for the same data collation script according to the types of the calculation policies, or the health score of the data collation script of the corresponding type is calculated for each type of calculation policy for a plurality of data collation scripts.
In one embodiment of this embodiment, calculating the health score of the data collation script comprises:
s11, counting the execution failure amount of the data checking script in the period time;
when the data flow platform executes the checking task, if the execution fails, the data checking script of the embodiment feeds back the task failure, the execution failure amount is the total failure amount of the data checking script in executing the checking task, and if the execution failure amount in the unit period is too large, the efficiency is limited.
S12, the health score of the script is checked according to the execution failure amount adjustment data. Adjusting the health score of the data collation script according to the execution failure amount includes: judging whether the execution failure amount is larger than a first threshold value; when the execution failure amount is larger than a first threshold value, reducing the health score of the data checking script; when the execution failure amount is less than or equal to a first threshold, the health score of the data collation script is increased or maintained.
The health score of this embodiment may be set to a range, such as 0-10, where 10 indicates very healthy and the efficiency is the highest, and conversely, 0 may be set to the maximum (10) or an intermediate value (e.g., 5), and the initial health score of each data check script may be adjusted up or down in the subsequent adjustment process, and when the health score is already the maximum value but needs to be adjusted up, the maximum value is directly maintained.
In another implementation of this embodiment, calculating the health score of the data collation script includes: counting the execution failure rate of the data checking script in the period time; specifically, the health score of the data check script is adjusted according to the execution failure rate. Adjusting the health score of the data collation script based on the execution failure rate includes: judging whether the execution failure rate is greater than a second threshold value; when the execution failure rate is larger than a second threshold value, reducing the health score of the data checking script; when the execution failure rate is less than or equal to a second threshold, the health score of the data collation script is increased or maintained. The execution failure rate of the present embodiment is similar to the execution failure amount, except that the execution failure rate is a percentage of the failure amount to the total.
In another implementation of this embodiment, calculating the health score of the data collation script includes: calculating the average time consumption of the data checking script for executing the checking task; and checking the health score of the script according to the average time consumption adjusting data. Specifically, checking the health score of the script according to the average elapsed time adjustment data includes: judging whether the average consumed time is larger than a third threshold value; when the average consumed time is larger than a third threshold value, reducing the health score of the data checking script; when the average elapsed time is less than or equal to a third threshold, the health score of the data collation script is increased or maintained.
In one example, when data collation script a and data collation script B respectively execute 1200 and 2400 collation tasks of the same type in a day, the average time consumed for data collation script a to process a single collation task is 0.02 and 0.01h, and thus it can be known that the average time consumed for data collation script a is longer, and for example, when the third threshold is 0.015h, the health score of data collation script a needs to be increased or maintained, and the health score of data collation script B needs to be decreased.
In one embodiment of this embodiment, calculating the health score of the data collation script comprises: analyzing the rule integrity of the data checking script, wherein the rule integrity is used for representing the similarity of the data checking script and a preset standard script; and adjusting the health score of the data checking script according to the rule integrity.
The data checking script of the embodiment comprises a checking algorithm, a feedback algorithm and the like, wherein each algorithm further comprises an input parameter, an output parameter, a threshold value, an interactive object and the like, and for the reason of research and development personnel, bug may be left when the data checking script is edited, and when the data checking script is incomplete relative to a preset standard script, the working efficiency of the data checking script is directly influenced. Specifically, if the rule integrity is greater than the fourth threshold and the data check script does not lack specified parameters, the health score of the data check script is increased or maintained; and when the rule integrity is less than or equal to a fourth threshold value, or the data checking script lacks specified parameters, reducing the health score of the data checking script.
In one embodiment of this embodiment, calculating the health score of the data collation script comprises: counting the abnormal processing efficiency of the data checking script, wherein the abnormal processing efficiency is the feedback rate or the feedback duration of the abnormal data generated by the data checking script; and checking the health score of the script according to the abnormal processing failure adjustment data. The efficiency in exception handling needs to be determined according to the processing time or the processing rate of the exception data of a feedback person when the exception data is produced after the check tasks are processed by the data check script, for example, one data check script processes 100 check tasks in 1h, wherein 90 processes are successful, 10 processes are failed and generate exception data, and for 10 exception data, 5 of the exception data are processed by a user, the average time is 2h, and the rest 5 exception data are not processed and fed back in the feedback period, so that the feedback rate of the data check script is 50%, and the feedback time is 2 h.
In one embodiment of this embodiment, calculating the health score of the data collation script comprises: acquiring the signal-to-noise ratio of a data checking script executing a checking task; and checking the health score of the script according to the signal-to-noise ratio adjustment data. The signal-to-noise ratio is the ratio of normal signals and abnormal signals generated by the data checking script when the checking task is executed, and the abnormal signals have no regularity, which may cause disorder to the system and affect the normal signals, so the efficiency of the data checking script with high signal-to-noise ratio is also low.
In this embodiment, the policy for adjusting the health score of the data check script includes multiple policies, which are triggered and executed at different stages, such as before online, when the script works online, when the script is idle online, and the like. Calculating the health score of the data reconciliation script comprises: before the data checking script is on line, adjusting the health score of the data checking script by adopting a first strategy; when the data checking script executes the checking task on line, a second strategy is adopted to adjust the health score of the data checking script; and when the data checking script does not execute the checking task on the line, adjusting the health score of the data checking script by adopting a third strategy.
In an implementation manner of this embodiment, the method includes a plurality of adjustment policies, each of which adjusts the health score of the data check script at different stages, fig. 3 is a flowchart of adjusting the health score according to an embodiment of the present invention, and as shown in fig. 3, the flowchart includes:
step S110, configuring corresponding health score for at least one check rule (data check script). The initial value of the health score corresponding to each collation rule may be the same, and is, for example, 10.
And step S120, adjusting the health score corresponding to the check rule according to a preset health score adjusting strategy. Specifically, when the check rule runs, an operation log is collected, and the health score corresponding to the check rule is adjusted according to log information and a preset health score adjusting strategy. The preset health score adjusting strategy comprises the following steps:
the strategy 1 is to count the failure quantity of the execution of the check rule in a preset time period (which can be lmin), wherein if the failure quantity is greater than a certain threshold value, the health score corresponding to the check rule is reduced by 1; because the record of the check execution failure falls into the database, if the failure rate is not high, but the failure amount is large, pressure is applied to the database of the platform;
strategy 2, counting the failure rate of a preset time period (which can be lmin); if the failure rate is greater than a certain threshold value, subtracting 1 from the health score corresponding to the checking rule; if the checking quantity is not high, but almost all the checking fails, the checking rule has no meaning;
strategy 3, counting the average consumed time of a preset time period (which can be lmin), wherein if the average consumed time is greater than a certain threshold value, the health score corresponding to the checking rule is reduced by 1; this is because the long time consumption will occupy the platform resource and cause the low resource utilization;
strategy 4, checking the information completeness of the rule, namely subtracting 1 from the health score corresponding to the checking rule if the information completeness is less than a certain threshold value; for example, if the verification rule is not matched with the alarm to whom the verification fails, the verification failure is probably not concerned by people;
strategy 5, checking abnormal processing timeliness, namely, if the processing timeliness is less than a certain threshold value, subtracting 1 from the health score corresponding to the checking rule; when the checking is abnormal, people usually need to check and feed back, if the checking is abnormal, the feedback is not carried out on time all the time, the checking rule is not considered by the people;
and 6, checking abnormal noise rate, namely checking high interference of abnormal noise, if noise is not reduced all the time, indicating that the rule is not important, and reducing the health score by 1 by cleaning abnormal data in a fixed period (such as 1 day) when the noise is more than 50%.
In the execution process, the rule is checked for each piece of data, before the rule is on-line, whether the health score needs to be reduced or not is firstly checked according to a policy 4, and if the health score does not meet the policy, the current health score is 10-1. During the online operation of the rules, whether the scores are to be subtracted is determined regularly according to the strategies 1, 2 and 3, the score is subtracted by 1 if 1 strategy is hit, and the score is subtracted by 3 if 3 strategies are hit. During the on-line idle period of the rule, data fed back to check abnormity is periodically cleaned to judge whether the strategies 5 and 6 are hit or not, and if the strategies are hit, the scores are deducted.
And S130, performing offline processing on the check rule with the health score smaller than a preset threshold value. For example, the preset threshold is 5 points, and the offline processing is specifically fusing deactivation processing, that is, disabling the data checking rule with a low score.
In the embodiment, the health scores are set for the check rules, and the health score corresponding to each check rule is dynamically adjusted by using the corresponding health score adjusting strategy, so that the check rules with wrong compilation and poor quality can be automatically offline, the overall check noise and the false alarm rate are reduced, and the system resources are saved.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
Example 2
In this embodiment, a processing device for checking a script is further provided, which is used to implement the foregoing embodiments and preferred embodiments, and the description that has been already made is omitted. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
An embodiment provides a processing apparatus for checking a script, and fig. 4 is a block diagram of a configuration of the processing apparatus for checking a script according to an embodiment of the present invention, the apparatus including: a determination module 40, a calculation module 42, a downline module 44, wherein,
a determining module 40, configured to determine a data checking script of the data streaming platform;
a calculating module 42, configured to calculate a health score of the data reconciliation script, where the health score is used to characterize a reconciliation efficiency of the data reconciliation script;
and the offline module 44 is configured to offline the data checking script on the data transfer platform when the health score is lower than a preset value.
Optionally, the calculation module includes: a first statistical unit for counting an execution failure amount of the data collation script within a cycle time; and the first adjusting unit is used for adjusting the health score of the data checking script according to the execution failure amount.
Optionally, the calculation module includes: the second statistical unit is used for counting the execution failure rate of the data checking script in the period time; and the second adjusting unit is used for adjusting the health score of the data checking script according to the execution failure rate.
Optionally, the calculation module includes: a calculating unit, configured to calculate an average consumed time for the data collation script to perform the collation task; and the third adjusting unit is used for adjusting the health score of the data checking script according to the average consumed time.
Optionally, the calculation module includes: the analysis unit is used for analyzing the rule integrity of the data checking script, wherein the rule integrity is used for representing the similarity of the data checking script and a preset standard script; and the fourth adjusting unit is used for adjusting the health score of the data checking script according to the rule integrity.
Optionally, the calculation module includes: a third statistical unit, configured to count an exception handling efficiency of the data checking script, where the exception handling efficiency is a feedback rate or a feedback duration of exception data generated by the data checking script; and the fifth adjusting unit is used for adjusting the health score of the data check script according to the abnormal processing failure.
Optionally, calculating the health score of the data checking script includes: the acquisition unit is used for acquiring the signal to noise ratio of the data checking script to execute the checking task; and the sixth adjusting unit is used for adjusting the health score of the data check script according to the signal-to-noise ratio.
Optionally, the calculation module includes: a seventh adjusting unit, configured to adjust a health score of the data checking script by using a first policy before the data checking script is online; the eighth adjusting unit is used for adjusting the health score of the data checking script by adopting a second strategy when the data checking script executes the checking task on line; and when the data checking script does not execute the checking task on line, adjusting the health score of the data checking script by adopting a third strategy.
It should be noted that, the above modules may be implemented by software or hardware, and for the latter, the following may be implemented, but not limited to: the modules are all positioned in the same processor; alternatively, the modules are respectively located in different processors in any combination.
Example 3
Embodiments of the present invention also provide a storage medium having a computer program stored therein, wherein the computer program is arranged to perform the steps of any of the above method embodiments when executed.
Alternatively, in an aspect of the present embodiment, the storage medium may be configured to store a computer program for executing the steps of:
s1, determining a data checking script of the data circulation platform;
s2, calculating the health score of the data verification script, wherein the health score is used for representing the verification efficiency of the data verification script;
and S3, when the health score is lower than a preset value, downloading the data checking script from the data circulation platform.
Optionally, in this embodiment, the storage medium may include, but is not limited to: various media capable of storing computer programs, such as a usb disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk.
Embodiments of the present invention also provide an electronic device comprising a memory having a computer program stored therein and a processor arranged to run the computer program to perform the steps of any of the above method embodiments.
Optionally, the electronic apparatus may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.
Optionally, in an aspect of this embodiment, the processor may be configured to execute the following steps by a computer program:
s1, determining a data checking script of the data circulation platform;
s2, calculating the health score of the data verification script, wherein the health score is used for representing the verification efficiency of the data verification script;
and S3, when the health score is lower than a preset value, downloading the data checking script from the data circulation platform.
Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and this embodiment is not described herein again.
It will be apparent to those skilled in the art that the modules or steps of the present invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and alternatively, they may be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described may be performed in an order different than that described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the principle of the present invention should be included in the protection scope of the present invention.
Claims (19)
1. A method for processing a verification script, comprising:
determining a data checking script of a data circulation platform;
calculating a health score of the data reconciliation script, wherein the health score is used for representing the reconciliation efficiency of the data reconciliation script;
when the health score is lower than a preset value, the data checking script is downloaded from the data circulation platform;
wherein calculating the health score of the data collation script comprises:
before the data checking script is on-line, adjusting the health score of the data checking script by adopting a first strategy;
when the data checking script executes the checking task on line, adjusting the health score of the data checking script by adopting a second strategy; and when the data checking script does not execute the checking task on line, adjusting the health score of the data checking script by adopting a third strategy.
2. The method of claim 1, wherein calculating the health score of the data collation script comprises:
counting the execution failure amount of the data checking script in the period time;
and adjusting the health score of the data checking script according to the execution failure amount.
3. The method of claim 2, wherein adjusting the health score of the data collation script in accordance with the amount of execution failure comprises:
judging whether the execution failure amount is larger than a first threshold value;
when the execution failure amount is larger than the first threshold value, reducing the health score of the data checking script; when the execution failure amount is less than or equal to the first threshold, increasing or maintaining a health score of the data collation script.
4. The method of claim 1, wherein calculating the health score of the data collation script comprises:
counting the execution failure rate of the data checking script in the period time;
and adjusting the health score of the data checking script according to the execution failure rate.
5. The method of claim 4, wherein adjusting the health score of the data collation script in accordance with the execution failure rate comprises:
judging whether the execution failure rate is greater than a second threshold value;
when the execution failure rate is greater than the second threshold, reducing a health score of the data checking script; when the execution failure rate is less than or equal to the second threshold, increasing or maintaining the health score of the data reconciliation script.
6. The method of claim 1, wherein calculating the health score of the data collation script comprises:
calculating the average time consumption of the data checking script for executing the checking task;
and adjusting the health score of the data check script according to the average consumed time.
7. The method of claim 6, wherein adjusting the health score of the data collation script based on the average elapsed time comprises:
judging whether the average consumed time is larger than a third threshold value;
when the average elapsed time is greater than the third threshold, reducing the health score of the data verification script; increasing or maintaining the health score of the data reconciliation script when the average elapsed time is less than or equal to the third threshold.
8. The method of claim 1, wherein calculating the health score of the data collation script comprises:
analyzing the rule integrity of the data checking script, wherein the rule integrity is used for representing the similarity of the data checking script and a preset standard script;
and adjusting the health score of the data checking script according to the rule integrity.
9. The method of claim 1, wherein calculating the health score of the data collation script comprises:
counting the abnormal processing efficiency of the data checking script, wherein the abnormal processing efficiency is the feedback rate or the feedback duration of the abnormal data generated by the data checking script;
and adjusting the health score of the data checking script according to the abnormal processing failure.
10. The method of claim 1, wherein calculating the health score of the data collation script comprises:
acquiring the signal-to-noise ratio of the data checking script to execute the checking task;
and adjusting the health score of the data checking script according to the signal-to-noise ratio.
11. A processing apparatus for collating a script, comprising:
the determining module is used for determining a data checking script of the data circulation platform;
the calculation module is used for calculating the health score of the data verification script, wherein the health score is used for representing the verification efficiency of the data verification script;
the offline module is used for offline the data checking script on the data circulation platform when the health score is lower than a preset value;
wherein the calculation module comprises: a seventh adjusting unit, configured to adjust a health score of the data checking script by using a first policy before the data checking script is online;
the eighth adjusting unit is used for adjusting the health score of the data checking script by adopting a second strategy when the data checking script executes the checking task on line; and when the data checking script does not execute the checking task on line, adjusting the health score of the data checking script by adopting a third strategy.
12. The apparatus of claim 11, wherein the computing module comprises:
a first statistical unit for counting an execution failure amount of the data collation script within a cycle time;
and the first adjusting unit is used for adjusting the health score of the data checking script according to the execution failure amount.
13. The apparatus of claim 11, wherein the computing module comprises:
the second statistical unit is used for counting the execution failure rate of the data checking script in the period time;
and the second adjusting unit is used for adjusting the health score of the data checking script according to the execution failure rate.
14. The apparatus of claim 11, wherein the computing module comprises:
a calculating unit, configured to calculate an average consumed time for the data collation script to perform the collation task;
and the third adjusting unit is used for adjusting the health score of the data checking script according to the average consumed time.
15. The apparatus of claim 11, wherein the computing module comprises:
the analysis unit is used for analyzing the rule integrity of the data checking script, wherein the rule integrity is used for representing the similarity of the data checking script and a preset standard script;
and the fourth adjusting unit is used for adjusting the health score of the data checking script according to the rule integrity.
16. The apparatus of claim 11, wherein the computing module comprises:
a third statistical unit, configured to count an exception handling efficiency of the data checking script, where the exception handling efficiency is a feedback rate or a feedback duration of exception data generated by the data checking script;
and the fifth adjusting unit is used for adjusting the health score of the data check script according to the abnormal processing failure.
17. The apparatus of claim 11, wherein calculating the health score of the data collation script comprises:
the acquisition unit is used for acquiring the signal to noise ratio of the data checking script to execute the checking task;
and the sixth adjusting unit is used for adjusting the health score of the data check script according to the signal-to-noise ratio.
18. A storage medium, in which a computer program is stored, wherein the computer program is arranged to perform the method of any of claims 1 to 10 when executed.
19. An electronic device comprising a memory and a processor, wherein the memory has stored therein a computer program, and wherein the processor is arranged to execute the computer program to perform the method of any of claims 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910285898.4A CN110188116B (en) | 2019-04-10 | 2019-04-10 | Processing method and device for checking script, storage medium and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910285898.4A CN110188116B (en) | 2019-04-10 | 2019-04-10 | Processing method and device for checking script, storage medium and electronic device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110188116A CN110188116A (en) | 2019-08-30 |
| CN110188116B true CN110188116B (en) | 2020-11-10 |
Family
ID=67714090
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910285898.4A Expired - Fee Related CN110188116B (en) | 2019-04-10 | 2019-04-10 | Processing method and device for checking script, storage medium and electronic device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110188116B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101370008A (en) * | 2007-08-13 | 2009-02-18 | 杭州安恒信息技术有限公司 | System for real-time intrusion detection of SQL injection WEB attacks |
| CN103902442A (en) * | 2012-12-25 | 2014-07-02 | 中国移动通信集团公司 | Method and system for evaluating cloud software health degree |
| CN105786682A (en) * | 2016-02-29 | 2016-07-20 | 上海新炬网络信息技术有限公司 | Implementation system and method for avoiding software performance failure |
| CN106649552A (en) * | 2016-11-07 | 2017-05-10 | 湖北省农村信用社联合社网络信息中心 | Normalized data cleaning method |
| CN107908532A (en) * | 2017-05-05 | 2018-04-13 | 平安科技（深圳）有限公司 | A kind of database performance diagnostic method, device and equipment |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11475488B2 (en) * | 2017-09-11 | 2022-10-18 | Accenture Global Solutions Limited | Dynamic scripts for tele-agents |
-
2019
- 2019-04-10 CN CN201910285898.4A patent/CN110188116B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101370008A (en) * | 2007-08-13 | 2009-02-18 | 杭州安恒信息技术有限公司 | System for real-time intrusion detection of SQL injection WEB attacks |
| CN103902442A (en) * | 2012-12-25 | 2014-07-02 | 中国移动通信集团公司 | Method and system for evaluating cloud software health degree |
| CN105786682A (en) * | 2016-02-29 | 2016-07-20 | 上海新炬网络信息技术有限公司 | Implementation system and method for avoiding software performance failure |
| CN106649552A (en) * | 2016-11-07 | 2017-05-10 | 湖北省农村信用社联合社网络信息中心 | Normalized data cleaning method |
| CN107908532A (en) * | 2017-05-05 | 2018-04-13 | 平安科技（深圳）有限公司 | A kind of database performance diagnostic method, device and equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110188116A (en) | 2019-08-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110795203B (en) | Resource scheduling method, device, system and computing equipment | |
| CN101321084A (en) | Method and apparatus for generating configuration rules for computing entities within a computing environment using association rule mining | |
| US20130232254A1 (en) | Cloud resource utilization management | |
| CN1604040B (en) | Dynamic transaction control method and system within a host transaction processing system | |
| CN110958161B (en) | Block chain link point monitoring method and device and storage medium | |
| CN114357495B (en) | Prediction machine under-chain aggregation method, device, equipment and medium based on block chain | |
| CN110264171B (en) | Distributed bill processing method, device, equipment and computer readable storage medium | |
| EP3996348A1 (en) | Predicting performance of a network order fulfillment system | |
| CN110070392B (en) | User loss early warning method and device | |
| CN118193072A (en) | Method and device for controlling server dormancy, storage medium and electronic equipment | |
| US20050089063A1 (en) | Computer system and control method thereof | |
| CN110188116B (en) | Processing method and device for checking script, storage medium and electronic device | |
| CN106961358A (en) | Web application system cluster method for monitoring operation states and its system based on daily record | |
| CN106780023A (en) | Automated financial transactions method and transaction system | |
| CN109800085A (en) | Detection method, device, storage medium and the electronic equipment of resource distribution | |
| CN113721950A (en) | Resource loss prevention and control processing method and device | |
| CN109462510B (en) | CDN node quality evaluation method and device | |
| CN112511312A (en) | Assembled consensus method and system | |
| CN116342161A (en) | Data processing method, device, equipment and storage medium for customer churn | |
| CN110851486A (en) | Data storage method and device | |
| US11455558B2 (en) | Method and system for managing events using automated rule generation | |
| CN113742400A (en) | Network data acquisition system and method based on self-adaptive constraint conditions | |
| CN112052147B (en) | Monitoring method, electronic device and storage medium | |
| CN118101440B (en) | Industrial Internet of things data analysis method based on cloud computing, internet of things system and equipment | |
| CN112737848B (en) | Object type determination method and device, storage medium and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20201110 |