CN114637677A - Data system test method and device, electronic equipment and storage medium - Google Patents
Data system test method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114637677A CN114637677A CN202210272406.XA CN202210272406A CN114637677A CN 114637677 A CN114637677 A CN 114637677A CN 202210272406 A CN202210272406 A CN 202210272406A CN 114637677 A CN114637677 A CN 114637677A
- Authority
- CN
- China
- Prior art keywords
- data
- defect
- signal
- data system
- rerun
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003860 storage Methods 0.000 title claims abstract description 21
- 238000010998 test method Methods 0.000 title abstract description 8
- 230000007547 defect Effects 0.000 claims abstract description 261
- 230000008439 repair process Effects 0.000 claims abstract description 85
- 238000012360 testing method Methods 0.000 claims abstract description 79
- 238000000034 method Methods 0.000 claims abstract description 75
- 238000011144 upstream manufacturing Methods 0.000 claims description 31
- 238000004590 computer program Methods 0.000 claims description 23
- 230000008569 process Effects 0.000 abstract description 25
- 230000006854 communication Effects 0.000 abstract description 20
- 238000004891 communication Methods 0.000 abstract description 16
- 238000010923 batch production Methods 0.000 abstract description 4
- 238000007726 management method Methods 0.000 description 29
- 238000012545 processing Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 230000001960 triggered effect Effects 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 238000013102 re-test Methods 0.000 description 4
- 230000002950 deficient Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004140 cleaning Methods 0.000 description 2
- 238000013524 data verification Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 230000008054 signal transmission Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3604—Software analysis for verifying properties of programs
- G06F11/3612—Software analysis for verifying properties of programs by runtime analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/362—Software debugging
- G06F11/366—Software debugging using diagnostics
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Factory Administration (AREA)
Abstract
The invention discloses a method and a device for testing a data system, electronic equipment and a storage medium. The test method of the data system comprises the following steps: acquiring a data defect signal in a data system; repairing the data defect in the data system according to the data defect signal to form defect repair information; triggering the data system to operate and rerun according to the defect repair information; and after the operation rerun of the data system is finished, updating the data defect signal into a to-be-verified state, and outputting the to-be-verified signal of the data system. By triggering the data system to re-run according to the defect repair information, the data system can be directly and automatically notified of re-run batch of the data system after the data defect is repaired, the offline notification process is reduced, and the communication efficiency is improved. Meanwhile, the automatic operation re-batch process of the data system can be realized, the manual workload and the risk of manual operation are reduced, the testing efficiency of the data system is improved, and the testing cost of the data system is reduced.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a method and an apparatus for testing a data system, an electronic device, and a storage medium.
Background
In the process of information system construction, besides the transaction system which needs to meet daily operation of enterprises, a large number of management analysis systems which support enterprise operation decisions are needed. The management analytics class system may be a downstream system in a data system. The management analysis system can obtain data through a data warehouse in the data system, and then process the obtained data to provide decision support for enterprise managers in the forms of query, report forms, graphs and the like. Wherein a data warehouse is a theme-oriented, integrated, time-varying data collection, but where the information itself is relatively stable. The data warehouse stores all business data in the enterprise and is used for supporting the management decision process. The data of the data warehouse is provided by an upstream system in the data system, the upstream system is a transaction system for providing data for the data warehouse, an information system for recording production and management of enterprises is a data source of the data warehouse, and the data is provided to the data warehouse in a message or file mode. Typically, upstream systems provide data to a data warehouse primarily by way of files.
After the data system is built, the data system needs to be tested to ensure normal use of the data system. During the testing of data systems, when a data warehouse or a downstream system issues a data quality problem, a data defect may be proposed in the defect management system and processed by the upstream system. And after the upstream system repairs the data defects, providing the data files for the data warehouse again, and informing technicians of the data warehouse to perform batch running again, wherein the technicians of the data warehouse manually clear the original batch running data according to the data files provided by the upstream system and manually trigger batch running operation to perform batch running again. After the batch running operation of the data warehouse is finished, the data warehouse technical personnel inform data defect extraction personnel of the data warehouse or data defect extraction personnel of a downstream system to verify data. In the test process, after the upstream system repairs the data defects, the upstream system notifies a technician of the data warehouse to perform batch running, and after the batch running operation is completed, the data warehouse technician notifies a data defect presenter of the data warehouse or a data defect presenter of the downstream system to verify data, which are all offline communication notification processes, so that the communication efficiency is low. And the data warehouse is operated manually according to the reruns process of batch running operation of the repaired data files provided by the upstream system, so that the manual workload and the risk of manual operation are increased.
Disclosure of Invention
The invention provides a method and a device for testing a data system, electronic equipment and a storage medium, which are used for improving the testing efficiency of the data system and reducing the testing cost of the data system.
According to an aspect of the present invention, there is provided a method for testing a data system, including:
acquiring a data defect signal in a data system;
repairing the data defects in the data system according to the data defect signal to form defect repair information;
triggering the data system to operate and rerun according to the defect repair information;
and after the operation rerun of the data system is finished, updating the data defect signal to be in a to-be-verified state, and outputting the to-be-verified signal of the data system.
Optionally, repairing the data defect in the data system according to the data defect signal to form defect repair information, including:
and repairing the data defect in the upstream system in the data system according to the data defect signal to form the defect repairing information.
Optionally, after the forming the defect repair information, further comprising:
the upstream system resupplies data to a data warehouse in the data system.
Optionally, triggering the data system job rerun according to the defect repair information includes:
receiving the defect repair information;
sending a notice to be reviewed according to the defect repairing information;
and triggering the data warehouse operation in the data system to rerun after acquiring the approval information.
Optionally, the pending notification includes at least one of a mail form, a short message form, and a WeChat form.
Optionally, triggering the data warehouse job rerun in the data system after obtaining the approval information includes:
and triggering the operation program in the data warehouse to rerun after the approval information is acquired.
Optionally, after the obtaining of the approval information triggers the operation program in the data warehouse to perform rerun, the method further includes:
the data warehouse provides data back to downstream ones of the data systems.
Optionally, after the running of the data system is completed, updating the data defect signal to a to-be-verified state, and outputting the to-be-verified signal of the data system, where the updating includes:
after the operation program of the data warehouse is completely rerun, an operation rerun completion signal is formed;
updating the data defect signal to be in a state to be verified according to the operation rerun completion signal;
and outputting the signal to be verified of the downstream system in the data system according to the state to be verified of the defect data signal.
Optionally, after outputting the signal to be verified of the data system, the method further includes:
and forming a job rerun report.
According to another aspect of the present invention, there is provided a test apparatus for a data system, including:
the data defect signal acquisition module is used for acquiring data defect signals in a data system;
the defect repairing module is used for repairing the data defects in the data system according to the data defect signals to form defect repairing information;
the data rerun module is used for triggering the data system to perform operation rerun according to the defect repair information;
and the to-be-verified signal output module is used for updating the data defect signal to be in a to-be-verified state and outputting the to-be-verified signal of the data system after the operation of the data system is restarted.
Optionally, the data defect signal obtaining module is connected to the defect repairing module, the data rerunning module is connected to the data defect signal obtaining module, and the data defect signal obtaining module is further configured to obtain the defect repairing information and transmit the defect repairing information to the data rerunning module.
Optionally, the data rereading module includes:
a receiving unit configured to receive the defect repair information;
the notification unit is used for sending a to-be-approved notification according to the defect repair information;
and the rerun unit is used for triggering the data warehouse operation rerun in the data system after acquiring the approval information.
According to another aspect of the present invention, there is provided an electronic apparatus including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the method according to any of the embodiments of the invention.
According to another aspect of the present invention, there is provided a computer-readable storage medium having stored thereon computer instructions for causing a processor to execute a method according to any one of the embodiments of the present invention.
According to another aspect of the invention, a computer program product is provided, comprising a computer program which, when executed by a processor, implements a method according to any of the embodiments of the invention.
According to the technical scheme of the embodiment of the invention, the data defect signal in the data system is acquired, the data defect in the data system is repaired according to the data defect signal to form the defect repair information, and the data system operation is triggered to be rerun according to the defect repair information, so that the operation rerun batch of the data system can be directly and automatically notified after the data defect is repaired, the offline notification process is reduced, and the communication efficiency is improved. Meanwhile, the automatic operation re-batch process of the data system can be realized, the manual workload and the risk of manual operation are reduced, the testing efficiency of the data system is improved, and the testing cost of the data system is reduced. And finally, after the operation of the data system is finished again, updating the data defect signal into a to-be-verified state, and outputting the to-be-verified signal of the data system, so that a tester of the data system performs verification test on the repaired data in the data system.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present invention, nor do they necessarily limit the scope of the invention. Other features of the present invention will become apparent from the following description.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a flowchart of a method for testing a data system according to an embodiment of the present invention;
FIG. 2 is a flow chart of another testing method for a data system according to an embodiment of the present invention;
FIG. 3 is a flow chart of another testing method for a data system according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a testing apparatus of a data system according to an embodiment of the present invention;
FIG. 5 is a schematic structural diagram of a testing apparatus of another data system according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of an electronic device implementing the interest rate prediction method according to the embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the term "comprises/comprising" and any variations thereof is intended to cover non-exclusive inclusions, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that, in the embodiment of the present invention, the acquisition, storage and/or processing of the related private data complies with the relevant regulations of the national laws and regulations, and does not violate the customs of the public order.
Fig. 1 is a flowchart of a method for testing a data system according to an embodiment of the present invention, where the method is applicable to a situation that a data system has a data defect during a test process and the data defect is repaired and then retested, and the method may be executed by a testing apparatus of the data system, where the testing apparatus of the data system may be implemented in a form of hardware and/or software, and the testing apparatus of the data system may be configured in an electronic device. As shown in fig. 1, the method includes:
s110, acquiring a data defect signal in a data system;
the data system may include, among other things, an upstream system, a data warehouse, and a downstream system. The upstream system is a transaction system for providing data for the data warehouse, an information system for recording the production and operation of enterprises, and a data source of the data warehouse, and provides the data to the data warehouse in a message or file mode. In this embodiment, the upstream system provides data to the data warehouse in a file manner. A data warehouse is a theme-oriented, integrated, time-varying collection of data, but the information itself is relatively stable. The data warehouse stores all business data in the enterprise and is used for supporting the management decision process. The downstream system can be a management analysis system which obtains data from a data warehouse, and after the data is obtained by the management analysis system, the obtained data is processed, and decision support is provided for enterprise managers in the forms of query, report forms, graphs and the like.
Data defect signals in a data system may be provided by data warehouses in the data system or by testers of downstream systems. In the process of testing the data system, the upstream system provides data to the data warehouse, the data warehouse performs batch operation on the data, then the data after the batch operation is provided to the downstream system, and a tester of the downstream system tests according to the data. When the batch operation is carried out in the data warehouse or a tester of the downstream system tests according to the data, the data provided by the upstream system is easy to find out to have defects, at the moment, the batch operator of the data warehouse or the tester of the downstream system can be used as a data defect proposing person, a data defect signal is formed according to the data defects, the data defect signal can comprise data information with defects and is sent to a testing device of the data system, and therefore the testing device of the data system can obtain the data defect signal. The data defect signal may be sent to a defect management system in a testing apparatus of the data system, and the defect management system may obtain the data defect signal. For example, when a downstream system tester finds a data defect, a data defect signal is provided to a defect management system in a testing device of the data system, so that the defect management system acquires the data defect signal.
S120, repairing the data defects in the data system according to the data defect signals to form defect repairing information;
after the data defect signal is acquired, data repair can be performed according to the data defect corresponding to the data defect signal. For example, the data defect signal may be sent to a data repairer of the data system, and the data repairer may perform manual data repair according to the data defect corresponding to the data defect signal. After the data defect is repaired, defect repair information is formed and used for representing that the data defect is repaired. For example, the data repair personnel may feed back the defect repair information to a defect management system within the testing apparatus of the data system so that the defect management system may obtain the defect repair information.
S130, triggering data system operation rerun according to the defect repair information;
when the defect repair information is formed, the data defects in the data system are repaired, at the moment, the data system operation can be automatically triggered to run and batch again according to the defect repair information, the operation of the data system can be directly and automatically notified to run and batch again after the data defects are repaired, the offline notification process is reduced, and the communication efficiency is improved. Meanwhile, the automatic operation re-batch process of the data system can be realized, the manual workload and the risk of manual operation are reduced, the testing efficiency of the data system is improved, and the testing cost of the data system is reduced. For example, a data rerun system within a test apparatus of a data system may obtain defect repair information from a defect management system, then automatically trigger data of the data system according to the defect repair information, and invoke a data job within the data system to rerun.
And S140, after the operation of the data system is restarted, updating the data defect signal to be in a to-be-verified state, and outputting the to-be-verified signal of the data system.
The data system testing device can acquire the operation rerun state of the data system, after the operation rerun of the data system is completed, the data defect signal can be automatically updated to be in a to-be-verified state according to the operation rerun completion state of the data system, the to-be-verified signal of the data system is formed according to the to-be-verified state of the data defect signal and is output to the data system, so that a tester of the data system can perform rerun test on data in the data system after receiving the to-be-verified signal, and retest of the data system after data defect repair is achieved until the data verification of the data system passes the test. For example, the data rerun system may obtain the operation rerun state of the data system in real time, and after the operation rerun of the data system is completed, the data rerun system may automatically update the data defect signal to the to-be-verified state according to the operation rerun completion state of the data system, and simultaneously form the to-be-verified signal of the data system and notify a tester of the data system, so that the tester of the data system may perform a rerun test on the data in the data system after receiving the to-be-verified signal.
According to the technical scheme of the embodiment of the invention, the data defect signal in the data system is acquired, the data defect in the data system is repaired according to the data defect signal to form the defect repair information, and the data system operation rerun is triggered according to the defect repair information, so that the operation rerun of the data system can be directly and automatically notified after the data defect is repaired, the offline notification process is reduced, and the communication efficiency is improved. Meanwhile, the automatic operation re-batch process of the data system can be realized, the manual workload and the risk of manual operation are reduced, the testing efficiency of the data system is improved, and the testing cost of the data system is reduced. And finally, after the operation of the data system is restarted, updating the data defect signal to be in a to-be-verified state, and outputting the to-be-verified signal of the data system, so that a tester of the data system carries out verification test on the repaired data in the data system.
On the basis of the technical scheme, the method for repairing the data defect in the data system according to the data defect signal to form defect repairing information comprises the following steps:
and repairing the data defect in the upstream system in the data system according to the data defect signal to form defect repairing information.
Specifically, the upstream system is a final source of data in the data system, and after the data defect signal is acquired, data repair can be performed on the corresponding data defect in the upstream system according to the data defect signal, so that the data in the data system can be repaired from the final source, and it is ensured that the data in the upstream system, the data warehouse and the downstream system in the data system are all defect-free. And then after the data repair is completed, defect repair information is formed.
On the basis of the above technical solution, after the defect repair information is formed, the method further includes:
the upstream system re-provides the data to the data warehouse in the data system.
Specifically, after the data defect of the upstream system is repaired, the upstream system can provide the data to the data warehouse again to ensure the accuracy of the data provided by the upstream system to the data warehouse, and when the data in the subsequent data warehouse is run again, the data in the data warehouse can be ensured to be the data after the defect is repaired, so that the retest of the data system is realized.
Fig. 2 is a flowchart of another data system testing method according to an embodiment of the present invention, which is a refinement of the foregoing embodiments. As shown in fig. 2, the method includes:
s210, acquiring a data defect signal in a data system;
s220, repairing the data defects in the data system according to the data defect signals to form defect repairing information;
s230, receiving defect repair information;
the defect repair information may be fed back to the defect management system, and after the defect management system receives the defect repair information, the data rerun system in the test apparatus of the data system may automatically receive the defect repair information as a start signal of the data rerun system. Illustratively, the data rereading system and the defect management system are connected through an interface, and the interface in the data rereading system is used as a receiving unit of the data rereading system. And when the defect management system receives and outputs the defect repair information, an interface on the data rerun system automatically receives the defect repair information so as to start the data rerun system to operate.
S240, sending a to-be-approved notice according to the defect repair information;
after the data reruns system receives the defect repair information, a notification unit in the data reruns system can form a to-be-approved notification according to the defect repair information and send the to-be-approved notification to the approval unit. The approval unit can judge whether to approve according to the defect repair information. For example, the notification unit may send a pending approval notification to an approver, and the approver may perform complex logic judgment according to experience and data approval requirements to determine whether the repaired data can be approved.
Optionally, the pending notification includes at least one of a mail form, a text message form, and a WeChat form.
The notification unit may send an online pending notification, which includes at least one of an email format, a short message format, and a WeChat format. By sending the pending notification on line, the communication efficiency can be improved, the communication process and the content can be recorded, and the risk of losing the communication process and the content is reduced.
And S250, triggering the data warehouse operation in the data system to rerun after acquiring the approval information.
When the examination and approval unit judges that the repaired data can be run and approved according to the defect repair information, the examination and approval unit can send examination and approval information to be fed back to the data rerun system, and the data rerun system triggers the operation of the data warehouse to run and approve again after obtaining the examination and approval information. For example, the approval information sent by the approval unit may be fed back to a rerun unit of the data rerun system, and the rerun unit interacts with a data warehouse in the data system. And after the re-run unit acquires the approval information, triggering the data warehouse to operate and re-run and approve.
And S260, after the operation of the data system is restarted, updating the data defect signal to be in a to-be-verified state, and outputting the to-be-verified signal of the data system.
On the basis of the technical scheme, after the approval information is acquired, the data warehouse operation in the data system is triggered to rerun, and the method comprises the following steps:
and triggering an operation program in the data warehouse to rerun after the approval information is acquired.
The operation batch running process of the data warehouse is a program execution process for batch processing of the acquired data by the data warehouse. After the re-run unit acquires the approval information, a trigger signal is sent to the data warehouse to trigger an operation run-approval program in the data warehouse, so that the data warehouse can perform operation run-approval, and the re-run-approval of the data in the data warehouse is realized.
Optionally, after the job program in the data warehouse is triggered to perform a rerun after the approval information is acquired, the method further includes:
the data warehouse provides data back to downstream systems in the data system.
After the data warehouse performs operation batch running again, the data warehouse can provide data to the downstream system again, so that the data acquired by the downstream system is repaired, the accuracy of the data provided by the data warehouse to the downstream system is ensured, and the data system is retested when the downstream system performs data testing. When the retest of the data system passes, the downstream system can be ensured to provide correct decision support for enterprise managers after processing according to the data.
Fig. 3 is a flowchart of another data system testing method according to an embodiment of the present invention, which is a refinement of the foregoing embodiments. As shown in fig. 3, the method includes:
s310, acquiring a data defect signal in a data system;
s320, repairing the data defects in the data system according to the data defect signals to form defect repairing information;
s330, triggering the data system to operate and rerun according to the defect repair information;
s340, after the operation program of the data warehouse is completely rerun, forming an operation rerun completion signal;
and after the test device of the data system triggers the data system to operate the rerun batch according to the defect repair information, establishing an interactive process between the test device of the data system and the data system. After the re-batching of the operation program of the data warehouse is completed, the testing device of the data system can form an operation re-batching completion signal according to the operation program re-batching state of the data warehouse, and the operation re-batching completion signal is used for representing the operation re-batching completion state of the data warehouse. For example, a data rerun system in a test device of the data system may perform data job rerun cleaning on a data warehouse and set an automatic running number of data job rerun of the data warehouse according to the defect repair information, and then the data warehouse performs data job rerun after the data job rerun cleaning, and a rerun unit in the data rerun system may obtain the automatic running number of the data warehouse for performing data job rerun in real time. When the auto running number set by the re-running unit is reached, the re-running unit may form a job re-running completion signal according to the auto running number.
S350, updating the data defect signal to be in a state to be verified according to the operation rerun completion signal;
after the data rerun system forms the operation rerun completion signal, the data rerun system can send the to-be-verified state of the data defect signal to the defect management system, so that the data defect signal in the defect management system is updated to the to-be-verified state. Illustratively, the data rerun system comprises an updating unit, wherein the updating unit is used for acquiring a job rerun completion signal of the rerun unit, and then the updating unit updates the data defect signal into a state to be verified according to the job rerun completion signal and sends the data defect signal to the defect management system.
And S360, outputting the to-be-verified signal of the downstream system in the data system according to the to-be-verified state of the defect data signal.
When the defect data signal is in a to-be-verified state, the data rerun system can form a to-be-verified signal of the downstream system according to the to-be-verified state of the defect data signal and send the to-be-verified signal to a tester of the downstream system, so that the tester of the downstream system can re-verify and test data in the downstream system after receiving the to-be-verified signal, and the retest of the data system after the data defect is repaired is realized until the data verification of the data system passes the test. Illustratively, the data rerun system can send the signal to be verified in an online mode, and illustratively, the data rerun system can send the signal to be verified in at least one of a mail mode, a short message mode and a WeChat mode, so that the communication efficiency can be improved, the communication process and content can be recorded, and the risk of losing the communication process and the content is reduced. In addition, the tester of the downstream system may present personnel for the defect. And when the tester of the downstream system receives the signal to be verified and then re-verifies and tests the data in the downstream system, if the test result is not defective, the test of the data system is finished. If the test result is defective, the data defect signal can be continuously sent, and the data repairing and retesting processes can be repeated until the test result of the data system is not defective.
On the basis of the above technical solutions, after outputting the signal to be verified of the data system, the method further includes:
and forming a job rerun report.
After the test device of the data system outputs the signal to be verified of the data system, the test device of the data system can form a job rerun report according to the job rerun process of the data warehouse, and the job rerun report can record the job rerun process of the data warehouse. For example, the data rerun system may record the states of the data part, the automatic run number, the rerun result, and the like of the job rerun of the data warehouse according to the job rerun process of the data warehouse, thereby implementing the test process record of the data system. When the job rerun report is transmitted to the test administrator, the test administrator can be facilitated to perform management and optimization of the test apparatus of the data system based on the job rerun report.
Fig. 4 is a schematic structural diagram of a testing apparatus of a data system according to an embodiment of the present invention. As shown in fig. 4, the apparatus includes a data defect signal acquisition module 101, a defect repair module 102, a data rerun module 103, and a to-be-verified signal output module 104;
the data defect signal acquiring module 101 is configured to acquire a data defect signal in a data system; the defect repair module 102 is configured to repair a data defect in the data system according to the data defect signal to form defect repair information; the data rerun module 103 is used for triggering data system operation rerun according to the defect repair information; the to-be-verified signal output module 104 is configured to update the data defect signal to be in a to-be-verified state after the operation rerun of the data system is completed, and output the to-be-verified signal of the data system.
The data defect signal acquiring module 101 may be a defect management system, and the data defect signal acquiring module 101 may acquire a data defect signal provided by a defect proposing person. The defect repair module 102 may be an upstream system repair person in the data system, and the defect repair module 102 may repair the data defect of the upstream system in the data system according to the data defect signal and then form defect repair information. The data rerun module 103 and the to-be-verified signal output module 104 may be a data rerun system, and the data rerun module 103 may trigger a data warehouse job rerun in the data system after acquiring the defect repair information, and simultaneously acquire a state of the data warehouse job rerun in real time, and form a job rerun completion signal after the data warehouse job rerun is completed. The to-be-verified signal output module 104 updates the data defect signal to a to-be-verified state according to the job rerun completion signal, and forms and outputs a to-be-verified signal. And a downstream system tester of the data system can perform data re-verification test in the data system according to the to-be-verified state of the data defect signal to complete the test process of the data system.
The test device of the data system provided by the embodiment of the invention can execute the test method of the data system provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
On the basis of the above technical solution, with reference to fig. 4, the data defect signal obtaining module 101 is connected to the defect repairing module 102, the data rereading module 103 is connected to the data defect signal obtaining module 101, and the data defect signal obtaining module 101 is further configured to obtain defect repairing information and transmit the defect repairing information to the data rereading module 103.
The defect repair information formed by the defect repair module 102 can be transmitted to the data rerun module 103 through the data defect signal acquisition module 101, which is beneficial to realizing automatic transmission of signals of a test device of a data system and improving test efficiency. For example, the defect repair module 102 may be an upstream system repair person in the data system, the data defect signal acquisition module 101 may be a defect management system, the data rerun module 103 may be a part of the data rerun system, and the defect management system and the data rerun system may be connected through an interface. When the defect repair information formed by the defect repair module 102 is fed back to the data defect signal acquisition module 101, the data defect signal acquisition module 101 may directly feed back the defect repair information to the data rerun module 103 through the interface, thereby facilitating automatic transmission of the defect repair information and improving the test efficiency.
Optionally, the defect repair module is specifically configured to repair the data defect in the upstream system in the data system according to the data defect signal, and form defect repair information.
Optionally, the defect repair module, after forming the defect repair information, an upstream system in the data system resupplies the data to a data warehouse in the data system.
Fig. 5 is a schematic structural diagram of another data system testing apparatus according to an embodiment of the present invention.
As shown in fig. 5, the data rerun module 103 includes:
a receiving unit 1031 for receiving defect repair information;
a notification unit 1032, configured to send a to-be-approved notification according to the defect repair information;
a rerun unit 1033, configured to trigger rerun of data warehouse jobs in the data system after obtaining the approval information.
The pending notification sent by the notification unit 1032 may be sent to an approver, and the approver determines whether to re-approve the job of the data warehouse according to the pending notification. When the examining and approving personnel determines that the re-running approval of the operation of the data warehouse is needed, the examining and approving personnel sends examining and approving information to the re-running unit 1033, and the re-running unit 1033 triggers the re-running of the operation of the data warehouse in the data system after acquiring the examining and approving information.
Optionally, the pending notification includes at least one of a mail form, a text message form, and a WeChat form.
Optionally, the rerun unit is specifically configured to trigger an operation program in the data warehouse to rerun after obtaining the approval information.
Optionally, after the rerun unit triggers the job program in the data warehouse to rerun after acquiring the approval information, the data warehouse in the data system provides the data to the downstream system in the data system again.
Optionally, with continued reference to fig. 5, the rerun unit 1033 is further configured to form a job rerun complete signal after the job program rerun of the data warehouse is completed; at this time, the to-be-verified signal output module 104 includes:
an updating unit 1041, configured to update the data defect signal to a to-be-verified state according to the job rerun completion signal;
the updating unit 1041 is further configured to output a to-be-verified signal of a downstream system in the data system according to the to-be-verified state of the defect data signal.
Optionally, with continued reference to fig. 5, the data rerun system further comprises:
a report forming unit 105 for forming a job rerun report after outputting a signal to be verified of the data system.
The job rerun report can be sent to a test manager, and the management and optimization of the test device of the data system can be facilitated for the test manager according to the job rerun report.
The embodiment of the invention also provides the electronic equipment. FIG. 6 illustrates a schematic structural diagram of an electronic device 10 that may be used to implement an embodiment of the present invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 6, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a Read Only Memory (ROM)12, a Random Access Memory (RAM)13, and the like, wherein the memory stores a computer program executable by the at least one processor, and the processor 11 can perform various suitable actions and processes according to the computer program stored in the Read Only Memory (ROM)12 or the computer program loaded from a storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data necessary for the operation of the electronic apparatus 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.
A number of components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, or the like; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The processor 11 performs the various methods and processes described above, such as a method of testing a data system.
In some embodiments, the method for testing a data system provided by any of the embodiments of the present invention may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the method for testing a data system described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured by any other suitable means (e.g., by means of firmware) to perform the method of testing the data system provided by any embodiment of the invention.
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be performed. A computer program can execute entirely on a machine, partly on a machine, as a stand-alone software package partly on a machine and partly on a remote machine or entirely on a remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.
Embodiments of the present invention also provide a storage medium containing computer-executable instructions, which when executed by a computer processor, perform a method of testing a data system, the method comprising:
acquiring a data defect signal in a data system;
repairing the data defect in the data system according to the data defect signal to form defect repair information;
triggering the data system to operate and rerun according to the defect repair information;
and after the operation rerun of the data system is finished, updating the data defect signal to be in a to-be-verified state, and outputting the to-be-verified signal of the data system.
Of course, the storage medium provided by the embodiment of the present invention contains computer-executable instructions, and the computer-executable instructions are not limited to the method operations described above, and may also perform related operations in the test method of the data system provided by any embodiment of the present invention.
Embodiments of the present invention also provide a computer program product, which includes computer-executable instructions, when executed by a computer processor, for performing the method for testing a data system provided in any of the embodiments of the present invention.
Of course, the computer program product provided in the embodiments of the present application has computer-executable instructions that are not limited to the method operations described above, and may also perform related operations in the method provided in any embodiments of the present invention.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present invention may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solution of the present invention can be achieved.
The above-described embodiments should not be construed as limiting the scope of the invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (15)
1. A method for testing a data system, comprising:
acquiring a data defect signal in a data system;
repairing the data defect in the data system according to the data defect signal to form defect repair information;
triggering the data system to operate and rerun according to the defect repair information;
and after the operation rerun of the data system is finished, updating the data defect signal to be in a to-be-verified state, and outputting the to-be-verified signal of the data system.
2. The method of claim 1, wherein repairing a data defect in the data system based on the data defect signal to form defect repair information comprises:
and repairing the data defect in the upstream system in the data system according to the data defect signal to form the defect repairing information.
3. The method of claim 2, further comprising, after forming the defect repair information:
the upstream system resupplies data to a data warehouse in the data system.
4. The method of claim 1, wherein triggering the data system job rerun according to the defect repair information comprises:
receiving the defect repair information;
sending a notice to be reviewed according to the defect repairing information;
and triggering the data warehouse operation in the data system to rerun after acquiring the approval information.
5. The method of claim 4, wherein the notification to approve comprises at least one of a mail form, a text message form, and a WeChat form.
6. The method of claim 4, wherein triggering a data warehouse job rerun in the data system after obtaining approval information comprises:
and triggering the operation program in the data warehouse to rerun after the approval information is acquired.
7. The method of claim 6, after triggering a re-run of a job program in the data warehouse after obtaining the approval information, further comprising:
the data warehouse provides data back to downstream ones of the data systems.
8. The method of claim 6, wherein updating the data defect signal to a verified state after the re-running of the job of the data system is completed, and outputting the verified signal of the data system comprises:
after the operation program of the data warehouse is completely rerun, an operation rerun completion signal is formed;
updating the data defect signal to be in a state to be verified according to the operation rerun completion signal;
and outputting the signal to be verified of the downstream system in the data system according to the state to be verified of the defect data signal.
9. The method of claim 1, further comprising, after outputting the signal to be authenticated for the data system:
and forming a job rerun report.
10. A test apparatus for a data system, comprising:
the data defect signal acquisition module is used for acquiring data defect signals in a data system;
the defect repairing module is used for repairing the data defects in the data system according to the data defect signals to form defect repairing information;
the data rerun module is used for triggering the data system to operate and rerun according to the defect repair information;
and the to-be-verified signal output module is used for updating the data defect signal to be in a to-be-verified state and outputting the to-be-verified signal of the data system after the operation of the data system is restarted.
11. The apparatus of claim 10, wherein the data defect signal obtaining module is connected to the defect repairing module, and the data rerunning module is connected to the data defect signal obtaining module, and is further configured to obtain the defect repairing information and transmit the defect repairing information to the data rerunning module.
12. The apparatus of claim 10, wherein the data re-tracking module comprises:
a receiving unit configured to receive the defect repair information;
the notification unit is used for sending a to-be-approved notification according to the defect repair information;
and the re-running unit is used for triggering the data warehouse operation in the data system to re-run after acquiring the approval information.
13. An electronic device, characterized in that the electronic device comprises:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.
14. A computer-readable storage medium storing computer instructions for causing a processor to perform the method of any one of claims 1-9 when executed.
15. A computer program product, characterized in that the computer program product comprises a computer program which, when being executed by a processor, carries out the method according to any one of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210272406.XA CN114637677A (en) | 2022-03-18 | 2022-03-18 | Data system test method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210272406.XA CN114637677A (en) | 2022-03-18 | 2022-03-18 | Data system test method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114637677A true CN114637677A (en) | 2022-06-17 |
Family
ID=81948939
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210272406.XA Pending CN114637677A (en) | 2022-03-18 | 2022-03-18 | Data system test method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114637677A (en) |
-
2022
- 2022-03-18 CN CN202210272406.XA patent/CN114637677A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114637677A (en) | Data system test method and device, electronic equipment and storage medium | |
| CN114693116A (en) | Method and device for detecting code review validity and electronic equipment | |
| CN116431505A (en) | Regression testing method and device, electronic equipment, storage medium and product | |
| CN115826955A (en) | Application publishing method, system, electronic device and storage medium | |
| CN114741291A (en) | Method, device, equipment and medium for automatically submitting vulnerability information | |
| CN113296836B (en) | Method for training model, test method, device, electronic equipment and storage medium | |
| CN114116688A (en) | Data processing and data quality inspection method, device and readable storage medium | |
| CN114443493A (en) | Test case generation method and device, electronic equipment and storage medium | |
| CN117271373B (en) | Automatic construction method and device for test cases, electronic equipment and storage medium | |
| CN115775134A (en) | Method, device, equipment and medium for managing product test information | |
| CN115437955A (en) | Project testing method, device, equipment and storage medium | |
| CN115878168A (en) | Software development method and device, electronic equipment and storage medium | |
| CN115129718A (en) | Information processing method, device, equipment and storage medium | |
| CN115422088A (en) | Method, device, equipment and medium for processing software bugs | |
| CN115600819A (en) | Risk assessment method and device, electronic equipment and storage medium | |
| CN118298853A (en) | Feedback method and device for speech recognition test abnormality | |
| CN115794623A (en) | Batch task processing method and device, electronic equipment and storage medium | |
| CN108376119B (en) | Visualized processing method and device for research and development data | |
| CN116932337A (en) | Early warning method and early warning system based on software release process | |
| CN112559321A (en) | Method, system, electronic device and storage medium for determining access test case | |
| CN116909925A (en) | Test flow management method and management system based on agile iteration | |
| CN115599682A (en) | Environment availability detection method, device, equipment and storage medium | |
| CN117171034A (en) | Execution result statistics method, device, equipment and storage medium based on brain graph use cases | |
| CN115757097A (en) | Test case generation method and device, electronic equipment and storage medium | |
| CN117541053A (en) | Project risk prompting method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |