CN113688030B - SAS automatic data verification method and system based on file driving - Google Patents
SAS automatic data verification method and system based on file driving Download PDFInfo
- Publication number
- CN113688030B CN113688030B CN202110796674.7A CN202110796674A CN113688030B CN 113688030 B CN113688030 B CN 113688030B CN 202110796674 A CN202110796674 A CN 202110796674A CN 113688030 B CN113688030 B CN 113688030B
- Authority
- CN
- China
- Prior art keywords
- verification
- file
- sas
- data
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Prevention of errors by analysis, debugging or testing of software
- G06F11/3604—Analysis of software for verifying properties of programs
- G06F11/3612—Analysis of software for verifying properties of programs by runtime analysis
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention relates to a SAS automatic data verification method and system based on file driving, wherein the verification method comprises the following steps: s1: building an SAS automatic verification framework; s2: acquiring a verification data source file and storing the verification data source file in a corresponding driving file position; s3: judging whether the driving file position of the to-be-verified SAS module stores a data file or not, if so, carrying out verification processing on the data file on the driving file position on which the to-be-verified SAS module depends by the verification SAS module to obtain a produced data file, otherwise, returning to the step S3 until all the verification SAS modules finish verification, and entering the step S4; s4: and the difference generating SAS module performs verification processing on the data file in the position of the driving file on which the difference generating SAS module depends, obtains a difference result file and completes data verification. Compared with the prior art, the method and the device realize automatic file in-place judgment and can improve verification efficiency.
Description
Technical Field
The invention relates to the field of computer data processing, in particular to a SAS automatic data verification method and system based on file driving.
Background
The data validation effort involved in implementing OLAP (Online Analytical Process) systems involves a large amount of processing logic. In the prior art, a verifier needs to write one or a plurality of SAS (STATISTICAL ANALYSIS SYSTEM) modules, rewrite variable processing logic, make an interdependence among SAS modules and have strict execution sequence, then place data files required by variable processing to a designated position, sequentially execute the SAS modules in an SAS workbench, compare the running result of the SAS modules with the running result of a system to obtain a difference result file, and after the verifier checks the data difference reason, repair the system defect or the SAS module defect, rerun the system or the SAS module, and repeatedly perform the steps until the verification passes. In the prior art, currently, because the SAS module cannot automatically determine whether the data file is in place and automatically operate after the data file is in place, the operation sequence of the SAS module needs to be controlled by a verifier, so that a verification result cannot be generated in time, and the efficiency is low.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a SAS automatic data verification method and system based on file driving.
The aim of the invention can be achieved by the following technical scheme:
an SAS automatic data verification method based on file driving comprises the following steps:
s1: constructing an SAS automatic verification architecture, wherein the automatic verification architecture comprises a plurality of verification SAS modules and a difference generation SAS module;
s2: acquiring a verification data source file and storing the verification data source file in a corresponding driving file position;
S3: judging whether the driving file position of the verification SAS module to be verified stores a data file or not, if so, carrying out verification processing on the data file on the driving file position on which the verification SAS module depends by the verification SAS module to obtain a produced data file and storing the produced data file in the corresponding driving file position, otherwise, returning to the step S3 until all the verification SAS modules finish verification, and entering the step S4;
S4: and the difference generating SAS module performs verification processing on the data file in the position of the driving file on which the difference generating SAS module depends, obtains a difference result file and completes data verification.
Preferably, the data source of the verification data source file comprises a plurality of bins, approval system data, collect urging system data and card issuing core system data.
Preferably, each of the SAS verification module and the SAS difference generation module includes a listening submodule and an executing submodule, where the listening submodule detects in real time whether a data file is stored in a driving file position on which the module depends, and the executing submodule performs verification processing on the data file in the driving file position on which the module depends.
Preferably, the listening submodule adopts a Quartz middleware to scan and drive the file in place at the file position at fixed time intervals.
Preferably, the SAS automatic verification architecture further comprises a database for storing verification data source files and production data files.
An SAS automatic data verification system based on file driving comprises an SAS automatic verification architecture, a data source file acquisition module and a task configuration module,
The automatic verification architecture comprises a plurality of verification SAS modules and a difference generation SAS module;
The data source file acquisition module is used for acquiring a verification data source file and storing the verification data source file in a corresponding driving file position;
the task configuration module is used for judging whether the driving file position of the to-be-verified SAS module stores a data file or not, if so, the verification SAS module performs verification processing on the data file on the driving file position on which the to-be-verified SAS module depends, the output data file is obtained and stored in the corresponding driving file position, otherwise, the data file is judged and verified again until all the verification SAS modules complete verification;
after all the SAS modules are verified, the SAS module performs verification processing on the data files in the position of the driving file on which the SAS module depends, and obtains a difference result file to complete data verification.
Preferably, the data source of the verification data source file comprises a plurality of bins, approval system data, collect urging system data and card issuing core system data.
Preferably, each of the SAS verification module and the SAS difference generation module includes a listening submodule and an executing submodule, where the listening submodule detects in real time whether a data file is stored in a driving file position on which the module depends, and the executing submodule performs verification processing on the data file in the driving file position on which the module depends.
Preferably, the listening submodule adopts a Quartz middleware to scan and drive the file in place at the file position at fixed time intervals.
Preferably, the SAS automatic verification architecture further comprises a database for storing verification data source files and production data files.
Compared with the prior art, the invention has the following advantages:
(1) The invention constructs an SAS automatic verification framework based on file dependence and driving relation, and utilizes the verification SAS module and the difference generation SAS module to sequentially execute verification data source files according to the appointed sequence, automatically judges whether the data files are in place or not, and automatically operates after the data files are in place, without manually adjusting and checking the sequence, thereby effectively improving the efficiency and accuracy of SAS verification;
(2) The file driving relation among the SAS modules can be adjusted according to the verification requirement, the method can be realized quickly, the expansibility is good, the corresponding adjustment is carried out according to the data verification process of the OLAP system, the application range is wide, and the real-time operation can be carried out under different data scenes;
(3) According to the invention, the SAS module can effectively improve the stability and reliability of the verification process according to whether the data file is stored in the dependent driving file position or not as the execution identifier, avoid the phenomenon of disordered verification steps in the data verification process, and improve the verification accuracy;
(4) The SAS automatic verification architecture of the invention utilizes the concurrency form to judge and execute the states of a plurality of SAS modules simultaneously so as to maximally realize the utilization rate of server resources and has high verification efficiency.
Drawings
FIG. 1 is a flow chart of the present invention;
fig. 2 is a file logic diagram of an SAS automatic verification architecture in an embodiment of the present invention.
Detailed Description
The invention will now be described in detail with reference to the drawings and specific examples. Note that the following description of the embodiments is merely an example, and the present invention is not intended to be limited to the applications and uses thereof, and is not intended to be limited to the following embodiments.
Examples
The invention discloses a SAS automatic data verification method based on file driving, which comprises the steps of obtaining a verification data source file, carrying out data verification, comparing a verification result with a data system operation result of an OLAP system, generating a difference result file, and completing the result verification of the OLAP system, wherein the method is shown in figure 1 and comprises the following steps:
S1: an SAS automatic verification architecture is constructed, the automatic verification architecture comprising a plurality of verification SAS modules, a difference generation SAS module, and a database for storing verification data source files and yield data files. The SAS verification module and the SAS difference generation module comprise a monitoring submodule and an execution submodule, the monitoring submodule detects whether a data file is stored in a driving file position relied on by the SAS verification module in real time, and the monitoring submodule scans the file in place in the driving file position by adopting a Quartz middleware at fixed time intervals. The execution sub-module performs verification processing on the data file at the location of the drive file on which it depends. Specifically, the driving file position is a file path, and whether the data file is stored in the driving file position is judged by checking whether the file path has a corresponding file name.
In this embodiment, as shown in fig. 2, the SAS device includes four SAS verification modules and one SAS difference generation module, where the driving file positions corresponding to SAS verification module 1 are driving file positions 1 and 2, the driving file position corresponding to SAS verification module 1 is driving file position 3, the driving file positions corresponding to SAS verification module 3 are driving file positions 4 and 5, the driving file positions corresponding to SAS verification module 4 are driving file positions 6 and 7, and the driving file positions of the SAS difference generation module are driving file positions 7 and 8; the data files generated by SAS module 1 are stored in drive file location 5, the data files generated by SAS module 2 are stored in drive file location 6, the data files generated by SAS module 3 are stored in drive file location 7, and the data files generated by SAS module 4 are stored in drive file location 8.
S2: and acquiring the verification data source file and storing the verification data source file in a corresponding driving file position.
In this embodiment, verification data source files 1,2,3, 4 are acquired and stored in drive file locations 1,2,3, 4, respectively. The data source of the verification data source file comprises a plurality of bins, approval system data, collect urging system data and card issuing core system data.
S3: judging whether the driving file position of the to-be-verified SAS module stores a data file or not, if so, performing verification processing on the data file on the driving file position on which the SAS module depends by the SAS module, obtaining a produced data file and storing the produced data file in the corresponding driving file position, otherwise, returning to the step S3 until all the SAS modules are verified, and entering the step S4.
The invention adopts a concurrent processing mode, specifically, in the embodiment, the verification processing execution of all the verification SAS modules is completed by 3 rounds of interval scanning, specifically:
(1) The monitoring submodules of the verification SAS modules 1 and 2 detect that a data file is in place, a verification data source file 1 and a verification data source file 2 are stored in the driving file position of the verification SAS module 1, and the verification SAS module 1 performs verification processing on the verification data source file 1 and the verification data source file 2 to obtain a production data file 1; the driving file position of the verification SAS module 2 is stored with a verification data source file 3, and the verification SAS module 2 performs verification processing on the verification data source file 3 to obtain a production data file 2;
(2) The monitoring submodule of the verification SAS module 3 detects that a data file is in place, a verification data source file 4 and a production data file 1 are stored in a driving file position of the verification SAS module 1, and the verification SAS module 1 performs verification processing on the verification data source file 4 and the production data file 1 to obtain the production data file 3
(3) The monitoring submodule of the verification SAS module 4 detects that the data file is in place, the output data files 2 and 3 are stored in the driving file position of the verification SAS module 4, and the verification SAS module 4 performs verification processing on the output data files 2 and 3 to obtain the output data file 4.
S4: and the difference generating SAS module performs verification processing on the data file in the position of the driving file on which the difference generating SAS module depends, obtains a difference result file and completes data verification.
In this embodiment, the monitoring submodule of the difference generating SAS module detects that the data file is in place, the output data files 3 and 4 are stored in the driving file position of the difference generating SAS module, the difference generating SAS module performs verification processing on the output data files 3 and 4, and compares the verification processing result with the output system file of the OLAP system to obtain a difference result file, so as to complete data verification.
After the difference result file is obtained, the difference reason is obtained according to the difference result file, after the OLAP system defect or the SAS module defect is repaired, the OLAP system or the SAS automatic verification framework is restarted, and the steps are repeated until verification passes.
Based on the same inventive concept, the embodiment of the invention also provides a SAS automatic data verification system based on file driving, and because the principle of solving the problems of the devices is similar to that of a SAS automatic data verification method based on file driving, the implementation of the method can be referred to, and the repetition is omitted.
An SAS automatic data verification system based on file driving comprises an SAS automatic verification architecture, a data source file acquisition module and a task configuration module,
The automated verification architecture includes a plurality of verification SAS modules and a difference generation SAS module;
the data source file acquisition module is used for acquiring a verification data source file and storing the verification data source file in a corresponding driving file position;
The task configuration module is used for judging whether the driving file position of the to-be-verified SAS module stores a data file or not, if so, the verification SAS module performs verification processing on the data file on the driving file position on which the to-be-verified SAS module depends, the output data file is obtained and stored in the corresponding driving file position, otherwise, the data file is judged and verified again until all the verification SAS modules complete verification;
after all the verification SAS modules complete verification, the difference generation SAS module performs verification processing on the data files in the positions of the driving files on which the difference generation SAS module depends, obtains difference result files and completes data verification.
The data source of the verification data source file comprises a plurality of bins, approval system data, collect urging system data and card issuing core system data.
The SAS verification module and the SAS difference generation module comprise a monitoring submodule and an execution submodule, the monitoring submodule detects whether the data file is stored in the position of the driving file relied on by the SAS verification module in real time, and the execution submodule verifies the data file in the position of the driving file relied on by the SAS verification module.
The monitoring sub-module scans and drives the file in place on the file position by adopting the Quartz middleware at fixed time intervals.
The SAS automatic verification architecture further comprises a database for storing verification data source files and yield data files.
The above embodiments are merely examples, and do not limit the scope of the present invention. These embodiments may be implemented in various other ways, and various omissions, substitutions, and changes may be made without departing from the scope of the technical idea of the present invention.
Claims (6)
1. The automatic data verification method of the SAS based on file driving is characterized by comprising the following steps:
s1: constructing an SAS automatic verification architecture, wherein the automatic verification architecture comprises a plurality of verification SAS modules and a difference generation SAS module;
s2: acquiring a verification data source file and storing the verification data source file in a corresponding driving file position;
S3: judging whether the driving file position of the verification SAS module to be verified stores a data file or not, if so, carrying out verification processing on the data file on the driving file position on which the verification SAS module depends by the verification SAS module to obtain a produced data file and storing the produced data file in the corresponding driving file position, otherwise, returning to the step S3 until all the verification SAS modules finish verification, and entering the step S4;
S4: the difference generating SAS module performs verification processing on the data files in the positions of the driving files on which the SAS module depends to obtain difference result files, and data verification is completed;
The system comprises a verification SAS module, a difference generation SAS module and an execution sub-module, wherein the verification SAS module and the difference generation SAS module comprise a monitoring sub-module and an execution sub-module, the monitoring sub-module detects whether a data file is stored in a driving file position relied by the monitoring sub-module in real time, the monitoring sub-module adopts a Quartz middleware to scan the file in place condition in the driving file position at fixed time intervals, and the execution sub-module performs verification processing on the data file in the driving file position relied by the monitoring sub-module;
The verification data source files are sequentially executed according to the appointed sequence by using the verification SAS module and the difference generation SAS module, whether the data files are in place or not is automatically judged, and the data files are automatically operated after the data files are in place.
2. The method for automatically verifying SAS data based on file driving of claim 1, wherein the data source of the verification data source file comprises a plurality of bins, approval system data, collect system data and card issuing core system data.
3. The file drive based SAS automatic data verification method of claim 1 wherein the SAS automatic verification architecture further comprises a database for storing verification data source files and production data files.
4. The SAS automatic data verification system based on file driving is characterized by comprising an SAS automatic verification architecture, a data source file acquisition module and a task configuration module,
The automatic verification architecture comprises a plurality of verification SAS modules and a difference generation SAS module;
The data source file acquisition module is used for acquiring a verification data source file and storing the verification data source file in a corresponding driving file position;
the task configuration module is used for judging whether the driving file position of the to-be-verified SAS module stores a data file or not, if so, the verification SAS module performs verification processing on the data file on the driving file position on which the to-be-verified SAS module depends, the output data file is obtained and stored in the corresponding driving file position, otherwise, the data file is judged and verified again until all the verification SAS modules complete verification;
After all the verification SAS modules complete verification, the difference generation SAS module performs verification processing on the data files in the positions of the driving files on which the difference generation SAS module depends, obtains difference result files and completes data verification;
The system comprises a verification SAS module, a difference generation SAS module and a difference generation SAS module, wherein the verification SAS module and the difference generation SAS module comprise a monitoring submodule and an execution submodule, the monitoring submodule detects whether a data file is stored in a driving file position relied by the monitoring submodule in real time, and the execution submodule performs verification processing on the data file in the driving file position relied by the monitoring submodule;
The monitoring submodule adopts a Quartz middleware to scan and drive a file in place at a file position at fixed time intervals;
The verification data source files are sequentially executed according to the appointed sequence by using the verification SAS module and the difference generation SAS module, whether the data files are in place or not is automatically judged, and the data files are automatically operated after the data files are in place.
5. The file drive based SAS automatic data verification system of claim 4 wherein the data source of the verification data source file comprises a plurality of bins, approval system data, credit system data, and card issuing core system data.
6. The file drive based SAS automatic data verification system of claim 4 wherein the SAS automatic verification architecture further comprises a database for storing verification data source files, production data files.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110796674.7A CN113688030B (en) | 2021-07-14 | 2021-07-14 | SAS automatic data verification method and system based on file driving |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110796674.7A CN113688030B (en) | 2021-07-14 | 2021-07-14 | SAS automatic data verification method and system based on file driving |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113688030A CN113688030A (en) | 2021-11-23 |
| CN113688030B true CN113688030B (en) | 2024-07-30 |
Family
ID=78577133
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110796674.7A Active CN113688030B (en) | 2021-07-14 | 2021-07-14 | SAS automatic data verification method and system based on file driving |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113688030B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108958775A (en) * | 2018-07-18 | 2018-12-07 | 郑州云海信息技术有限公司 | A kind of detection method of SAS holding firmware online upgrading reliability |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104424070A (en) * | 2013-08-27 | 2015-03-18 | 鸿富锦精密工业（深圳）有限公司 | SAS (Serial Attached Small Computer System Interface) testing device, system and method |
| JP2015056021A (en) * | 2013-09-11 | 2015-03-23 | 富士通株式会社 | Host bus driver verification device, host bus verification system, and program |
| EP3312783A1 (en) * | 2016-10-18 | 2018-04-25 | Mu Sigma Business Solutions Pvt. Ltd. | Model validation system and method |
| CN107832066A (en) * | 2017-10-27 | 2018-03-23 | 郑州云海信息技术有限公司 | A kind of SAS holding firmwares automatic Verification, updating system and method based on cloud platform |
| CN110928535B (en) * | 2019-12-10 | 2024-08-16 | 深圳前海微众银行股份有限公司 | Derived variable deployment method, device, equipment and readable storage medium |
-
2021
- 2021-07-14 CN CN202110796674.7A patent/CN113688030B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108958775A (en) * | 2018-07-18 | 2018-12-07 | 郑州云海信息技术有限公司 | A kind of detection method of SAS holding firmware online upgrading reliability |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113688030A (en) | 2021-11-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115437336A (en) | Test method and device for test case, electronic equipment and storage medium | |
| CN111444094B (en) | Test data generation method and system | |
| CN105786695A (en) | Data test method and system | |
| CN117112400A (en) | Automatic test case generation platform | |
| CN118503270A (en) | NL2SQL data set construction method, device, equipment and medium | |
| CN102740332A (en) | Mobile terminal test method and test device | |
| CN113688030B (en) | SAS automatic data verification method and system based on file driving | |
| US20050203717A1 (en) | Automated testing system, method and program product using testing map | |
| CN100520732C (en) | Performance test script generation method | |
| CN117971633A (en) | An automated testing tool and method for software integrated development system | |
| CN112765014B (en) | An automatic testing system and working method for simultaneous operation by multiple users | |
| CN119359163B (en) | Product quality monitoring method, device, equipment and medium based on multi-stage decision | |
| CN120407404A (en) | Interface automation testing method, device and interface automation testing system | |
| CN110427316A (en) | Embedded software defect-restoration method therefor based on access behavior perception | |
| CN111243647B (en) | Flash memory programming parameter determination method and device, electronic equipment and storage medium | |
| CN114138768B (en) | Data management method and system based on financial cloud data warehouse and data lake | |
| CN114357919B (en) | Feedback convergence method and device for optimizing incentive constraint files | |
| CN116541226A (en) | Production test method, production test device and production test system of server | |
| CN117407318A (en) | Logistics system quality monitoring based method | |
| CN100501698C (en) | Method and system for automatic checking for man-machine command | |
| KR20220095167A (en) | Artificial Intelligence Verification System and Method | |
| CN120639523B (en) | A state preprocessing method for network protocol program analysis | |
| CN118276534B (en) | Automation system configuration and development methods, intelligent platforms, systems, media, and equipment | |
| CN117827644B (en) | Automated testing method and automated testing device | |
| CN119849981B (en) | A method for constructing a sequential sampling test model based on prior information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |