CN112463829A - Data checking method, device, equipment and storage medium - Google Patents

Data checking method, device, equipment and storage medium Download PDF

Info

Publication number
CN112463829A
CN112463829A CN202011311730.5A CN202011311730A CN112463829A CN 112463829 A CN112463829 A CN 112463829A CN 202011311730 A CN202011311730 A CN 202011311730A CN 112463829 A CN112463829 A CN 112463829A
Authority
CN
China
Prior art keywords
data
inspection
methods
target
checking
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011311730.5A
Other languages
Chinese (zh)
Inventor
王勇涛
刘浩
王立新
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Construction Bank Corp
Original Assignee
China Construction Bank Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Construction Bank Corp filed Critical China Construction Bank Corp
Priority to CN202011311730.5A priority Critical patent/CN112463829A/en
Publication of CN112463829A publication Critical patent/CN112463829A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2455—Query execution
    • G06F16/24564—Applying rules; Deductive queries
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • Quality & Reliability (AREA)
  • Stored Programmes (AREA)

Abstract

The application discloses a data inspection method, a device, equipment and a storage medium, wherein the method comprises the steps of responding to a data inspection instruction, obtaining inspection scheme data of a target system, wherein the inspection scheme data comprises a plurality of inspection methods, corresponding method dependence information and a method data mapping relation; performing execution sequence configuration on the plurality of inspection methods based on the corresponding method dependence information to obtain target execution sequences of the plurality of inspection methods; respectively acquiring target data to be inspected corresponding to a plurality of inspection methods based on method data mapping relations; and performing data inspection on the corresponding target data to be inspected based on the target execution sequence of the plurality of inspection methods to obtain a data inspection result of the target system. The technical scheme who utilizes this application to provide can carry out data inspection rationally in an orderly manner, promotes the degree of automation of data inspection, and is favorable to promoting data inspection's efficiency and reliability.

Description

Data checking method, device, equipment and storage medium
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data checking method, apparatus, device, and storage medium.
Background
Along with the continuous development of the business of each company and the Internet, the number of personnel, the number of systems and the complexity in the company continuously rise, once violation matters occur, serious economic loss can be brought, the importance of compliance management is continuously promoted, at present, a set of complete compliance management flow is arranged in each company, and managers of the company can perform data inspection according to the system data of each department in the company so as to achieve the purpose of compliance audit.
In the prior art, because each system may be provided by different manufacturers, the data formats and specifications of the systems are inconsistent, when data inspection is performed, a compliance manager mainly finds out violations among the system data through manual unified formats after exporting operation data and manual or auxiliary office automation software, obtains data inspection results, and completes compliance audit. As company systems become more and personnel increase rapidly, data inspection needs to consume a large amount of human resources, and has high cost, low efficiency and high error probability, so that a more accurate and efficient method needs to be provided.
Disclosure of Invention
In order to solve the problems in the prior art, the application provides a data checking method, a device, equipment and a storage medium. The technical scheme is as follows:
one aspect of the present application provides a data checking method, including:
responding to a data inspection instruction, acquiring inspection scheme data of a target system, wherein the inspection scheme data comprises a plurality of inspection methods, corresponding method dependence information and a method data mapping relation;
performing execution sequence configuration on the plurality of inspection methods based on the corresponding method dependence information to obtain target execution sequences of the plurality of inspection methods;
respectively acquiring target data to be inspected corresponding to the plurality of inspection methods based on the method data mapping relation;
and performing data inspection on the corresponding target data to be inspected based on the target execution sequence of the plurality of inspection methods to obtain a data inspection result of the target system.
Another aspect of the present application provides a data inspection apparatus, including:
the system comprises a scheme data acquisition module, a data processing module and a data processing module, wherein the scheme data acquisition module is used for responding to a data inspection instruction and acquiring inspection scheme data of a target system, and the inspection scheme data comprises a plurality of inspection methods, corresponding method dependence information and a method data mapping relation;
an execution sequence configuration module, configured to perform execution sequence configuration on the plurality of inspection methods based on the corresponding method dependency information, so as to obtain target execution sequences of the plurality of inspection methods;
the data to be inspected acquisition module is used for respectively acquiring the data to be inspected of the targets corresponding to the plurality of inspection methods based on the method data mapping relation;
and the data inspection module is used for performing data inspection on the corresponding target data to be inspected based on the target execution sequence of the plurality of inspection methods to obtain a data inspection result of the target system.
Another aspect of the present application provides a data checking apparatus, which includes a processor and a memory, where at least one instruction or at least one program is stored in the memory, and the at least one instruction or the at least one program is loaded by the processor and executed to implement the data checking method as described above.
Another aspect of the present application provides a computer-readable storage medium, in which at least one instruction or at least one program is stored, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the data checking method as described above.
The data checking method, the device, the equipment and the storage medium have the following technical effects:
the method comprises the steps of responding to a data inspection instruction, obtaining inspection scheme data of a target system, wherein the inspection scheme data comprise a plurality of inspection methods, corresponding method dependence information and method data mapping relations; then, the execution sequence configuration can be carried out on the plurality of checking methods based on the corresponding method dependence information, so that the target execution sequence of the plurality of checking methods is obtained, and the data checking is favorably and orderly carried out; and then, respectively acquiring target data to be inspected corresponding to a plurality of inspection methods based on the method data mapping relation, and performing data inspection on the corresponding target data to be inspected based on the target execution sequence of the plurality of inspection methods to obtain a data inspection result of the target system. The technical scheme provided by the embodiment of the specification can be utilized to carry out data inspection reasonably and orderly, the automation degree of the data inspection is improved, and the efficiency and the reliability of the data inspection are improved.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Drawings
In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a schematic diagram of an application environment provided by an embodiment of the present application;
fig. 2 is a schematic flow chart of a data checking method provided in an embodiment of the present application;
FIG. 3 is a schematic flow chart diagram of another data checking method provided in the embodiments of the present application;
FIG. 4 is a schematic flow chart diagram illustrating another data checking method provided in the embodiments of the present application;
FIG. 5 is a schematic flow chart diagram of another data checking method provided in the embodiments of the present application;
FIG. 6 is a schematic flow chart diagram illustrating another data checking method provided in the embodiments of the present application;
FIG. 7 is a schematic flow chart diagram illustrating another data checking method provided in the embodiments of the present application;
FIG. 8 is a schematic flow chart diagram illustrating another data checking method provided by an embodiment of the present application;
FIG. 9 is a schematic diagram of a data checking apparatus according to an embodiment of the present application;
fig. 10 is a block diagram of a hardware structure of a server for implementing a data checking method according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application. Examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the same or similar elements or elements having the same or similar function throughout.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or server that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Referring to fig. 1, fig. 1 is a schematic diagram of an application environment provided by the present application, and as shown in fig. 1, the application environment may include a data inspection server 01 and a plurality of systems to be inspected 02.
In the embodiment of the present specification, the data inspection server 01 may be configured to perform data inspection in combination with data of at least one system to be inspected 02, and the data inspection server 01 may include a network communication unit, a processor, a memory, and the like. Wherein the processor may be used to obtain data, process data, perform data inspection, and the like; the memory can be used for storing data to be checked obtained after acquisition and processing, data checking results obtained after data checking and the like. Alternatively, the data checking server 01 may include a server operating independently, or a distributed server, or a server cluster composed of a plurality of servers. In a specific embodiment, the data inspection server 01 may include an acquisition unit, a scheduling unit, a data inspection unit, a storage unit, a configuration unit, and a communication unit, where the acquisition unit may be configured to perform acquisition of source system data and related data processing; the scheduling unit may be configured to perform execution order configuration on the plurality of inspection methods based on the corresponding method-dependent information, obtain target execution orders of the plurality of inspection methods, and control execution of the data inspection unit; the data checking unit may be configured to perform a corresponding checking method; the storage unit can be used for storing the data to be checked obtained after the acquisition and the processing, the data checking result obtained after the data checking, and the like; the configuration unit can be used for providing configuration information required in the data inspection process, such as data access addresses and the like; the communication unit may be used to implement data communication between the units and assist in completing sending and receiving of messages, and it should be noted that this is only one embodiment provided in the application, and in practical applications, the data checking server 01 may also include more or less units, or have a different configuration from that described in the above embodiments.
In this embodiment, a plurality of systems to be inspected 02 may generate source system data and the like, so that the data inspection server 01 can obtain required data to perform data inspection, and in one embodiment, the system to be inspected 02 may include servers for providing different service functions, such as a card punching system, an reimbursement system, a task allocation system, and the like, and specifically, the system to be inspected 02 may be an independent physical server, or a server cluster or a distributed system formed by a plurality of physical servers. In practical applications, data inspection is often required for one or more target systems, where the target system may be a department or other systems set according to actual inspection needs, and a target system may be associated with at least one system 02 to be inspected, for example, the target system may be a human resource department, and when compliance inspection is required for the department, a reimbursement system may be required, and a personnel entry system may wait for relevant data of the inspection system 02.
The data inspection server 01 may be connected to a plurality of inspection systems 02 via a network by various wireless communication methods or wired communication methods. The network may be a wide area network or a local area network, or a combination of the two, which is not limited herein; in practical applications, the data checking server 01 may further include a display unit, and the display unit may be configured to visually display the real-time checking progress data when the data checking is performed.
Fig. 2 is a flow chart of a data inspection method provided in an embodiment of the present application, and the present specification provides the method operation steps as described in the embodiment or the flow chart, but more or less operation steps can be included based on conventional or non-inventive labor. The order of steps recited in the embodiments is merely one manner of performing the steps in a multitude of orders and does not represent the only order of execution. In practice, the system or server product may be implemented in a sequential or parallel manner (e.g., parallel processor or multi-threaded environment) according to the embodiments or methods shown in the figures. Specifically, as shown in fig. 2, the method may include:
s201: and acquiring the inspection scheme data of the target system in response to the data inspection instruction.
Specifically, the inspection plan data of the target system includes a plurality of inspection methods and corresponding method dependency information, and a method data mapping relationship.
In the embodiment of the present disclosure, the data checking method may include a compliance auditing method, and the target system may correspond to a department system in a company, and with the continuous development of business and internet of each company, a company may include multiple departments, such as an IT department, a human resource department, and the like, the corresponding department can be determined by combining with the actual inspection requirement to be used as a target system to carry out data inspection, and illegal personnel or illegal items can be found out so as to carry out corresponding treatment subsequently, thus reducing the loss caused by the illegal items, and the informatization degree is gradually improved at present, in order to facilitate company management, the electronic platforms with different service functions are often established to operate and manage in different modules, the electronic platforms, namely the system to be inspected during data inspection, can provide corresponding operation data or operation data to perform data inspection; for example, the system to be inspected may include an operating system, reimbursement system, learning system, etc. corresponding to persons with different access rights. One target system is often associated with data of a plurality of systems to be checked, namely, the data of the plurality of systems to be checked are needed when one department needs to be subjected to compliance check; for example, when the target system includes a human resources department system, a reimbursement system and a personnel entry system may be required to wait for relevant data of the inspection system when data compliance inspection needs to be performed on the department.
It can be understood that, in practical application, the target systems may include a plurality of target systems, and the corresponding inspection plan data is respectively obtained, that is, data inspection may be performed on the plurality of target systems at a time, and the data inspection requirements may be specifically determined in combination with actual data inspection requirements, which is not limited in this application.
In this embodiment, when the data checking method includes a compliance auditing method, the data checking instruction may include an auditing instruction, the checking scheme data may include auditing scheme data, and the checking method may include an auditing method; the data inspection instruction can be sent out by the timing inspection unit, and optionally, a manager can also send out the data inspection instruction by combining with the actual compliance inspection requirement.
Referring to fig. 3, in an embodiment of the present disclosure, before acquiring the inspection plan data of the target system in response to the data inspection instruction, the method may further include:
s301: collecting a plurality of source system data;
in the embodiment of the present specification, the source system data may include data of the system to be inspected as described above, and specifically, the source system data may include, but is not limited to, an reimbursement system, a learning system, an operating system corresponding to persons with different access rights, and a person entry system.
In practical application, a data acquisition command can be sent out by the data acquisition unit at regular time to acquire source system data corresponding to each system to be inspected at regular time, so that the integrity of the source system data acquisition is improved, and the subsequent data inspection is more rigorous and reliable; corresponding data acquisition instructions can be sent by management personnel when data inspection is needed, required source system data are acquired for follow-up data inspection, acquisition resources and storage space consumption are reduced according to the requirement, and the data inspection instructions can be automatically sent to perform follow-up data inspection after the data acquisition is completed. In addition, the data of the source system can be acquired according to a preset acquisition sequence by combining with the actual acquisition requirement; and the acquired system to be inspected can be set by combining with the inspection requirement of actual data, and can be flexibly increased or reduced only by configuring corresponding parameters, which is not limited by the application.
In the embodiment of the present description, different data acquisition modes may be used to acquire source system data according to the characteristics of the system to be inspected and in combination with actual application requirements. In practical applications, the data acquisition manners described above may include, but are not limited to: direct transfer, access, import export, and reverse; specifically, the direct transmission type can directly acquire source system data according to a preset acquisition format without developing an additional acquisition interface; for example, when the system to be checked supports application Programming interface (api) access, data acquisition may be performed by using an access method, and source system data may be acquired by calling an application program interface; in some embodiments, the system to be inspected may be a third-party system, or the system may be developed earlier, only provide partial data export, and the like, and the source system data may be obtained through export import; in other embodiments, the system to be inspected may not provide a data access manner, and at this time, source system data may be acquired by using a reverse manner, such as web crawlers or RPAs (robotics Automation), before actual data acquisition, corresponding parameter information may be configured according to a corresponding data acquisition manner, for example, a related data access address of the system to be inspected may be preconfigured by using a direct transfer manner to acquire data, an API interface access address and authentication information of the system to be inspected may be preconfigured by using an access manner to acquire data, location information of an import file may be preconfigured by using an import/export manner to acquire data, an access address and user login information of the corresponding system to be inspected may be preconfigured by using a reverse manner to acquire data, and the parameter information may further include an acquisition policy of the system to be inspected, such as full or incremental acquisition, and acquisition conditions and data boundaries, etc., which should not be construed as limiting the present application.
The data of the source system is acquired by combining different data acquisition modes according to the characteristics of the system to be inspected and the actual application requirements, so that the data acquisition process is more flexible and efficient.
S303: respectively carrying out data structuring processing on the source system data to obtain corresponding structured data;
in practical application, because each system to be inspected is probably provided by different manufacturers, the data format and the standard between systems are often inconsistent, so that data structuring processing is necessary to be carried out on a plurality of source system data, the data are stored according to a preset data format, corresponding data inspection is conveniently carried out subsequently, the data acquisition efficiency is improved, the consumption of human resources is reduced, the dependence on manpower is reduced, the automation degree of the data inspection is improved, and the efficiency and the reliability of the data inspection are favorably improved.
S305: respectively carrying out compliance screening on the corresponding structured data to obtain corresponding compliance data;
because the source system data collected in practical application may include some illegal items or error data, it is necessary to perform compliance screening on the corresponding structured data to obtain corresponding compliance data, which is beneficial to improving the reliability of data inspection.
S307: respectively extracting characteristic values of the corresponding compliance data to obtain a plurality of data to be checked;
by extracting the characteristic value of the corresponding compliance data, the data with higher correlation with the data inspection can be acquired, and the efficiency of the data inspection is improved.
S309: and constructing a source database based on the plurality of data to be checked.
In this embodiment of the present specification, a source database may be constructed based on a plurality of data to be checked obtained after collection processing, so that corresponding data to be checked may be obtained from the source database to complete data checking during subsequent data checking, specifically, data of a source system may be continuously collected and processed to increase and update data in the source database, deletion of data in the source database may also be realized by data archiving or cleaning, and required data may be read from the source database according to an established method data mapping relationship during subsequent auditing, so as to realize offline data checking.
Through gathering a plurality of source system data, carry out data structure to a plurality of source system data of aforesaid respectively and handle, comply with screening and eigenvalue and draw, be favorable to acquireing the data of examining that accord with the standard, need not the staff and manually carry out data washing, reduce the degree of automation to artifical, promote data inspection, and be favorable to promoting data inspection's efficiency and reliability.
In the embodiment of the specification, when a plurality of source system data are collected, and data structuring, compliance screening and characteristic value extraction are performed on the source system data, collection progress monitoring can be performed to obtain real-time collection degree data, and the real-time collection progress data are visually displayed based on a preset display unit, so that collection progress information can be visually obtained and displayed, and the flexibility of data inspection is improved; in some embodiments, collection error statistics may also be performed, so as to facilitate timely error processing and improve reliability of data inspection, which is not limited in this application.
In some embodiments, the method may further include:
(1) performing data archiving on the data to be inspected in the source database at intervals of first preset time according to the data importance identification of the data to be inspected in the source database;
specifically, the first preset time may be determined according to actual application requirements. In practical application, the importance of the data to be inspected can be labeled after the data to be inspected is obtained, and a data importance identifier is generated, so that the data to be inspected can be filed regularly according to the importance of the data, for example, the data can be classified and stored according to the importance, or only the key data is stored, and the data backtracking can be performed flexibly and conveniently in the follow-up process.
(2) And cleaning the data to be detected in the source database at intervals of second preset time according to the acquisition time of the data to be detected in the source database.
Specifically, the second preset time may be determined by combining actual application requirements, and in other embodiments, the multiple data to be checked may be cleaned in a hierarchical manner by combining the acquisition time of the data to be checked in the source database and the data importance identifier, for example, early data with weak importance may be cleaned first, which is beneficial to releasing the storage space of the source database, and realizing reusability of the source database, thereby improving the performance of the data checking system.
In an embodiment of this specification, referring to fig. 4, the acquiring of the inspection plan data of the target system may include:
s401: acquiring data inspection requirement information of a target system;
s403: determining target data to be checked from the source database according to the data checking demand information;
s405: acquiring corresponding inspection methods according to the data inspection requirement information, wherein the corresponding inspection methods are used as the plurality of inspection methods;
s407: respectively generating method dependent information corresponding to the corresponding inspection methods according to the data inspection requirement information;
s409: and respectively establishing mapping relations between the plurality of inspection methods and the target data to be inspected, and taking the mapping relations as the method data mapping relations.
In practical applications, because different target systems have different corresponding required data, for example, data that may be generated by personnel operations of a human resource department relates to a personnel management and development system, a learning system, and some general systems such as an reimbursement system, while data that may be generated by personnel of an IT department does not relate to a personnel management and development system, IT is necessary to determine required target data to be checked from a source database according to data checking demand information of the target system, and IT is possible to determine which checking methods need to be used for data checking (compliance audit) for the target system to determine non-compliant personnel or matters; in addition, each inspection method needs to establish a mapping relation with corresponding target data to be inspected so as to be capable of acquiring corresponding data to be inspected from a source database according to the method data mapping relation during subsequent data inspection, and the corresponding inspection method is utilized to perform data inspection. In one embodiment, the data to be inspected of the target and the corresponding inspection method can be automatically determined according to the data inspection requirement information of the target system; optionally, the auditor may also determine the required target data to be checked and the corresponding auditing method.
The data to be inspected of the target is determined from the source database according to the data inspection requirement information of the target system, and the required inspection method and the corresponding method dependence information are determined, so that the method data mapping relation is established, and the automation degree of data inspection is favorably improved.
S203: and arranging the execution sequence of the plurality of inspection methods based on the corresponding method dependence information to obtain the target execution sequence of the plurality of inspection methods.
In an embodiment of the present specification, referring to fig. 5, performing an execution order configuration on the plurality of inspection methods based on the corresponding method dependency information, and obtaining a target execution order of the plurality of inspection methods may include:
s501: determining the inspection methods depended on by the plurality of inspection methods according to the corresponding method dependence information;
in the embodiments of the present specification, due to the complexity of data inspection, there may be dependency relationship between inspection methods, for example, inspection method b depends on inspection method a, that is, inspection method b may need to perform data inspection based on the inspection result of result inspection method a, and first, the inspection method that each inspection method depends on may be determined according to the method dependency information corresponding to each inspection method.
S503: when the number of the checking methods depended on by any checking method is less than one, the checking method is configured into a first execution batch;
in one embodiment, when the number of the check methods that any check method depends on is less than one, that is, the check method does not have a dependent check method, the check method may be directly executed and configured into the first execution batch.
S505: when the checking method depended by any checking method is more than or equal to one, configuring the checking method into a second execution batch;
s507: performing linearization processing according to the method dependence information corresponding to the inspection methods in the second execution batch to obtain the execution sequence of the inspection methods in the second execution batch;
in one embodiment, when the number of the check methods depended on by any check method is greater than or equal to one, that is, the check method depended on by the check method exists, the execution of the check method needs to wait for the completion of the execution of the check method depended on by the check method to be executed, and the check method can be configured into a second execution batch. Then, the second execution batch is linearized according to the method dependency information corresponding to each inspection method, for example, each inspection method in the second execution batch is sequentially arranged after the inspection method that it depends on, and the execution order of the inspection methods in the second execution batch is obtained.
S509: and determining the target execution sequence of the plurality of inspection methods according to the execution sequence of the inspection methods in the second execution batch and the inspection methods in the first execution batch.
In one embodiment, determining the target execution order of the plurality of inspection methods according to the execution order of the inspection methods in the second execution batch and the inspection method in the first execution batch may include: the inspection methods in the first execution batch are arranged in parallel before other inspection methods, and then the inspection methods in the second execution batch are arranged in sequence after the first execution batch according to the execution sequence of the inspection methods in the second execution batch.
In a specific embodiment, for example, there are 5 inspection methods a, b, c, d and e in total, wherein the inspection methods a and b have no dependent inspection method, the inspection method c depends on the inspection methods a and d, the inspection method d depends on the inspection methods a and b, and the inspection method e depends on the inspection methods c and d, when the execution order configuration is performed on the plurality of inspection methods based on the corresponding method dependency information, the inspection methods a and b having no dependent inspection method are first configured in a first execution batch, the inspection methods c, d and e are configured in a second execution batch, and then the execution order of the inspection methods in the second execution batch is { d, c, e, finally, the execution sequence of the inspection methods in the second execution batch and the target execution sequence of the inspection methods determined by the inspection method in the first execution batch are { { a, b }, d, c, e }, i.e. the inspection methods a and b are executed in parallel, the inspection method d needs to be executed after the execution of a and b is completed, the inspection method c needs to be executed after the execution of d is completed, the inspection method e needs to be executed after the execution of c is completed, and the method waiting in execution can be realized by using a synchronous waiting algorithm.
The target execution sequence of the plurality of inspection methods is obtained by configuring the execution sequence of the plurality of inspection methods based on the corresponding method dependence information, so that the data inspection by utilizing the plurality of inspection methods is reasonably and orderly, the waiting condition is prevented, and the efficiency and the reliability of the data inspection are improved.
S205: and respectively acquiring target data to be checked corresponding to the plurality of checking methods based on the method data mapping relation.
As described above, each inspection method establishes a mapping relationship with corresponding target data to be inspected in the source database, and the target data to be inspected corresponding to each inspection method can be respectively obtained from the pre-established source database based on the above method data mapping relationship, so as to implement offline data inspection.
S207: and performing data inspection on the corresponding target data to be inspected based on the target execution sequence of the plurality of inspection methods to obtain a data inspection result of the target system.
In the embodiment of the present specification, the data inspection result may include, but is not limited to, an offender or an offender information of the target system. The corresponding data to be inspected of the target are subjected to data inspection through the target execution sequence based on the plurality of inspection methods, so that the data inspection result of the target system is obtained, and illegal items or illegal personnel in the target system can be determined so as to be convenient for subsequent corresponding processing, reduce the loss caused by the illegal items, and be beneficial to improving the automation degree of the data inspection and improving the efficiency and reliability of the data inspection.
In this embodiment of the present description, before performing data inspection on the corresponding data to be inspected of the target based on the target execution sequence of the multiple inspection methods, the method may further include performing pre-inspection on the corresponding data to be inspected of the target, specifically including data specification inspection and data integrity inspection, which is beneficial to improving the tightness and accuracy of data inspection.
As shown in fig. 6, in this embodiment, when it is determined that there is more than one target data to be inspected corresponding to any inspection method, the performing data inspection on the corresponding target data to be inspected based on the target execution sequence of the plurality of inspection methods may include:
s601: performing data association on more than one data to be inspected corresponding to the inspection method according to preset parameters to obtain associated data to be inspected;
in practical applications, in some embodiments, an inspection method may perform data inspection only on data to be inspected of a target, for example, if an inspection method wishes to inspect whether a person in a certain door checks attendance on time, only the data to be inspected of the attendance system needs to be acquired for data inspection; however, in other embodiments, an inspection method may also rely on a plurality of target data to be inspected for data inspection, for example, an inspection method is desired to inspect a certain department of personnel for compliance reimbursement, and can acquire the data to be inspected of the purchasing system and correlate the data to be inspected of the reimbursement system, for example, whether the reimbursement application reported by an employee has a corresponding record in the data of the purchasing system, and whether the data can be correlated in time, etc.
Specifically, the preset parameter may be determined according to actual data inspection requirements of each inspection method, and in an embodiment of the present specification, the preset parameter may be the same entry among the data to be inspected, for example, a user ID and a timestamp, and data association may be performed on the more than one data to be inspected according to the user ID and the timestamp, for example, behavior data generated by the same user in two systems to be inspected in a preset time period is determined to perform compliance audit, and a record of non-compliance is determined.
In another embodiment, performing data association on more than one data to be inspected corresponding to the inspection method according to preset parameters may further include performing time marking on the target data to be inspected by using time information, determining a range of the data to be inspected by using an intersection of the time marks, and subsequently performing the inspection method according to the range of the data to be inspected, so as to screen out invalid data and improve the efficiency of data inspection. For example, data inspection is performed on a certain department, one inspection method needs to use data to be inspected of the system a and the system B for three months of 7, 8 and 9, but data acquisition is incomplete or some data do not exist, for example, the system a only has data of 7 and 8 months, the time mark of the system a is 7-8 months, the system B only has data of 8 and 9 months, the time mark of the system a is 8-9 months, the range of the data to be inspected determined by the intersection of the time marks is 8 months, and data inspection only needs to be performed on the data of the system AB for 8 months based on the corresponding inspection method.
S603: and performing data inspection on the associated data to be inspected based on the target execution sequence of the plurality of inspection methods.
By determining the associated data to be inspected corresponding to each inspection method and then performing data inspection on the associated data to be inspected based on the corresponding inspection method, the efficiency and reliability of data inspection are improved, management personnel do not need to manually associate the data, dependence on manual work is reduced, and the automation degree of the data inspection is improved.
As shown in fig. 7, in an embodiment of the present specification, the method may further include:
s701: acquiring at least one piece of data check rule information;
s703: generating a corresponding checking method according to the at least one piece of data checking rule information;
s705: establishing a rule mapping relation between the at least one piece of data inspection rule information and the corresponding inspection method;
in the embodiment of the present specification, a corresponding inspection method needs to be generated according to at least one piece of data inspection rule information, and a rule mapping relationship between the data inspection rule information and the corresponding inspection method is established, so that the data inspection rule information is subsequently used to perform a normative inspection on the corresponding inspection method. In practical applications, for example, the data check rule information may include that a company person needs to log in at a preset location, and login in a different place belongs to an illegal operation, the generated corresponding check method may include obtaining a login IP of a user, determining whether a preset range requirement is met, and outputting whether the user violates a rule, and in practical applications, when a more complicated data check is performed, a corresponding check method may need to be generated according to a plurality of data check rule information, and the check method may include a plurality of steps, so it is necessary to perform a normative check on the check method according to the corresponding data check rule information before executing the check method.
Accordingly, as shown in fig. 8, before performing the sequential configuration of the plurality of inspection methods based on the corresponding method-dependent information, the method may further include:
s801: and performing normative check on the plurality of checking methods according to the rule mapping relation.
When the normative check is passed, the plurality of checking methods are sequentially arranged in execution based on the corresponding method dependency information.
In practical application, the normative checking of the plurality of checking methods according to the rule mapping relationship may include determining whether each checking method satisfies corresponding data checking rule information, whether each checking method is legal, whether uniform calling interface information exists, and the like.
In an embodiment of the present specification, the method may further include:
(1) when the data inspection is carried out on the corresponding data to be inspected of the target based on the target execution sequence of the plurality of inspection methods, monitoring the inspection progress to obtain real-time inspection progress data;
(2) and visually displaying the real-time inspection progress data based on a preset display unit.
In the embodiment of the specification, the real-time inspection progress data are obtained by monitoring the inspection progress and are visually displayed, so that the inspection progress information can be visually obtained and displayed, and the flexibility of data inspection is improved; in some embodiments, method execution time statistics can be performed, which is beneficial to integrating execution conditions after data inspection is completed, and facilitates subsequent data backtracking and analysis, so as to adjust corresponding inspection methods or related parameters and improve reliability of data inspection. In some embodiments, the method for checking the abnormality is stored on site and corresponding warning information is sent according to the severity of the abnormality and the pre-configured parameters, so that error processing can be performed in time, and the reliability of data checking can be improved.
In the embodiment of the present specification, the inspection result database may be constructed based on the data inspection result obtained after the data inspection, and the data in the inspection result database may also be managed by data archiving or cleaning.
In some embodiments, the method may further include:
(1) performing data archiving on the data inspection results in the inspection result database every third preset time according to the importance identification of the data inspection results in the inspection result database;
specifically, the third preset time may be determined according to actual application requirements. In practical application, the data inspection result can be subjected to importance labeling after being obtained to obtain a corresponding importance identifier, for example, the data inspection result generated by some methods is only used for data inspection by other methods, but not used as a final data inspection result, so that the importance of the data inspection result is weaker, and the data inspection result can be subjected to corresponding labeling (for example, distinguishing on naming) so as to regularly archive the data in the inspection result database according to the importance of the data, for example, the data can be classified and stored according to the importance, or only the key data is stored, which is beneficial to flexibly and conveniently backtracking the data subsequently, reduces the complexity of data management on the inspection result database, and improves the quality of data management.
(2) And cleaning the data inspection results in the inspection result database every fourth preset time according to the acquisition time of the data inspection results in the inspection result database.
Specifically, the fourth preset time may be determined in combination with an actual application requirement, and in some embodiments, the data in the inspection result database may be cleaned in a hierarchical manner in combination with the acquisition time and the importance identifier of the data inspection result in the inspection result database, for example, early data with weak importance may be cleaned first, which is beneficial to release of a storage space of the inspection result database, implementation of reuse of the inspection result database, and improvement of performance of the data inspection system.
In the embodiment of the present specification, after the data inspection result is obtained, a mapping relationship between the data inspection result and the corresponding inspection method may also be established, and a mapping relationship between the data inspection result and the corresponding data to be inspected may also be established, so that the corresponding inspection method or the data to be inspected may be determined to determine the usage records of different inspection methods according to the mapping relationships, and thus data backtracking may be performed (for example, it is determined whether the mapping between the inspection method and the data is reasonable), which is beneficial to improving the reliability of data inspection.
In this embodiment, in order to improve timeliness of data inspection, in some embodiments, when an update message of target data to be inspected is received during data inspection of the corresponding target data to be inspected based on a target execution sequence of a plurality of inspection methods, the updated target data to be inspected may be acquired, the updated target data to be inspected is subjected to data inspection based on the target execution sequence of the plurality of inspection methods, an originally obtained data inspection result is covered with an obtained new data inspection result, and the inspection result database is updated according to the new data inspection result.
In some embodiments, a complex inspection method may include many steps of operations, in order to reduce the computational complexity of the inspection method, the inspection method may be split into a plurality of sub-inspection methods, a method dependency is established by the plurality of sub-inspection methods to implement the entire complex inspection method, and output data of a previous sub-inspection method is used as input data of a subsequent sub-inspection method depending on the previous sub-inspection method, which is beneficial to improving data processing efficiency, thereby improving data inspection efficiency. In addition, similar operations may also be abstracted to one inspection method to provide result data to other inspection methods for further computation.
In other embodiments, when the target data to be inspected corresponding to the plurality of inspection methods are obtained, the data may be loaded as needed, for example, after the target data to be inspected corresponding to one inspection method is obtained, data inspection is performed first, and after a corresponding data inspection result is obtained, the data to be inspected for the target by the next inspection method is continuously loaded.
In the embodiment of the specification, the acquisition of the data to be inspected which meet the specifications is facilitated by acquiring the data of the plurality of source systems and respectively carrying out data structuring processing, compliance screening and characteristic value extraction on the data of the plurality of source systems, the data to be inspected which meet the specifications is not required to be manually cleaned by workers, the dependence on the manual work is reduced, the automation degree of data inspection is improved, and the efficiency and the reliability of the data inspection are improved; the execution sequence configuration is carried out on the plurality of checking methods based on the corresponding method dependence information to obtain the target execution sequence of the plurality of checking methods, so that the data checking is favorably and orderly carried out by utilizing the plurality of checking methods, the waiting condition is prevented, and the data checking efficiency is improved; the data to be checked of the corresponding target is obtained based on the data mapping relation of the method, the data to be checked of the corresponding target is checked based on the target execution sequence of the multiple checking methods, the data checking result of the target system is obtained, and violation matters or violation personnel in the target system can be determined so as to be convenient for subsequent corresponding processing, reduce the loss caused by the violation matters and be beneficial to improving the efficiency and reliability of data checking; by determining the associated data to be inspected corresponding to each inspection method and then performing data inspection on the associated data to be inspected based on the corresponding inspection method, the efficiency and reliability of data inspection are improved, management personnel do not need to manually associate the data, dependence on manual work is reduced, and the automation degree of the data inspection is improved; the data in the source database and the data in the inspection result database are filed, so that the data backtracking can be flexibly and conveniently carried out subsequently, the data in the source database and the data in the inspection result database can be cleaned, the storage space of the source database or the inspection result database can be released, the source database or the inspection result database can be reused, and the performance of the data inspection system can be improved.
An embodiment of the present application further provides a data checking apparatus, as shown in fig. 9, the apparatus may include:
a plan data obtaining module 910, configured to obtain, in response to a data inspection instruction, inspection plan data of a target system, where the inspection plan data includes a plurality of inspection methods and corresponding method dependency information, and a method data mapping relationship;
an execution sequence configuration module 920, configured to perform execution sequence configuration on the multiple inspection methods based on the corresponding method dependency information, so as to obtain target execution sequences of the multiple inspection methods;
a to-be-inspected data obtaining module 930, configured to obtain to-be-inspected data of the targets corresponding to the plurality of inspection methods respectively based on the method data mapping relationships;
a data checking module 940, configured to perform data checking on the corresponding target data to be checked based on the target execution sequence of the multiple checking methods, so as to obtain a data checking result of the target system.
Specifically, the execution sequence configuration module 920 may include:
a dependent method determination unit configured to determine, based on the corresponding method dependent information, inspection methods on which the plurality of inspection methods depend, respectively;
a first sequential configuration unit, configured to configure, when any inspection method depends on less than one inspection method, the dependent inspection method less than one corresponding inspection method into a first execution batch;
a second sequential configuration unit, configured to configure, when the number of the check methods depended on by any check method is greater than or equal to one, a corresponding check method that is greater than or equal to one of the dependent check methods into a second execution batch;
the linearization unit is used for carrying out linearization processing on the inspection methods in the second execution batch according to the method dependence information corresponding to the inspection methods in the second execution batch to obtain the execution sequence of the inspection methods in the second execution batch;
a target execution order determination unit configured to determine a target execution order of the plurality of inspection methods according to the execution order of the inspection methods in the second execution lot and the inspection methods in the first execution lot.
In some embodiments, the apparatus may further include:
the source system data acquisition module is used for acquiring a plurality of source system data before responding to the data inspection instruction and acquiring the inspection scheme data of the target system;
the structured processing module is used for respectively carrying out data structured processing on the source system data to obtain corresponding structured data;
the compliance screening module is used for respectively carrying out compliance screening on the corresponding structured data to obtain corresponding compliance data;
the characteristic value extraction module is used for respectively extracting the characteristic values of the corresponding compliance data to obtain a plurality of data to be checked;
and the source database construction module is used for constructing a source database based on the plurality of data to be checked.
In some embodiments, the scheme data acquiring module 910 may include:
a requirement information acquisition unit for acquiring data inspection requirement information of a target system;
the target data to be inspected determining unit is used for determining target data to be inspected from the source database according to the data inspection requirement information;
an inspection method acquisition unit configured to acquire a corresponding inspection method as the plurality of inspection methods according to the data inspection requirement information;
a method dependency information generating unit, configured to generate method dependency information corresponding to the corresponding inspection methods according to the data inspection requirement information;
and the method data mapping relation establishing unit is used for respectively establishing the mapping relation between the plurality of inspection methods and the target data to be inspected, and taking the mapping relation as the method data mapping relation.
In some embodiments, when it is determined that there is more than one target data to be checked corresponding to any checking method, the data checking module 940 may include:
the data association unit is used for performing data association on more than one data to be inspected corresponding to the inspection method according to preset parameters to obtain associated data to be inspected;
and the data checking unit is used for performing data checking on the associated data to be checked based on the target execution sequence of the plurality of checking methods.
In some embodiments, the apparatus may further include:
the inspection rule information acquisition module is used for acquiring at least one piece of data inspection rule information;
the checking method generating module is used for generating a corresponding checking method according to the at least one piece of data checking rule information;
a rule mapping relationship establishing module, configured to establish a rule mapping relationship between the at least one piece of data inspection rule information and the corresponding inspection method;
correspondingly, the above apparatus may further include:
and a method normative checking module, configured to perform normative checking on the plurality of checking methods according to the rule mapping relationship before performing sequential configuration on the plurality of checking methods based on the corresponding method dependency information, and when the normative checking passes, perform the data checking module 940.
In some embodiments, the apparatus may further include:
the data archiving module to be checked is used for archiving the data to be checked in the source database at intervals of first preset time according to the data importance identification of the data to be checked in the source database;
and the to-be-inspected data cleaning module is used for cleaning the data to be inspected in the source database at intervals of second preset time according to the acquisition time of the data to be inspected in the source database.
The device and method embodiments in the device embodiment are based on the same application concept.
The embodiment of the present application provides a computer device, which includes a processor and a memory, where at least one instruction or at least one program is stored in the memory, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the data checking method provided by the above method embodiment.
The memory may be used to store software programs and modules, and the processor may execute various functional applications and data processing by operating the software programs and modules stored in the memory. The memory can mainly comprise a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs needed by functions and the like; the storage data area may store data created according to use of the apparatus, and the like. Further, the memory may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory may also include a memory controller to provide the processor access to the memory.
The method embodiments provided in the embodiments of the present application may be executed in a mobile terminal, a computer terminal, a server, or a similar computing device, that is, the computer device may include a mobile terminal, a computer terminal, a server, or a similar computing device. The server may be an independent physical server, or may be a server cluster or a distributed system formed by a plurality of physical servers. To operate inFor example, fig. 10 is a block diagram of a hardware structure of a server for implementing the data checking method according to an embodiment of the present application. As shown in fig. 10, the server 1000 may have a relatively large difference due to different configurations or performances, and may include one or more Central Processing Units (CPUs) 1010 (the processor 1010 may include but is not limited to a Processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 1030 for storing data, and one or more storage media 1020 (e.g., one or more mass storage devices) for storing applications 1023 or data 1022. Memory 1030 and storage media 1020 may be, among other things, transient or persistent storage. The program stored in the storage medium 1020 may include one or more modules, each of which may include a series of instruction operations for a server. Still further, the central processor 1010 may be configured to communicate with the storage medium 1020 and execute a series of instruction operations in the storage medium 1020 on the server 1000. The Server 1000 may also include one or more power supplies 1060, one or more wired or wireless network interfaces 1050, one or more input-output interfaces 1040, and/or one or more operating systems 1021, such as a Windows ServerTM，Mac OS XTM，UnixTM,LinuxTM，FreeBSDTMAnd so on.
The Processor 1010 may be an integrated circuit chip having Signal processing capabilities, such as a general purpose Processor, a Digital Signal Processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like, wherein the general purpose Processor may be a microprocessor or any conventional Processor, or the like.
Input-output interface 1040 may be used to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the server 1000. In one example, i/o Interface 1040 includes a Network adapter (NIC) that may be coupled to other Network devices via a base station to communicate with the internet. In one example, the input/output interface 1040 may be a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.
The operating system 1021 may include system programs for handling various basic system services and performing hardware related tasks, such as framework layer, core library layer, driver layer, etc., for implementing various underlying services and handling hardware based tasks.
It will be understood by those skilled in the art that the structure shown in fig. 10 is merely illustrative and is not intended to limit the structure of the electronic device. For example, server 1000 may also include more or fewer components than shown in FIG. 10, or have a different configuration than shown in FIG. 10.
Embodiments of the present application further provide a computer-readable storage medium, where the storage medium may be disposed in a server to store at least one instruction or at least one program for implementing a data checking method in the method embodiments, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the data checking method provided in the method embodiments.
Alternatively, in this embodiment, the storage medium may be located in at least one network server of a plurality of network servers of a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
As can be seen from the embodiments of the data inspection method, the data inspection device, the data inspection equipment or the storage medium provided by the application, in the application, by acquiring a plurality of source system data, and performing data structuring processing, compliance screening and characteristic value extraction on the plurality of source system data, the data to be inspected which meets the specifications can be acquired conveniently, and the data cleaning by workers is not needed, so that the dependence on the manual work is reduced, the automation degree of the data inspection is improved, and the efficiency and the reliability of the data inspection are improved; the execution sequence configuration is carried out on the plurality of checking methods based on the corresponding method dependence information to obtain the target execution sequence of the plurality of checking methods, so that the data checking is favorably and orderly carried out by utilizing the plurality of checking methods, the waiting condition is prevented, and the data checking efficiency is improved; the data to be checked of the corresponding target is obtained based on the data mapping relation of the method, the data to be checked of the corresponding target is checked based on the target execution sequence of the multiple checking methods, the data checking result of the target system is obtained, and violation matters or violation personnel in the target system can be determined so as to be convenient for subsequent corresponding processing, reduce the loss caused by the violation matters and be beneficial to improving the efficiency and reliability of data checking; by determining the associated data to be inspected corresponding to each inspection method and then performing data inspection on the associated data to be inspected based on the corresponding inspection method, the efficiency and reliability of data inspection are improved, management personnel do not need to manually associate the data, dependence on manual work is reduced, and the automation degree of the data inspection is improved; the data in the source database and the data in the inspection result database are filed, so that the data backtracking can be flexibly and conveniently carried out subsequently, the data in the source database and the data in the inspection result database can be cleaned, the storage space of the source database or the inspection result database can be released, the source database or the inspection result database can be reused, and the performance of the data inspection system can be improved.
It should be noted that: the sequence of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. And specific embodiments thereof have been described above. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the apparatus, device and storage medium embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference may be made to some descriptions of the method embodiments for relevant points.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is only exemplary of the present application and should not be taken as limiting the present application, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims (10)

1. A method of data inspection, the method comprising:
responding to a data inspection instruction, acquiring inspection scheme data of a target system, wherein the inspection scheme data comprises a plurality of inspection methods, corresponding method dependence information and a method data mapping relation;
performing execution sequence configuration on the plurality of inspection methods based on the corresponding method dependence information to obtain target execution sequences of the plurality of inspection methods;
respectively acquiring target data to be inspected corresponding to the plurality of inspection methods based on the method data mapping relation;
and performing data inspection on the corresponding target data to be inspected based on the target execution sequence of the plurality of inspection methods to obtain a data inspection result of the target system.
2. The method of claim 1, wherein the configuring the execution order of the plurality of inspection methods based on the corresponding method dependency information, and obtaining the target execution order of the plurality of inspection methods comprises:
determining the inspection methods on which the plurality of inspection methods depend respectively according to the corresponding method-dependent information;
when the number of the checking methods depended on by any checking method is less than one, configuring the checking methods depended on by less than one corresponding checking method into a first execution batch;
when the number of the checking methods depended on by any checking method is more than or equal to one, configuring the checking methods depended on by more than or equal to one corresponding checking method into a second execution batch;
according to the method dependency information corresponding to the inspection methods in the second execution batch, carrying out linearization processing on the inspection methods in the second execution batch to obtain the execution sequence of the inspection methods in the second execution batch;
determining a target execution order of the plurality of inspection methods according to the execution order of the inspection methods in the second execution batch and the inspection methods in the first execution batch.
3. The method of claim 1, further comprising:
acquiring a plurality of source system data before acquiring inspection plan data of a target system in response to a data inspection instruction;
respectively carrying out data structuring processing on the source system data to obtain corresponding structured data;
respectively carrying out compliance screening on the corresponding structured data to obtain corresponding compliance data;
respectively extracting characteristic values of the corresponding compliance data to obtain a plurality of data to be checked;
and constructing a source database based on the plurality of data to be checked.
4. The method of claim 3, wherein the obtaining inspection plan data for the target system comprises:
acquiring data inspection requirement information of a target system;
determining target data to be inspected from the source database according to the data inspection requirement information;
acquiring corresponding inspection methods according to the data inspection requirement information, and taking the corresponding inspection methods as the plurality of inspection methods;
respectively generating method dependent information corresponding to the corresponding checking methods according to the data checking demand information;
and respectively establishing mapping relations between the plurality of inspection methods and the target data to be inspected, and taking the mapping relations as the method data mapping relations.
5. The method according to claim 1, wherein when it is determined that there is more than one target data to be checked corresponding to any one checking method, the performing data checking on the corresponding target data to be checked based on the target execution sequence of the plurality of checking methods comprises:
performing data association on more than one data to be inspected corresponding to the inspection method according to preset parameters to obtain associated data to be inspected;
and performing data inspection on the associated data to be inspected based on the target execution sequence of the plurality of inspection methods.
6. The method of claim 1, further comprising:
acquiring at least one piece of data check rule information;
generating a corresponding checking method according to the at least one piece of data checking rule information;
establishing a rule mapping relation between the at least one piece of data inspection rule information and the corresponding inspection method;
accordingly, before performing a sequential configuration of the plurality of inspection methods based on the corresponding method-dependent information, the method further comprises:
carrying out normative check on the plurality of checking methods according to the rule mapping relation;
when the normative check passes, performing sequential configuration on the plurality of checking methods based on the corresponding method dependency information.
7. The method of claim 3, further comprising:
performing data archiving on the data to be inspected in the source database at intervals of first preset time according to the data importance identification of the data to be inspected in the source database;
and cleaning the data to be detected in the source database at intervals of second preset time according to the acquisition time of the data to be detected in the source database.
8. A data inspection apparatus, characterized in that the apparatus comprises:
the system comprises a scheme data acquisition module, a data processing module and a data processing module, wherein the scheme data acquisition module is used for responding to a data inspection instruction and acquiring inspection scheme data of a target system, and the inspection scheme data comprises a plurality of inspection methods, corresponding method dependence information and a method data mapping relation;
an execution sequence configuration module, configured to perform execution sequence configuration on the plurality of inspection methods based on the corresponding method dependency information, so as to obtain target execution sequences of the plurality of inspection methods;
the data to be inspected acquisition module is used for respectively acquiring the data to be inspected of the targets corresponding to the plurality of inspection methods based on the method data mapping relation;
and the data inspection module is used for performing data inspection on the corresponding target data to be inspected based on the target execution sequence of the plurality of inspection methods to obtain a data inspection result of the target system.
9. A data inspection apparatus, characterized in that the apparatus comprises a processor and a memory, in which at least one instruction or at least one program is stored, which is loaded and executed by the processor to implement the data inspection method according to any one of claims 1 to 7.
10. A computer-readable storage medium, wherein at least one instruction or at least one program is stored in the storage medium, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the data inspection method according to any one of claims 1 to 7.
CN202011311730.5A 2020-11-20 2020-11-20 Data checking method, device, equipment and storage medium Pending CN112463829A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011311730.5A CN112463829A (en) 2020-11-20 2020-11-20 Data checking method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011311730.5A CN112463829A (en) 2020-11-20 2020-11-20 Data checking method, device, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN112463829A true CN112463829A (en) 2021-03-09

Family

ID=74798339

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011311730.5A Pending CN112463829A (en) 2020-11-20 2020-11-20 Data checking method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN112463829A (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102096857A (en) * 2010-12-27 2011-06-15 大唐软件技术股份有限公司 Collaboration method and device for data processing process
US20180293147A1 (en) * 2017-04-06 2018-10-11 Dell Products, Lp Autonomous Information Technology Diagnostic Checks
CN110019566A (en) * 2019-03-13 2019-07-16 平安信托有限责任公司 Data checking, device, computer equipment and storage medium based on data warehouse
CN110795479A (en) * 2019-10-08 2020-02-14 中国建设银行股份有限公司 Method and device for distributed ETL scheduling based on data
CN111950988A (en) * 2020-08-18 2020-11-17 北京字节跳动网络技术有限公司 Distributed workflow scheduling method and device, storage medium and electronic equipment

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102096857A (en) * 2010-12-27 2011-06-15 大唐软件技术股份有限公司 Collaboration method and device for data processing process
US20180293147A1 (en) * 2017-04-06 2018-10-11 Dell Products, Lp Autonomous Information Technology Diagnostic Checks
CN110019566A (en) * 2019-03-13 2019-07-16 平安信托有限责任公司 Data checking, device, computer equipment and storage medium based on data warehouse
CN110795479A (en) * 2019-10-08 2020-02-14 中国建设银行股份有限公司 Method and device for distributed ETL scheduling based on data
CN111950988A (en) * 2020-08-18 2020-11-17 北京字节跳动网络技术有限公司 Distributed workflow scheduling method and device, storage medium and electronic equipment

Similar Documents

Publication Publication Date Title
CN111078140B (en) Nuclear power station file uploading management method and device, terminal equipment and medium
KR20060061759A (en) Automatic validation and calibration of transaction-based performance models
CN108388507A (en) Batch script testing method and device
CN101206569A (en) Method, system and program product for dynamically identifying components contributing to service degradation
EP1631002A2 (en) Automatic configuration of network performance models
CN111400189A (en) Code coverage rate monitoring method and device, electronic equipment and storage medium
CN109598486A (en) A kind of method and apparatus for checking abnormal order
CN110928681A (en) Data processing method and device, storage medium and electronic device
CN113760677A (en) Abnormal link analysis method, device, equipment and storage medium
CN114564294A (en) Intelligent service arranging method and device, computer equipment and storage medium
CN112948275A (en) Test data generation method, device, equipment and storage medium
CN111459629A (en) Azkaban-based project operation method and device and terminal equipment
CN111061733A (en) Data processing method and device, electronic equipment and computer readable storage medium
JP2017016507A (en) Test management system and program
CN113282308A (en) Mirror image construction method and device, storage medium and electronic device
CN113094224B (en) Server asset management method and device, computer equipment and storage medium
CN115471215B (en) Business process processing method and device
CN115147031B (en) Clearing workflow execution method, device, equipment and medium
CN112463829A (en) Data checking method, device, equipment and storage medium
CN115344582A (en) Metadata management method and device, electronic equipment and computer storage medium
CN110096638B (en) Irrigated area meteorological and remote sensing information monitoring platform
CN112948131A (en) Data acquisition method and equipment
CN112070391A (en) Workshop production task management system, method and device
CN112734359A (en) Science and technology project progress tracking system
CN112148459B (en) Processing method, device, readable medium and equipment for node association data

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination