CN116739427A - Data quality link management method, device, equipment and storage medium - Google Patents

Data quality link management method, device, equipment and storage medium Download PDF

Info

Publication number
CN116739427A
CN116739427A CN202310738838.XA CN202310738838A CN116739427A CN 116739427 A CN116739427 A CN 116739427A CN 202310738838 A CN202310738838 A CN 202310738838A CN 116739427 A CN116739427 A CN 116739427A
Authority
CN
China
Prior art keywords
data
checking
target
rule
check
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310738838.XA
Other languages
Chinese (zh)
Inventor
彭薇雯
王雪峰
李荣荣
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Construction Bank Corp
CCB Finetech Co Ltd
Original Assignee
China Construction Bank Corp
CCB Finetech Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Construction Bank Corp, CCB Finetech Co Ltd filed Critical China Construction Bank Corp
Priority to CN202310738838.XA priority Critical patent/CN116739427A/en
Publication of CN116739427A publication Critical patent/CN116739427A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063—Operations research, analysis or management
    • G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
    • G06Q10/06395—Quality analysis or management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/10—Office automation; Time management
    • G06Q10/103—Workflow collaboration or project management

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Human Resources & Organizations (AREA)
  • Strategic Management (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Quality & Reliability (AREA)
  • Economics (AREA)
  • Physics & Mathematics (AREA)
  • Operations Research (AREA)
  • Tourism & Hospitality (AREA)
  • Marketing (AREA)
  • General Business, Economics & Management (AREA)
  • Educational Administration (AREA)
  • Development Economics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Game Theory and Decision Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The application provides a data quality link management method, a device, equipment and a storage medium. The method comprises the steps of determining target checking data to be checked, and determining a plurality of checking rules corresponding to the target checking data and rule grades corresponding to each checking rule; sequentially carrying out the checking of each checking rule on the target checking data to obtain the checking result corresponding to each checking rule; if the checking result of the checking rule at the target rule level shows that the checking is failed, determining that problem data exists in the target checking data, stopping executing tasks related to the target checking data, and automatically generating a data quality work order corresponding to the target checking data; and sending the data quality worksheet to a responsible department for processing. The application can effectively optimize the process of data quality management and improve the stability and reliability of the service by distinguishing high-level and low-level checking rules and carrying out targeted processing aiming at different conditions.

Description

Data quality link management method, device, equipment and storage medium
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a method, an apparatus, a device, and a storage medium for managing a data quality link.
Background
As a core resource in the digital age, data has become an important foundation and support for various industries of the economic society. From the commercial field to the fields of scientific research, medical treatment and health, the data play a significant role, and the value and the potential of the data asset can be really realized only through effective data quality management, so that the digital transformation and innovation development are promoted. With the rise and application of big data technology, the scale and variety of data become more and more diversified and complicated, the problem of data quality is also more and more prominent, and how to manage the data quality has become an urgent need for various industries.
Disclosure of Invention
The application provides a data quality link management method, a device, equipment and a storage medium, which are used for solving the problems of data quality checking flow and subsequent rectifying flow splitting in the prior art.
In a first aspect, the present application provides a method for managing a data quality link, which is characterized by comprising: determining target checking data to be checked, and determining a plurality of checking rules corresponding to the target checking data and rule grades corresponding to each checking rule; sequentially carrying out the checking of each checking rule on the target checking data according to a plurality of checking rules corresponding to the target checking data, and obtaining the checking result corresponding to each checking rule; if the checking result of the checking rule at the target rule level shows that the checking is failed, determining that problem data exists in the target checking data, stopping executing tasks related to the target checking data, and automatically generating a data quality work order corresponding to the target checking data; and sending the data quality worksheet to a responsibility department corresponding to the target checking data for processing.
In a second aspect, the present application provides a data quality link management apparatus, comprising: the determining module is used for determining target checking data to be checked, and determining a plurality of checking rules corresponding to the target checking data and rule grades corresponding to each checking rule; the checking module is used for sequentially checking each checking rule for the target checking data according to a plurality of checking rules corresponding to the target checking data, and acquiring a checking result corresponding to each checking rule; the judging module is used for determining that problem data exists in the target checking data if the checking result of the checking rule at the target rule level shows that the checking is failed, stopping executing tasks related to the target checking data and automatically generating a data quality work order corresponding to the target checking data; and the processing module is used for sending the data quality worksheet to a responsibility department corresponding to the target checking data for processing.
In a third aspect, the present application provides an electronic device comprising: a processor, and a memory communicatively coupled to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored by the memory to implement the data quality link management method as previously described.
In a fourth aspect, a computer readable storage medium is provided, which when stored with computer executable instructions for implementing the data quality link management method as described above when executed by a processor.
In a fifth aspect, a computer program product is provided, comprising a computer program/instruction, characterized in that the computer program/instruction, when executed by a processor, implements a data quality link management method as described above.
According to the data quality link management method, the device, the equipment and the storage medium, through distinguishing the high-level and low-level checking rules, targeted processing is carried out aiming at different conditions, the data quality management process can be effectively optimized, the stability and the reliability of the service are improved, and more commercial values are brought for the development of enterprises.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
Fig. 1 is a schematic diagram of an exemplary implementation of a data quality link management method according to an embodiment of the present application.
Fig. 2 is a schematic diagram of a blocking mechanism according to an embodiment of the present application.
Fig. 3 is a schematic diagram of an exemplary implementation of another data quality link management method according to an embodiment of the present application.
Fig. 4 is a flow chart of a data quality link management method according to an embodiment of the present application.
FIG. 5 is a flow interface schematic of a data quality worksheet of the present application.
Fig. 6 is a schematic diagram of a data quality link management apparatus according to an embodiment of the present application.
Fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Specific embodiments of the present application have been shown by way of the above drawings and will be described in more detail below. The drawings and the written description are not intended to limit the scope of the inventive concepts in any way, but rather to illustrate the inventive concepts to those skilled in the art by reference to the specific embodiments.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the application. Rather, they are merely examples of apparatus and methods consistent with aspects of the application as detailed in the accompanying claims.
It should be noted that, in the present disclosure, all the actions of obtaining information and authority and providing services are performed under the premise of conforming to the corresponding data protection rule policy of the country of the location and obtaining the authorization given by the owner of the corresponding device, which all conform to the rules of the related laws and regulations and do not violate the popular regulations.
Term interpretation:
the number blocking mechanism: in data checking, the use of a number blocking mechanism refers to detecting, verifying and filtering input data by using a digital technology or algorithm to ensure the accuracy and integrity of the data and prevent malicious or erroneous data input. The number blocking mechanism can help identify and intercept the problems of incorrect format, out-of-range value, data conflict, abnormal data and the like, and prevent the data from entering the subsequent processing flow. At the same time, the number blocking mechanism may also help improve data quality, reducing errors, fraud, and other problems.
Fig. 1 is a schematic diagram of an exemplary embodiment of a data quality link management method according to the present application, as shown in fig. 1, comprising the steps of:
s101, determining target checking data to be checked, and determining a plurality of checking rules corresponding to the target checking data and rule grades corresponding to each checking rule.
And determining which data to be checked exist in the service or the system to be used as target checking data to be checked, such as client information, order information, transaction information and the like.
And according to the characteristics of the target checking purpose and the target checking data, a plurality of checking rules corresponding to the target checking data are formulated. For example, for checking a mobile phone number, rules may be formulated as follows: must be an 11-bit number; the first digit must be a 1, etc.
The classification of the checking rules is based on the importance and urgency of the checking rules, and may be classified into a high class, a medium class, and a low class, for example. For example, it is necessary to check the postal code field of order information in the e-commerce website, and if this field is not filled, it belongs to the severity level; if the zip code is incorrect, it is of medium grade.
S102, sequentially checking each checking rule for the target checking data according to a plurality of checking rules corresponding to the target checking data, and obtaining the checking result corresponding to each checking rule.
And sequentially carrying out the checking of each checking rule on the target checking data to obtain the checking result corresponding to each checking rule. The checking result comprises passing checking and failing checking.
In the application, when the target checking data is checked by each checking rule in turn, offline checking, stream data checking and the like are supported. For service data with higher importance, real-time acquisition and real-time checking processing are necessary, and the real-time checking processing of the data can be realized through the real-time acquisition technology such as the FlinkCDC on the system realization.
And S103, if the check result of the check rule with the target rule level shows that the check fails, determining that problem data exists in the target check data, stopping executing tasks related to the target check data, and automatically generating a data quality work order corresponding to the target check data.
And determining all the checking rules which show that the checking is failed from the checking result, and acquiring rule grades of the checking rules which show that the checking is failed.
High-level check rules generally represent critical paths or key links of a business process, and the occurrence of data anomalies can seriously affect the whole business system. In the application, setting a high-level rule level as a target rule level, if a check result of a check rule with a high level shows that the check is failed, determining that problem data exists in target check data, on one hand, automatically generating a data quality work order aiming at the target check data according to the characteristics of the problem data, wherein the data quality work order comprises information such as problem data description, a data processing scheme, a responsible person and the like; on the other hand, execution of the task associated with the target check data is stopped to prevent the spread of the dirty data influence range.
And S104, sending the data quality worksheet to a responsibility department corresponding to the target checking data for processing.
And calling the responsibility person information of the data asset, pushing the data quality work order to related personnel of a corresponding responsibility department for processing, and enabling the data quality responsibility person and related processing personnel to carry out task circulation on the work order model on the processing task of the data quality problem. The responsibility department can process the problem data in time according to the generated data quality work order, including repairing, re-collecting, adjusting rules and other operations on the data.
The embodiment of the application provides a data quality link management method, which comprises the steps of determining target checking data to be checked, and determining a plurality of checking rules corresponding to the target checking data and rule grades corresponding to each checking rule; sequentially carrying out the checking of each checking rule on the target checking data according to a plurality of checking rules corresponding to the target checking data, and obtaining the checking result corresponding to each checking rule; if the checking result of the checking rule at the target rule level shows that the checking is failed, determining that problem data exists in the target checking data, stopping executing tasks related to the target checking data, and automatically generating a data quality work order corresponding to the target checking data; and sending the data quality worksheet to a responsibility department corresponding to the target checking data for processing. In the application, the checking rules are classified, and the middle and low level checking rules are relatively loose, so that problems of routine and secondary can be processed in the later stage, no obvious influence is generated on the service, and the operation of the whole service system is not required to be influenced for a small problem, so that when the middle and low level checking rules fail, the task associated with the target checking data is not required to be immediately stopped, the high level checking rules usually represent the key path or key link of the service flow, the serious influence can be caused on the whole service system due to the abnormal data, if the checking result of the high level checking rules shows that the checking result does not pass, the problem data exists in the target checking data, and the task associated with the target checking data must be immediately stopped to prevent the spread of the influence range of the dirty data, and the service risk and loss are avoided. The method for classifying the processing and checking rules aims at improving the efficiency and accuracy of data processing and provides better guarantee and support for the business process. By distinguishing high-level and low-level checking rules, targeted processing is performed aiming at different conditions, the process of data quality management can be effectively optimized, the stability and reliability of the service are improved, and more commercial value is brought to the development of enterprises.
Further, fig. 2 is a schematic diagram of a blocking mechanism shown in the present application, and as shown in fig. 2, a plurality of checking rules corresponding to the target checking data are obtained from a checking rule base, and checking of each checking rule is sequentially performed on the target checking data, so as to obtain a checking result corresponding to each checking rule.
And analyzing all the checking results, if the checking result without the high-level checking rule shows that the checking is failed, determining that the problem data does not exist in the target checking data, and considering that the target checking data is normal.
If the check result of the high-level check rule shows that the check fails, determining that problem data exists in the target check data, triggering a blocking mechanism, stopping executing tasks related to the target check data so as to prevent the scope of influence of dirty data from spreading, automatically generating a data quality work order aiming at the target check data, and sending the data quality work order to a responsibility department corresponding to the target check data for processing. After the responsibility department finishes processing, the state of the data quality problem is changed into processed, but the circulation process of the data quality work order is not ended, and whether the problem data still exists in the target check data is checked; if the target check data still has problem data, the data quality work order is transferred to the responsibility department again for processing until the check is carried out to determine that the problem data does not exist in the target check data, and the transfer process of the data quality work order corresponding to the target check data is ended, so that the data problem can be rapidly and accurately identified and solved through repeated check and processing, the data quality and reliability are improved, and if the target check data still has problem data, the responsibility department can be timely informed of reprocessing so as to ensure the normal operation and service continuity of the service system.
Fig. 3 is a schematic diagram of an exemplary embodiment of yet another data quality link management method according to the present application, as shown in fig. 3, comprising the steps of:
s301, determining target checking data to be checked from a data warehouse.
Data to be checked is extracted from the data warehouse as target checking data, such as client information, transaction data, bank card information, credit rating information and the like.
S302, determining a checking scene of the target checking data, and determining a plurality of checking rules corresponding to the target checking data according to the checking scene.
And determining a checking scene of the target checking data. For example, for a financial institution, the check scenario may be classified into a private check, a daily check, and the like.
And selecting a corresponding check rule according to the determined check scene. Different checking scenes and rules have pertinence and key points, and are required to be selected and adjusted according to actual conditions.
S303, determining the rule level of each checking rule according to the importance degree and the sensitivity degree of the target checking data.
In order to better deal with problems in target inspection data and optimize data quality management, rule level setting needs to be performed for each inspection rule. Rule levels are generally classified into three levels, high, medium and low, with different rule levels representing different degrees of importance and sensitivity.
S304, according to a plurality of checking rules corresponding to the target checking data, checking each checking rule is sequentially carried out on the target checking data, and a checking result corresponding to each checking rule is obtained.
And sequentially carrying out the checking of each checking rule on the target checking data to obtain the checking result corresponding to each checking rule. The checking result comprises passing checking and failing checking.
S305, if the check result of the check rule in the target rule level shows that the check fails, determining that problem data exists in the target check data, stopping executing tasks related to the target check data, and automatically generating a data quality work order corresponding to the target check data.
By way of example, the rule levels of all the checking rules are set to be high, medium and low, if the checking result of the checking rule with the high level shows that the checking is failed, it is determined that there is problem data in the target checking data, the task associated with the target checking data must be stopped immediately to prevent the spread of the influence range of the dirty data and avoid the business risk and loss, while the checking rule with the medium and low level is relatively loose, some problems of regularity and secondarily can be processed in later period without obvious influence on the business.
And S306, sending the data quality worksheet to a responsibility department corresponding to the target checking data for processing.
And calling the responsibility person information of the data asset, pushing the data quality work order to related personnel of a corresponding responsibility department for processing, and enabling the data quality responsibility person and related processing personnel to carry out task circulation on the work order model on the processing task of the data quality problem.
In the embodiment of the application, a plurality of checking rules corresponding to the target checking data are determined according to the checking scene of the target checking data, the rule level of each checking rule is determined according to the importance degree and the sensitivity degree of the target checking data, then checking is carried out, and if certain target checking data fails to pass a certain high-level checking rule, the system immediately stops executing tasks related to the target checking data, so that the influence of potential risks or error data can be avoided, and the problem can be corrected in time. In addition, for data with higher importance and sensitivity, more strict check rules are formulated, so that the data security and privacy can be better protected. By the scheme, the efficiency and the precision of data processing can be improved, and more reliable data support is provided for enterprise decision.
Furthermore, in the application, the data quality of each department can be analyzed and checked based on the data quality work order historical data by acquiring the data quality work order historical data of each department in a preset time period, so that an enterprise can be helped to know which departments have poorer data quality and take corresponding measures to improve the data quality, and the enterprise can be helped to find that some business processes have bottlenecks or optimization spaces, thereby improving the business processes in a targeted manner and improving the production efficiency and quality.
Further, when the data quality worksheet is sent to the responsibility department corresponding to the target check data for processing, the mail interface is called, and notification mails corresponding to the data quality worksheet are sent to the responsibility department, so that time waste and risks of data quality reduction caused by unsmooth information transfer are avoided, timeliness and efficiency of data quality processing are effectively improved, communication cooperation effect is improved, and work supervision and feedback are enhanced.
Furthermore, unified arrangement analysis is carried out on the checking scenes, the checking rules corresponding to each checking scene, the rule level corresponding to each checking rule and the historical data of the data quality work orders so as to be converted into knowledge base storage, so that knowledge in the aspects of data management and management can be shared among departments, repeated labor caused by information island is avoided, the data quality problem is prevented from happening, and enterprises can be helped to monitor and control the data quality problem better. When similar problems reappear, related personnel can directly acquire a solution from a knowledge base, and accuracy and efficiency of data processing are improved.
Fig. 4 is a flow chart of a data quality link management method shown in the present application, and as shown in fig. 4, data to be checked is extracted from a data warehouse as target checking data, and a checking scene of the target checking data is determined. For example, for a financial institution, the check scenario may be classified into a private check, a daily check, and the like. And selecting a corresponding check rule according to the determined check scene. Different checking scenes and rules have pertinence and key points, and are required to be selected and adjusted according to actual conditions.
In order to better deal with problems in target inspection data and optimize data quality management, rule level setting needs to be performed for each inspection rule. Rule levels are generally classified into three levels, high, medium and low, with different rule levels representing different degrees of importance and sensitivity.
And sequentially carrying out the checking of each checking rule on the target checking data to obtain the checking result corresponding to each checking rule. The checking result comprises passing checking and failing checking.
If the checking result of the checking rule at the target rule level shows that the checking is failed, determining that problem data exists in the target checking data, stopping executing tasks related to the target checking data, and automatically generating a data quality work order corresponding to the target checking data. And when the data quality worksheet is sent to a responsibility department corresponding to the target check data for processing, calling a mail interface, and sending a notification mail corresponding to the data quality worksheet to the responsibility department.
In addition, the data quality of each department can be analyzed and checked based on the data quality work order historical data by acquiring the data quality work order historical data of each department in a preset time period; and uniformly sorting and analyzing the checking scenes, the checking rules corresponding to each checking scene, the rule level corresponding to each checking rule and the historical data of the data quality work orders to convert the data into a knowledge base for storage.
Fig. 5 is a schematic diagram of a circulation interface of a data quality work order, as shown in fig. 5, when the data quality work order is sent to a responsibility department corresponding to target checking data for processing, a mail interface is called, and a notification mail corresponding to the data quality work order is sent to the responsibility department, wherein the work order is in a new state. After the data quality work order is accepted, the data quality work order is transferred to specific personnel for processing. The problem state at this time is "in process". According to the actual situation of the problem, the data quality work order may need to be circulated for a plurality of times, and after the final-stage processing personnel actually solves the problem, the work order state is modified to be 'solved'. For the problem with the state of 'solved', the system continuously checks the quality check rule corresponding to the problem for 3 days from the solved T+1 to T+3 days, and if the rule passes the data quality check in 3 continuous days, the upstream system is automatically considered to really solve the problem, and the work order can be closed.
Fig. 6 is a schematic diagram of a data quality link management device according to the present application, and as shown in fig. 6, the data quality link management device 600 includes a determining module 601, a checking module 602, a judging module 603, and a processing module 604, where:
the determining module 601 is configured to determine target checking data to be checked, and determine a plurality of checking rules corresponding to the target checking data and rule levels corresponding to each checking rule;
the checking module 602 is configured to sequentially perform checking of each checking rule on the target checking data according to a plurality of checking rules corresponding to the target checking data, and obtain a checking result corresponding to each checking rule;
a judging module 603, configured to determine that problem data exists in the target inspection data if the inspection result of the inspection rule at the target rule level indicates that the inspection is failed, stop executing a task associated with the target inspection data, and automatically generate a data quality work order corresponding to the target inspection data;
and the processing module 604 is used for sending the data quality worksheet to a responsibility department corresponding to the target check data for processing.
The device can carry out targeted processing aiming at different conditions by distinguishing high-level and low-level checking rules, can effectively optimize the process of data quality management, improves the stability and reliability of the service, and brings more commercial value for the development of enterprises.
Further, the judging module 603 is further configured to: setting rule grades of all check rules to be high grade, medium grade and low grade; if any one of the check results of the check rules at the high level is displayed as the check fails, determining that problem data exists in the target check data.
Further, the determining module 601 is further configured to: determining target checking data to be checked from a data warehouse; determining a checking scene of the target checking data, and determining a plurality of checking rules corresponding to the target checking data according to the checking scene; and determining the rule level of each checking rule according to the importance degree and the sensitivity degree of the target checking data.
Further, the processing module 604 is further configured to: after the responsibility department finishes processing, the circulation process of the data quality work order is not finished, and whether the problem data still exists in the target checking data is checked; and if the problem data still exists in the target checking data, the data quality work order is transferred to the responsibility department again for processing until the checking is finished after the responsibility department finishes processing, and the transfer process of the data quality work order corresponding to the target checking data is ended when the checking is confirmed that the problem data does not exist in the target checking data.
Further, the data quality link management apparatus 600 further includes: an acquiring module 605, configured to acquire data quality work order history data of each department in a preset time period; the analysis module 606 is configured to analyze and check the data quality of each department based on the data quality work order history data.
Further, the judging module 603 is further configured to: and when the data quality worksheet is sent to a responsibility department corresponding to the target check data for processing, calling a mail interface, and sending a notification mail corresponding to the data quality worksheet to the responsibility department.
Further, the data quality link management apparatus 600 further includes: the knowledge base conversion module 607 is configured to perform unified arrangement analysis on the inspection scenes, the inspection rules corresponding to each inspection scene, the rule level corresponding to each inspection rule, and the historical data of the data quality work order, so as to convert the data into a knowledge base for storage.
Fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 7, the electronic device may include: a transceiver 71, a processor 72, a memory 73.
Processor 72 executes the computer-executable instructions stored in the memory, causing processor 72 to perform the aspects of the embodiments described above. The processor 72 may be a general purpose processor including a central processing unit CPU, a network processor (network processor, NP), etc.; but may also be a digital signal processor DSP, an application specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.
The memory 73 is connected to the processor 72 via a system bus and communicates with each other, the memory 73 being adapted to store computer program instructions.
The transceiver 71 may be configured to send the data quality worksheet to a responsible department corresponding to the target verification data for processing.
The system bus may be a peripheral component interconnect standard (peripheral component interconnect, PCI) bus or an extended industry standard architecture (extended industry standard architecture, EISA) bus, among others. The system bus may be classified into an address bus, a data bus, a control bus, and the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus. The transceiver is used to enable communication between the database access device and other computers (e.g., clients, read-write libraries, and read-only libraries). The memory may include random access memory (random access memory, RAM) and may also include non-volatile memory (non-volatile memory).
The electronic device provided by the embodiment of the application can be the terminal device of the embodiment.
The embodiment of the application also provides a chip for running the instruction, and the chip is used for executing the technical scheme of the data quality link management method in the embodiment.
The embodiment of the application also provides a computer readable storage medium, wherein the computer readable storage medium stores computer instructions, and when the computer instructions run on a computer, the computer is caused to execute the technical scheme of the data quality link management method of the embodiment.
The embodiment of the application also provides a computer program product, which comprises a computer program, wherein the computer program is stored in a computer readable storage medium, and at least one processor can read the computer program from the computer readable storage medium, and the technical scheme of the data quality link management method in the embodiment can be realized when the at least one processor executes the computer program.
Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It is to be understood that the application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims (17)

1. A method of data quality link management comprising:
determining target checking data to be checked, and determining a plurality of checking rules corresponding to the target checking data and rule grades corresponding to each checking rule;
sequentially carrying out the checking of each checking rule on the target checking data according to a plurality of checking rules corresponding to the target checking data, and obtaining the checking result corresponding to each checking rule;
if the checking result of the checking rule in the target rule level shows that the checking is failed, determining that problem data exists in the target checking data, stopping executing tasks related to the target checking data, and automatically generating a data quality work order corresponding to the target checking data;
and sending the data quality worksheet to a responsibility department corresponding to the target checking data for processing.
2. The method according to claim 1, wherein determining that there is problem data in the target inspection data if the inspection result of the inspection rule at the target rule level shows that the inspection is failed, comprises:
setting rule grades of all the check rules to be high grade, medium grade and low grade;
if any one of the check results of the check rules at the high level is displayed as the check fails, determining that problem data exists in the target check data.
3. The method according to claim 1 or 2, wherein the determining the target checking data to be checked, and determining a plurality of checking rules corresponding to the target checking data and a rule level corresponding to each checking rule, includes:
determining target checking data to be checked from a data warehouse;
determining a checking scene of target checking data, and determining a plurality of checking rules corresponding to the target checking data according to the checking scene;
and determining the rule level of each checking rule according to the importance degree and the sensitivity degree of the target checking data.
4. The method of claim 3, wherein after the sending the data quality worksheet to the responsibility department corresponding to the target verification data for processing, further comprises:
after the responsibility department finishes processing, checking whether the target checking data still has problem data or not without finishing the circulation process of the data quality work order;
and if the target check data still has problem data, the data quality worksheet is again transferred to the responsibility department for processing until the transfer process of the data quality worksheet corresponding to the target check data is ended when the check is carried out after the responsibility department finishes processing and the problem data does not exist in the target check data.
5. The method according to claim 4, further comprising:
acquiring historical data of data quality work orders of all departments within a preset time period;
and analyzing and checking the data quality of each department based on the historical data of the data quality work order.
6. The method according to claim 5, further comprising:
and calling a mail interface when the data quality work order is sent to a responsibility department corresponding to the target check data for processing, and sending a notification mail corresponding to the data quality work order to the responsibility department.
7. The method according to claim 6, further comprising:
and carrying out unified arrangement analysis on the check scenes, the check rules corresponding to each check scene, the rule grade corresponding to each check rule and the data quality work order historical data so as to convert the data quality work order historical data into a knowledge base for storage.
8. A data quality link management apparatus, comprising:
the determining module is used for determining target checking data to be checked, and determining a plurality of checking rules corresponding to the target checking data and rule grades corresponding to each checking rule;
the system comprises a checking module, a checking module and a data processing module, wherein the checking module is used for sequentially checking each checking rule for the target checking data according to a plurality of checking rules corresponding to the target checking data to acquire a checking result corresponding to each checking rule;
the judging module is used for determining that problem data exists in the target checking data if the checking result of the checking rule at the target rule level shows that the checking is failed, stopping executing tasks related to the target checking data and automatically generating a data quality work order corresponding to the target checking data;
and the processing module is used for sending the data quality worksheet to a responsibility department corresponding to the target check data for processing.
9. The apparatus of claim 8, wherein the determining module is further configured to:
setting rule grades of all the check rules to be high grade, medium grade and low grade;
if any one of the check results of the check rules at the high level is displayed as the check fails, determining that problem data exists in the target check data.
10. The apparatus according to claim 8 or 9, wherein the determining module is further configured to:
determining target checking data to be checked from a data warehouse;
determining a checking scene of target checking data, and determining a plurality of checking rules corresponding to the target checking data according to the checking scene;
and determining the rule level of each checking rule according to the importance degree and the sensitivity degree of the target checking data.
11. The apparatus of claim 10, wherein the processing module is further configured to:
after the responsibility department finishes processing, checking whether the target checking data still has problem data or not without finishing the circulation process of the data quality work order;
and if the target check data still has problem data, the data quality worksheet is again transferred to the responsibility department for processing until the transfer process of the data quality worksheet corresponding to the target check data is ended when the check is carried out after the responsibility department finishes processing and the problem data does not exist in the target check data.
12. The apparatus of claim 11, wherein the apparatus further comprises:
the acquisition module is used for acquiring the historical data of the data quality work orders of all departments within a preset time period;
and the analysis module is used for analyzing and checking the data quality of each department based on the data quality work order historical data.
13. The apparatus of claim 12, wherein the determining module is further configured to:
and calling a mail interface when the data quality work order is sent to a responsibility department corresponding to the target check data for processing, and sending a notification mail corresponding to the data quality work order to the responsibility department.
14. The apparatus of claim 13, wherein the apparatus further comprises:
and the knowledge base conversion module is used for carrying out unified arrangement analysis on the check scenes, the check rules corresponding to each check scene, the rule level corresponding to each check rule and the data quality work order historical data so as to convert the data quality work order historical data into a knowledge base for storage.
15. An electronic device, comprising: a processor, and a memory communicatively coupled to the processor;
the memory stores computer-executable instructions;
the processor executes computer-executable instructions stored in the memory to implement the method of any one of claims 1-7.
16. A computer readable storage medium having stored therein computer executable instructions which when executed by a processor are adapted to carry out the method of any one of claims 1-7.
17. A computer program product comprising a computer program which, when executed by a processor, implements the method of any of claims 1-7.
CN202310738838.XA 2023-06-20 2023-06-20 Data quality link management method, device, equipment and storage medium Pending CN116739427A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310738838.XA CN116739427A (en) 2023-06-20 2023-06-20 Data quality link management method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310738838.XA CN116739427A (en) 2023-06-20 2023-06-20 Data quality link management method, device, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN116739427A true CN116739427A (en) 2023-09-12

Family

ID=87913080

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310738838.XA Pending CN116739427A (en) 2023-06-20 2023-06-20 Data quality link management method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN116739427A (en)

Similar Documents

Publication Publication Date Title
CN110827032B (en) Intelligent wind control decision method and system and service processing method and system
US20190378073A1 (en) Business-Aware Intelligent Incident and Change Management
CN111127200A (en) Method and device for monitoring suspicious transactions of anti-money laundering
CN110569197B (en) Credible evidence analysis and measurement method for software credibility evaluation
US20120016802A1 (en) Automatic event management for regulation compliance
CN111144697A (en) Data processing method, data processing device, storage medium and electronic equipment
EP2693379A1 (en) Risk-management device
CN103440460A (en) Application system change validation method and system
CN114331735A (en) Financial risk model training method and device and financial risk prediction method and device
CN116662387A (en) Service data processing method, device, equipment and storage medium
CN116739427A (en) Data quality link management method, device, equipment and storage medium
CN115965451A (en) Data checking method, device, electronic equipment, medium and program product
CN115660842A (en) Accounting checking method, device, equipment and storage medium
CN114723548A (en) Data processing method, apparatus, device, medium, and program product
CN114418775A (en) Method, device, equipment and medium for checking annual fund investment data
Khamidovich THE ROLE OF FINANCIAL CONTROL IN INCREASING RESPONSIBILITY IN THE ORGANIZATION
Kim et al. The new approach to IT testing: real transaction-based automated validation solution
CN118096397A (en) Method, device, equipment and storage medium for monitoring suspicious transactions in insurance industry
CN114238414A (en) Monitoring method and device for suspicious transaction data of money laundering prevention
CN117372043A (en) Data anomaly determination method, device, equipment and medium
CN117113965A (en) Transaction file comparison method, device, equipment and storage medium
CN116739340A (en) Risk detection method and risk detection device
CN115860930A (en) Method, device and equipment for additional recording of anti-money laundering data and readable storage medium
CN113468211A (en) Report form checking method and device, electronic equipment and storage medium
Karimovich et al. UDK: 657.6-051 ISSUES OF RISK ASSESSMENT OF INTERNAL AUDIT IN BANKS OF UZBEKISTAN

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination