CN117217561A - Data quality management system - Google Patents
Data quality management system Download PDFInfo
- Publication number
- CN117217561A CN117217561A CN202311262087.5A CN202311262087A CN117217561A CN 117217561 A CN117217561 A CN 117217561A CN 202311262087 A CN202311262087 A CN 202311262087A CN 117217561 A CN117217561 A CN 117217561A
- Authority
- CN
- China
- Prior art keywords
- data
- unit
- management
- target
- checking
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003326 Quality management system Methods 0.000 title claims abstract description 15
- 238000000034 method Methods 0.000 claims abstract description 33
- 230000008569 process Effects 0.000 claims abstract description 28
- 230000007246 mechanism Effects 0.000 claims abstract description 12
- 238000012545 processing Methods 0.000 claims abstract description 6
- 238000007726 management method Methods 0.000 claims description 120
- 238000013523 data management Methods 0.000 claims description 23
- 238000012544 monitoring process Methods 0.000 claims description 22
- 238000004458 analytical method Methods 0.000 claims description 21
- 230000008520 organization Effects 0.000 claims description 13
- 238000005516 engineering process Methods 0.000 claims description 12
- 238000011156 evaluation Methods 0.000 claims description 11
- 238000012937 correction Methods 0.000 claims description 10
- 238000007689 inspection Methods 0.000 claims description 9
- 238000004891 communication Methods 0.000 claims description 8
- 238000005457 optimization Methods 0.000 claims description 6
- 238000012217 deletion Methods 0.000 claims description 3
- 230000037430 deletion Effects 0.000 claims description 3
- 238000011161 development Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000003442 weekly effect Effects 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000003908 quality control method Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000005067 remediation Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application provides a data quality management system, which comprises: the strategic target module is used for determining strategic targets according to the data of the target stages, and formulating corresponding staged targets for each target stage; the management and control mechanism module is used for carrying out personnel allocation on the data treatment and making a treatment system and a treatment flow according to the personnel allocation; the data quality checking system module is used for checking the data to construct a data quality checking system, and determining the direction and the dimension of data quality management through the data quality checking system; the management execution module is used for determining a target stage in which target data are located, analyzing and generating target personnel allocation information, a target treatment system and a target treatment process according to the target stage, and executing the target treatment process. The data quality and the service processing efficiency of each service system are improved, the interconnection and the intercommunication among the service systems and the information sharing are realized, and the data quality of a company is improved.
Description
Technical Field
The application relates to the field of data management, in particular to a data quality management system.
Background
With the continuous development and popularization of information technology, data becomes an important foundation for enterprise operation and decision making. However, as the amount of data continues to increase, data quality problems are also becoming increasingly prominent.
Data quality problems may cause problems such as enterprise decision errors, resource waste, customer complaints, etc., so data quality management becomes an important issue for enterprise management. In the early days, data quality was mainly managed manually, for example, by manually checking data, making standardized data input rules, and the like. As technology has evolved, various data quality management tools and techniques have emerged, such as data cleansing tools, data analysis tools, data management tools, etc., to help businesses better manage and maintain their data. Data quality management has now become an important component of enterprise management. The continuous development and application of various data quality management methods and technologies enable enterprises to manage and maintain data more effectively, improve the quality and value of the data, and provide powerful support for decision making and development of the enterprises.
At present, the quality of the data provided by the external channel is poor, and the history data is difficult to treat due to history reasons.
Disclosure of Invention
In view of the problems, the present application has been made to provide a data quality management system that overcomes the problems or at least partially solves the problems, including:
a data quality management system, the system comprising:
the strategic target module is used for determining strategic targets according to the data of target phases, and making corresponding phased targets for each target phase, wherein the target phases at least comprise three sub-phases;
the management and control mechanism module is used for carrying out personnel allocation on the data treatment and making a treatment system and a treatment flow according to the personnel allocation;
the data quality checking system is used for checking the data to construct a data quality checking system, and determining the direction and the dimension of data quality management through the data quality checking system, wherein the direction comprises a monitoring reporting direction, a service standard direction and a data standard direction, and the dimension at least comprises accuracy, completeness, consistency, uniqueness, effectiveness and timeliness;
the management execution module is used for determining a target stage in which target data are located, analyzing and generating target personnel allocation information, a target treatment system and a target treatment process according to the target stage, executing the target treatment process, and evaluating and monitoring the treated target data through the data quality inspection system when the target treatment process is executed.
Further, the strategic target module comprises a supervision reporting unit, an operation analysis unit and a digital intelligence enabling unit;
the supervision reporting unit is used for determining whether the quality of the data meets preset requirements or not, and managing illegal data in a first management direction, wherein the first management direction comprises source system optimization, source system data correction, reporting logic adjustment and supervision reporting;
the operation analysis unit is used for checking data through a preset checking rule, and managing the data violating the checking rule through a second management direction, wherein the preset checking rule comprises a direction service specification and a data specification, and the second management direction comprises source system optimization, source system data correction, checking rule adjustment and a standard service flow;
the intelligent counting enabling unit is used for matching corresponding user requirements, business scenes and market requirements through digital processing according to the data.
Further, the management and control mechanism module comprises an organization management unit, a system management unit and a flow management unit;
the organization management unit is used for managing personnel and division of each project group in the data treatment process, wherein the project group at least comprises a business group, an execution group and a technology group;
the system management unit is used for determining the data quality management responsibility system corresponding to the business group, the execution group and the technology group in the organization management unit;
the flow management unit is used for constructing corresponding data quality management flows according to the business group, the execution group and the technical group in the organization management unit.
Further, the management execution module includes a problem notification unit;
the problem notifying unit is used for detecting and determining the data quality problem on the target data according to the first preset time and the data quality checking system.
Further, the system also comprises a problem analysis unit;
when the data quality problem sent by the problem notifying unit is received, the problem analyzing unit is used for determining a data treatment communication conference according to the data quality problem and a department corresponding to the data quality problem; analyzing a cause of the problem and determining a solution corresponding to the cause of the problem through the data governance communication session.
Further, the system also comprises a problem management unit;
when the solution sent by the problem analysis unit is received, the problem management unit is used for executing management through the corresponding affiliated departments according to the solution.
Further, the system also comprises a governance evaluation unit;
when the solution execution is completed, the governance evaluation unit is used for evaluating the solution condition of the solution through a preset governance evaluation system.
Further, the system also comprises a continuous monitoring unit and a governance reporting unit;
the continuous monitoring unit is used for continuously monitoring the data quality problem through a preset quality checking system;
the management report unit is used for generating a data management special report of the data at a second preset time.
Further, the checking system module comprises a direction management group, wherein the direction management group comprises a supervision reporting direction unit, a service standard direction unit and a data standard direction unit;
the supervision and reporting direction unit is used for checking whether the data of the EAST supervision report and/or the data of the policy registration report meet the supervision and reporting requirements;
the service specification direction unit is used for checking whether service data in the data meets the service compliance requirement;
the data specification direction unit is used for checking whether the data maintains the consistency principle.
Further, the checking system module comprises a dimension management group, wherein the dimension management group comprises an accuracy unit, an integrity unit, a consistency unit, a uniqueness unit, a validity unit and a timeliness unit;
the accuracy unit is used for determining whether the data can correspond to the original data;
the integrity unit is used for determining whether the data has a deletion;
the consistency unit is used for determining whether the information attributes of the same information body in different data sets are the same;
the uniqueness unit is used for determining whether repeated records exist in the data;
the validity unit is used for determining whether the attribute of the data meets a preset constraint condition or not;
the timeliness unit is used for determining whether the update speed of the data meets the service requirement.
The application has the following advantages:
in the embodiment of the application, compared with the defects of poor quality of data provided by an external channel and difficult management of historical data due to historical reasons in the prior art, the application provides a solution of a data quality management system, which is specifically as follows: the strategy target module is used for determining a strategy target according to the data of target phases and formulating corresponding staged targets for each target phase, wherein the target phases at least comprise three sub-phases; the management and control mechanism module is used for carrying out personnel allocation on the data treatment and making a treatment system and a treatment flow according to the personnel allocation; the data quality checking system is used for checking the data to construct a data quality checking system, and determining the direction and the dimension of data quality management through the data quality checking system, wherein the direction comprises a monitoring reporting direction, a service standard direction and a data standard direction, and the dimension at least comprises accuracy, completeness, consistency, uniqueness, effectiveness and timeliness; the management execution module is used for determining a target stage in which target data are located, analyzing and generating target personnel allocation information, a target treatment system and a target treatment process according to the target stage, executing the target treatment process, and evaluating and monitoring the treated target data through the data quality inspection system when the target treatment process is executed. The problems that the quality of data provided by an external channel is poor, and historical data is difficult to manage due to historical reasons are solved through a strategy target module, a management and control mechanism module, a check system module and a management and execution module, so that the quality of data of each business link is managed and controlled, the data quality and business processing efficiency of each business system are improved, and the interconnection and intercommunication and information sharing among the business systems are realized; the data quality of enterprises is improved, the supervision reporting is met, the risk of the supervised reporting is reduced, meanwhile, the operation analysis of the enterprises is more efficient and accurate, and the effect of intelligent counting and enabling is achieved.
Drawings
In order to more clearly illustrate the technical solutions of the present application, the drawings that are needed in the description of the present application will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort to a person skilled in the art.
FIG. 1 is a block diagram illustrating a data quality management system according to an embodiment of the present application;
fig. 2 is a schematic diagram of an architecture of an embodiment of a data quality management system according to an embodiment of the present application.
Detailed Description
In order that the manner in which the above recited objects, features and advantages of the present application are obtained will become more readily apparent, a more particular description of the application briefly described above will be rendered by reference to the appended drawings. It will be apparent that the described embodiments are some, but not all, embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The inventors found by analyzing the prior art that: at present, the quality of the data provided by the external channel is poor, and the history data is difficult to treat due to history reasons.
Referring to fig. 1 and 2, a block diagram and a schematic diagram of a data quality management system according to an embodiment of the present application are shown;
a data quality management system, the system comprising:
the strategic target module 110 is configured to determine a strategic target according to data of target phases, and formulate a corresponding staged target for each target phase, where the target phases include at least three sub-phases;
the management and control mechanism module 120 is used for performing personnel allocation on the data management and control, and making a management system and a management flow according to the personnel allocation;
the checking system module 130 is configured to check the data to construct a data quality checking system, and determine a direction and a dimension of data quality management according to the data quality checking system, where the direction includes a monitoring reporting direction, a service specification direction and a data specification direction, and the dimension includes at least accuracy, integrity, consistency, uniqueness, validity and timeliness;
the management execution module 140 is configured to determine a target stage in which target data is located, analyze the target stage according to the target stage to generate target personnel allocation information, a target treatment system and a target treatment process, execute the target treatment process, and evaluate and monitor the treated target data through the data quality inspection system when the target treatment process is executed.
In the embodiment of the application, compared with the defects of poor quality of data provided by an external channel and difficult management of historical data due to historical reasons in the prior art, the application provides a solution of a data quality management system, which is specifically as follows: the strategy target module is used for determining a strategy target according to the data of target phases and formulating corresponding staged targets for each target phase, wherein the target phases at least comprise three sub-phases; the management and control mechanism module is used for carrying out personnel allocation on the data treatment and making a treatment system and a treatment flow according to the personnel allocation; the data quality checking system is used for checking the data to construct a data quality checking system, and determining the direction and the dimension of data quality management through the data quality checking system, wherein the direction comprises a monitoring reporting direction, a service standard direction and a data standard direction, and the dimension at least comprises accuracy, completeness, consistency, uniqueness, effectiveness and timeliness; the management execution module is used for determining a target stage in which target data are located, analyzing and generating target personnel allocation information, a target treatment system and a target treatment process according to the target stage, executing the target treatment process, and evaluating and monitoring the treated target data through the data quality inspection system when the target treatment process is executed. The problems that the quality of data provided by an external channel is poor, and historical data is difficult to manage due to historical reasons are solved through a strategy target module, a management and control mechanism module, a check system module and a management and execution module, so that the quality of data of each business link is managed and controlled, the data quality and business processing efficiency of each business system are improved, and the interconnection and intercommunication and information sharing among the business systems are realized; the data quality of enterprises is improved, the supervision reporting is met, the risk of the supervised reporting is reduced, meanwhile, the operation analysis of the enterprises is more efficient and accurate, and the effect of intelligent counting and enabling is achieved.
Next, a data quality management system in the present exemplary embodiment will be further described.
In an embodiment of the application, the strategic objective module is used for defining strategic objectives of data quality management, and gradually improving from the condition of meeting supervision and reporting, meeting management analysis to the condition of realizing intelligent number energization; in one embodiment, the strategic goals of data quality management are defined, and the steps from meeting supervision and reporting, meeting business analysis, to realizing intelligent data enabling are gradually promoted.
In an embodiment of the application, the management and control mechanism module is used for establishing overall work of the data management system project group and standardizing data quality management work by formulating relevant systems and processes; in one embodiment, the data governance project group is established to orchestrate the overall work, defining the relevant personnel and responsibility division. Meanwhile, a data quality management flow is established, a data quality management responsibility system is implemented, and a data management evaluation system is formulated.
In an embodiment of the present application, a management execution module is configured to determine a target stage in which target data is located, perform analysis according to the target stage to generate target personnel allocation information, a target treatment system, and a target treatment process, execute the target treatment process, and evaluate and monitor the treated target data through the data quality inspection system when the target treatment process is executed; in one embodiment, the abatement process is closed loop management of data for inspection, reporting, analysis, abatement, assessment, monitoring.
In an embodiment of the application, the checking system module is used for constructing a data quality checking system from three directions of supervision report, service specification and data specification and from six dimensions of accuracy, integrity, consistency, uniqueness, validity and timeliness, and guiding the direction and dimension of data quality management; in a specific embodiment, from three directions of supervision and report (such as policy registration), service specification and data specification, and six dimensions of accuracy, integrity, consistency, uniqueness, validity and timeliness, by combining actual service scenes and industry experience, a data quality standard is defined, a data quality check system is built, and the direction and dimension of data quality management are guided.
In an embodiment of the present application, the strategic objective module includes a supervision reporting unit, an operation analysis unit, and a data intelligence enabling unit; the supervision reporting unit is used for determining whether the quality of the data meets preset requirements or not, and managing illegal data in a first management direction, wherein the first management direction comprises source system optimization, source system data correction, reporting logic adjustment and supervision reporting; the operation analysis unit is used for checking data through a preset checking rule, and managing the data violating the checking rule through a second management direction, wherein the preset checking rule comprises a direction service specification and a data specification, and the second management direction comprises source system optimization, source system data correction, checking rule adjustment and a standard service flow; the intelligent counting enabling unit is used for matching corresponding user requirements, business scenes and market requirements through digital processing according to the data; in a specific embodiment, the supervision and reporting unit is configured to ensure that the quality of supervision and reporting data meets the supervision compliance requirement, and govern the data that violates the supervision rules through the governance system and the flow; the business analysis unit is used for improving the data quality of the company so that business analysis is efficient and accurate, checking data is carried out through the direction business specification and the data specification, and data violating the checking rules is treated; the intelligent counting and enabling unit is used for improving the data resource capacity and improving the enterprise efficiency and capacity, deeply mining the user demands by utilizing digital knowledge and technology, exploring multiple business scenes, agile insights into markets, optimizing insurance products, and providing careless services to be closely contacted with clients.
In an embodiment of the present application, the management and control mechanism module includes an organization management unit, a system management unit, and a flow management unit; the organization management unit is used for managing personnel and division of each project group in the data treatment process, wherein the project group at least comprises a business group, an execution group and a technology group; the system management unit is used for determining the data quality management responsibility system corresponding to the business group, the execution group and the technology group in the organization management unit; the flow management unit is used for constructing corresponding data quality management flows according to the business group, the execution group and the technical group in the organization management unit.
In an embodiment of the present application, the management execution module includes a problem notification unit; the problem notifying unit is used for detecting and determining a data quality problem on the target data according to a first preset time and the data quality checking system; in a specific embodiment, the problem notifying unit is used for periodically (such as daily, weekly, monthly) discovering the data quality problem through a quality check system, and sending the mail to a problem owner department; in one embodiment, the data source of the data governance report is manually run out to generate the data source of the data governance report by using a development script of the Crenplus database grammar, and the data governance problem ledgers are manually arranged and updated. In addition, by using a Kettle tool, data management reports (daily report, weekly report and monthly report) are automatically arranged, and a manual mail is sent to a problem owner department.
In an embodiment of the present application, the system further includes a problem analysis unit; when the data quality problem sent by the problem notifying unit is received, the problem analyzing unit is used for determining a data treatment communication conference according to the data quality problem and a department corresponding to the data quality problem; analyzing a cause of a problem and determining a solution corresponding to the cause of the problem by the data governance communication conference; in one embodiment, the owner of the irregular organization problem initiates a data governance communication session, analyzes the cause of the problem, and determines a problem solution; in a specific implementation, the problem owner department checks the issued problem list in the foreground page and the background database respectively, analyzes the problem to be a business problem or a system problem, and provides business management and control or system reconstruction requirements for the data quality problem. In addition, the data management project group irregularly organizes and carries out data management communication conferences, and summons related personnel (such as business, demand, development, operation and maintenance and data) can analyze and discuss problem reasons, determine the correction scheme of the problem, responsible persons and plan completion time. In addition, the correction scheme, the responsible person and the plan completion time of the problem are synchronously updated to the data management problem ledger.
In an embodiment of the present application, the system further includes a problem management unit; when the solution sent by the problem analysis unit is received, the problem management unit is used for executing management through the corresponding affiliated departments according to the solution; in one embodiment, the problem owner is prompted to perform remediation according to the determined problem solution; in a specific implementation, the data management project group actively pushes the problem owner to manage according to the problem determination and correction scheme, and meanwhile, the problem owner is assisted and pushed to determine the problem correction scheme by actively providing a problem list and actively analyzing the problem. In addition, after the problem owner department feeds back the problem to be rectified and online, the data management project group verifies whether the data management requirement is met, if so, the problem is closed, the problem state and the actual completion time of the data management problem account are updated, and if not, the problem is further pushed to be solved until the problem is completely solved.
In one embodiment of the application, the system further comprises a governance evaluation unit; when the solution is executed, the governance evaluation unit is used for evaluating the solution condition of the solution through a preset governance evaluation system; in one embodiment, the solution of the data quality problem is evaluated by using a governance evaluation system; in one embodiment, a data governance assessment system is established by referencing a corporate performance assessment system and combining data quality management metrics (wherein the data governance assessment system includes ratings, coefficients, and scoring criteria, the data governance assessment ratings and coefficients = problem count assessment score 0.6+ problem drop scale assessment score 0.4), and suggesting that the data governance assessment is incorporated into corporate department assessment.
In one embodiment of the application, the system further comprises a continuous monitoring unit and a governance reporting unit; the continuous monitoring unit is used for continuously monitoring the data quality problem through a preset quality checking system; the management report unit is used for generating a data management special report of the data at a second preset time; in a specific embodiment, the continuous monitoring unit continuously monitors the data quality problem by using a quality check system until the problem is completely solved; the management report unit is used for reporting the data management effect through a data management special report at regular intervals (monthly) so as to ensure that the data quality control is in place; in a specific implementation, a data quality management module of a data management platform is used for configuring periodic (daily, weekly and monthly) checking tasks of data quality rules, and continuously monitoring data quality problems; through a monthly checking task of the data management platform, a monthly problem list is manually generated by running batch by using a development script of a Creenplum database grammar, the problem account is managed and updated by using the problem account as a data management month report data source, and a data management special report is manufactured in a form of PPT (point-to-point) monthly to inform the data management effect and ensure that the data quality control is in place.
In an embodiment of the present application, the inspection system module includes a direction management group, where the direction management group includes a supervision reporting direction unit, a service specification direction unit, and a data specification direction unit; the supervision and reporting direction unit is used for checking whether the data of the EAST supervision report and/or the data of the policy registration report meet the supervision and reporting requirements; the service specification direction unit is used for checking whether service data in the data meets the service compliance requirement; the data specification direction unit is used for checking whether the data maintains the consistency principle or not; in a specific embodiment, the supervision and reporting direction unit is used for checking and checking data of the EAST supervision report and/or data of the policy registration report, and checking whether the data quality meets the supervision and reporting requirement; the service specification direction unit is used for checking whether the service data accords with the service compliance requirement; the data specification direction unit is used for checking whether the data among the systems keep the consistency principle.
In an embodiment of the present application, the inspection system module includes a dimension management group, where the dimension management group includes an accuracy unit, an integrity unit, a consistency unit, a uniqueness unit, a validity unit, and a timeliness unit; the accuracy unit is used for determining whether the data can correspond to the original data; the integrity unit is used for determining whether the data has a deletion; the consistency unit is used for determining whether the information attributes of the same information body in different data sets are the same; the uniqueness unit is used for determining whether repeated records exist in the data; the validity unit is used for determining whether the attribute of the data meets a preset constraint condition or not; the timeliness unit is used for determining whether the update speed of the data meets the service requirement; in a specific embodiment, the accuracy unit is configured to check whether the original data is actually and accurately recorded; the integrity unit is used for checking whether the data information is missing or not; the consistency unit is used for checking whether the information attributes of the same information body in different data sets are the same; the uniqueness unit is used for checking that each record of the data has no repeated record; the validity unit is used for checking whether the attribute of the data meets a certain constraint condition; the timeliness unit is used for checking whether the speed of data updating meets the service requirement.
The application has the beneficial effects that: based on the data quality management technology in three directions, by establishing a data quality management model and carrying out closed-loop management of checking, reporting, analyzing, managing, evaluating and monitoring on the data quality, potential problems and risks in a system can be effectively found, so that the data quality can be improved and optimized in time. In addition, through the data quality management technology, the data quality of enterprises is improved, the supervision reporting is met, the risk of the supervised reporting is reduced, meanwhile, the operation analysis of the enterprises is more efficient and accurate, and the digital intelligence enabling is realized.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described by differences from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other.
While preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the scope of the embodiments of the application.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or terminal device comprising the element.
The foregoing has described in detail a data quality management system provided by the present application, and specific examples have been provided herein to illustrate the principles and embodiments of the present application, the above examples being provided only to assist in understanding the methods of the present application and their core ideas; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.
Claims (10)
1. A data quality management system, the system comprising:
the strategic target module is used for determining strategic targets according to the data of target phases, and making corresponding phased targets for each target phase, wherein the target phases at least comprise three sub-phases;
the management and control mechanism module is used for carrying out personnel allocation on the data treatment and making a treatment system and a treatment flow according to the personnel allocation;
the data quality checking system is used for checking the data to construct a data quality checking system, and determining the direction and the dimension of data quality management through the data quality checking system, wherein the direction comprises a monitoring reporting direction, a service standard direction and a data standard direction, and the dimension at least comprises accuracy, completeness, consistency, uniqueness, effectiveness and timeliness;
the management execution module is used for determining a target stage in which target data are located, analyzing and generating target personnel allocation information, a target treatment system and a target treatment process according to the target stage, executing the target treatment process, and evaluating and monitoring the treated target data through the data quality inspection system when the target treatment process is executed.
2. The system of claim 1, wherein the strategic goal module comprises a supervisory reporting unit, an operation analysis unit, and a digital intelligence enabling unit;
the supervision reporting unit is used for determining whether the quality of the data meets preset requirements or not, and managing illegal data in a first management direction, wherein the first management direction comprises source system optimization, source system data correction, reporting logic adjustment and supervision reporting;
the operation analysis unit is used for checking data through a preset checking rule, and managing the data violating the checking rule through a second management direction, wherein the preset checking rule comprises a direction service specification and a data specification, and the second management direction comprises source system optimization, source system data correction, checking rule adjustment and a standard service flow;
the intelligent counting enabling unit is used for matching corresponding user requirements, business scenes and market requirements through digital processing according to the data.
3. The system of claim 1, wherein the management and control mechanism module comprises an organization management unit, a regimen management unit, and a flow management unit;
the organization management unit is used for managing personnel and division of each project group in the data treatment process, wherein the project group at least comprises a business group, an execution group and a technology group;
the system management unit is used for determining the data quality management responsibility system corresponding to the business group, the execution group and the technology group in the organization management unit;
the flow management unit is used for constructing corresponding data quality management flows according to the business group, the execution group and the technical group in the organization management unit.
4. The system according to claim 1, wherein the management execution module includes a problem notification unit;
the problem notifying unit is used for detecting and determining the data quality problem on the target data according to the first preset time and the data quality checking system.
5. The system of claim 4, further comprising a problem analysis unit;
when the data quality problem sent by the problem notifying unit is received, the problem analyzing unit is used for determining a data treatment communication conference according to the data quality problem and a department corresponding to the data quality problem; analyzing a cause of the problem and determining a solution corresponding to the cause of the problem through the data governance communication session.
6. The system of claim 5, further comprising a problem management unit;
when the solution sent by the problem analysis unit is received, the problem management unit is used for executing management through the corresponding affiliated departments according to the solution.
7. The system of claim 6, further comprising a abatement evaluation unit;
when the solution execution is completed, the governance evaluation unit is used for evaluating the solution condition of the solution through a preset governance evaluation system.
8. The system of claim 7, further comprising a continuous monitoring unit and a abatement reporting unit;
the continuous monitoring unit is used for continuously monitoring the data quality problem through a preset quality checking system;
the management report unit is used for generating a data management special report of the data at a second preset time.
9. The system of claim 1, wherein the checkbox system module comprises a direction management group, wherein the direction management group comprises a supervision reporting direction unit, a traffic specification direction unit, and a data specification direction unit;
the supervision and reporting direction unit is used for checking whether the data of the EAST supervision report and/or the data of the policy registration report meet the supervision and reporting requirements;
the service specification direction unit is used for checking whether service data in the data meets the service compliance requirement;
the data specification direction unit is used for checking whether the data maintains the consistency principle.
10. The system of claim 1, wherein the checkarchitecture module comprises a dimension management group, wherein the dimension management group comprises an accuracy unit, an integrity unit, a consistency unit, a uniqueness unit, a validity unit, and a timeliness unit;
the accuracy unit is used for determining whether the data can correspond to the original data;
the integrity unit is used for determining whether the data has a deletion;
the consistency unit is used for determining whether the information attributes of the same information body in different data sets are the same;
the uniqueness unit is used for determining whether repeated records exist in the data;
the validity unit is used for determining whether the attribute of the data meets a preset constraint condition or not;
the timeliness unit is used for determining whether the update speed of the data meets the service requirement.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311262087.5A CN117217561A (en) | 2023-09-27 | 2023-09-27 | Data quality management system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311262087.5A CN117217561A (en) | 2023-09-27 | 2023-09-27 | Data quality management system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117217561A true CN117217561A (en) | 2023-12-12 |
Family
ID=89038784
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311262087.5A Pending CN117217561A (en) | 2023-09-27 | 2023-09-27 | Data quality management system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117217561A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107491381A (en) * | 2017-07-04 | 2017-12-19 | 广西电网有限责任公司电力科学研究院 | A kind of equipment condition monitoring quality of data evaluating system |
| CN109034532A (en) * | 2018-06-20 | 2018-12-18 | 江苏网域科技有限公司 | A kind of data managing and control system based on big data |
| CN115731066A (en) * | 2022-12-08 | 2023-03-03 | 江苏达科数智技术有限公司 | Data management system and method for service audit platform |
| CN116701358A (en) * | 2023-06-15 | 2023-09-05 | 朱东 | Data processing method and system |
| CN116719799A (en) * | 2023-05-06 | 2023-09-08 | 神彩科技股份有限公司 | Environment-friendly data management method, device, computer equipment and storage medium |
-
2023
- 2023-09-27 CN CN202311262087.5A patent/CN117217561A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107491381A (en) * | 2017-07-04 | 2017-12-19 | 广西电网有限责任公司电力科学研究院 | A kind of equipment condition monitoring quality of data evaluating system |
| CN109034532A (en) * | 2018-06-20 | 2018-12-18 | 江苏网域科技有限公司 | A kind of data managing and control system based on big data |
| CN115731066A (en) * | 2022-12-08 | 2023-03-03 | 江苏达科数智技术有限公司 | Data management system and method for service audit platform |
| CN116719799A (en) * | 2023-05-06 | 2023-09-08 | 神彩科技股份有限公司 | Environment-friendly data management method, device, computer equipment and storage medium |
| CN116701358A (en) * | 2023-06-15 | 2023-09-05 | 朱东 | Data processing method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Mahmood et al. | Key factors that influence task allocation in global software development | |
| Colledani et al. | Analytical methods to support continuous improvements at Scania | |
| US20080189163A1 (en) | Information management system | |
| US8799210B2 (en) | Framework for supporting transition of one or more applications of an organization | |
| US10268978B2 (en) | Methods and systems for intelligent enterprise bill-of-process with embedded cell for analytics | |
| Nguyen et al. | A controlled experiment in assessing and estimating software maintenance tasks | |
| Fucci et al. | Needs and challenges for a platform to support large-scale requirements engineering: A multiple-case study | |
| CN107369120A (en) | A kind of smart business's management, service system and enterprise's activity assays | |
| Bartolini et al. | Business-driven IT management | |
| CN113762735A (en) | Data quality management system and method based on rule base | |
| CN110032154B (en) | Production control method and system for small-batch products | |
| US8606615B2 (en) | System for managing and tracking an inventory of elements | |
| CN113420987A (en) | Demand scheduling method, device, server and computer readable storage medium | |
| US20130041712A1 (en) | Emerging risk identification process and tool | |
| CN117217561A (en) | Data quality management system | |
| CN115082039A (en) | Data center management method and device, electronic equipment and storage medium | |
| CN115423305A (en) | IT service management method and device and server | |
| CN109359825B (en) | Bank digital operation product value analysis method | |
| US20200065733A1 (en) | Method and system for estimating efforts for software managed services production support engagements | |
| Kapel et al. | Incident Management in a Software-Defined Business: A Case Study | |
| Fucci et al. | Needs and challenges for a platform to support large-scale requirements engineering | |
| Bartolini et al. | Synthetic incident generation in the reenactment of IT support organization behavior | |
| Christa et al. | A systematic analysis of defects, incidents, tickets and service effort estimation | |
| Dytha et al. | Risk Analysis using Risk Failure Mode and Effect in Implementation of Integrated Digital Reporting with Power Bi Dashboard | |
| Caceres | Industry 4.0 and IoT: How to Improve the Knowledge for the Project Management before Carrying Out an Implementation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |