CN114547101B - Data quality evaluation method, device, equipment and storage medium for data center - Google Patents
Data quality evaluation method, device, equipment and storage medium for data center Download PDFInfo
- Publication number
- CN114547101B CN114547101B CN202210037302.0A CN202210037302A CN114547101B CN 114547101 B CN114547101 B CN 114547101B CN 202210037302 A CN202210037302 A CN 202210037302A CN 114547101 B CN114547101 B CN 114547101B
- Authority
- CN
- China
- Prior art keywords
- quality
- data
- evaluation
- quality inspection
- inspection evaluation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013441 quality evaluation Methods 0.000 title claims abstract description 56
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000007689 inspection Methods 0.000 claims abstract description 176
- 238000011156 evaluation Methods 0.000 claims abstract description 147
- 238000013210 evaluation model Methods 0.000 claims abstract description 43
- 238000001303 quality assessment method Methods 0.000 claims description 25
- 238000004590 computer program Methods 0.000 claims description 13
- 238000012544 monitoring process Methods 0.000 claims description 13
- 238000010586 diagram Methods 0.000 description 14
- 230000008569 process Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 238000010606 normalization Methods 0.000 description 5
- 238000003908 quality control method Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000006872 improvement Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24564—Applying rules; Deductive queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- General Factory Administration (AREA)
Abstract
The application provides a data quality evaluation method and device of a data center, electronic equipment and a computer readable storage medium. The data quality evaluation method of the data center station comprises the following steps: acquiring evaluation data to be inspected and quality inspection evaluation rules and weights corresponding to the evaluation data; inputting the to-be-inspected evaluation data, the corresponding inspection evaluation rules and weights into a preset inspection evaluation model, and outputting a corresponding data quality evaluation result; the quality inspection evaluation model comprises a plurality of pre-configured quality inspection evaluation objects and quality inspection evaluation rules and weights respectively bound with the quality inspection evaluation objects. According to the embodiment of the application, the quality evaluation of the data can be performed.
Description
Technical Field
The application belongs to the field of data quality evaluation, and particularly relates to a data quality evaluation method and device of a data center, electronic equipment and a computer readable storage medium.
Background
The data quality refers to the degree to which the data meets the purpose of use of data consumers and can meet the specific requirements of business scenes in a business environment.
In different business scenarios, the requirements of data consumers on the data quality are different, some people mainly pay attention to the accuracy and consistency of the data, and other people pay attention to the real-time property and correlation of the data. Therefore, as long as the data can meet the purpose of use, the data quality can be said to meet the requirements.
How to strengthen the control of the integrity, normalization, consistency, accuracy, uniqueness and timeliness of the data, gradually realize the improvement of the quality of the system business data, and need to establish a set of objective evaluation mechanism.
Therefore, how to evaluate the quality of data is a technical problem that needs to be solved by those skilled in the art.
Disclosure of Invention
The embodiment of the application provides a data quality evaluation method and device of a data center, electronic equipment and a computer readable storage medium, and the data quality evaluation method and device can be used for evaluating the quality of data.
In a first aspect, an embodiment of the present application provides a data quality evaluation method of a data center, including:
acquiring evaluation data to be inspected and quality inspection evaluation rules and weights corresponding to the evaluation data;
inputting the to-be-inspected evaluation data, the corresponding inspection evaluation rules and weights into a preset inspection evaluation model, and outputting a corresponding data quality evaluation result;
the quality inspection evaluation model comprises a plurality of pre-configured quality inspection evaluation objects and quality inspection evaluation rules and weights respectively bound with the quality inspection evaluation objects.
Optionally, before acquiring the to-be-inspected evaluation data and the corresponding quality inspection evaluation rule and weight, the method further includes:
acquiring data quality problem information;
determining a plurality of quality inspection evaluation objects based on the data quality problem information;
based on a plurality of quality inspection evaluation objects, respectively setting corresponding quality inspection evaluation rules and weights;
based on a plurality of quality inspection evaluation objects and quality inspection evaluation rules and weights respectively bound with the quality inspection evaluation objects, establishing a quality inspection evaluation model;
wherein the quality control evaluation object comprises a data table comprising fields of different data types.
Optionally, the method further comprises:
and changing the quality inspection evaluation rule and the weight corresponding to any quality inspection evaluation object.
Optionally, inputting the to-be-inspected evaluation data and the corresponding inspection evaluation rule and weight thereof into a preset inspection evaluation model, and outputting a corresponding data quality evaluation result, including:
inputting the to-be-inspected evaluation data, the corresponding inspection evaluation rules and weights into a preset inspection evaluation model, and outputting a corresponding data quality evaluation report;
the data quality assessment report comprises quality system related scores and trends of the quality assessment data to be inspected.
In a second aspect, an embodiment of the present application provides a data quality evaluation apparatus of a data center, including:
the first acquisition module is used for acquiring the to-be-inspected evaluation data and the corresponding quality inspection evaluation rules and weights;
the quality evaluation result output module is used for inputting the to-be-inspected evaluation data, the corresponding quality inspection evaluation rules and weights into a preset quality inspection evaluation model and outputting the corresponding data quality evaluation result;
the quality inspection evaluation model comprises a plurality of pre-configured quality inspection evaluation objects and quality inspection evaluation rules and weights respectively bound with the quality inspection evaluation objects.
Optionally, the apparatus further comprises:
the second acquisition module is used for acquiring data quality problem information;
the determining module is used for determining a plurality of quality inspection evaluation objects based on the data quality problem information;
the setting module is used for setting corresponding quality inspection evaluation rules and weights respectively based on the plurality of quality inspection evaluation objects;
the quality inspection evaluation model building module is used for building a quality inspection evaluation model based on a plurality of quality inspection evaluation objects and quality inspection evaluation rules and weights respectively bound with the quality inspection evaluation objects;
wherein the quality control evaluation object comprises a data table comprising fields of different data types.
Optionally, the apparatus further comprises:
and the changing module is used for changing the quality inspection evaluation rule and the weight corresponding to any quality inspection evaluation object.
Optionally, the quality evaluation result output module is configured to input the to-be-inspected quality evaluation data, the quality inspection evaluation rule and the weight corresponding to the to-be-inspected quality evaluation data into a preset quality inspection evaluation model, and output a corresponding data quality evaluation report;
the data quality assessment report comprises quality system related scores and trends of the quality assessment data to be inspected.
In a third aspect, an embodiment of the present application provides an electronic device, including: a processor and a memory storing computer program instructions;
the processor when executing the computer program instructions implements a data quality assessment method for a data center as shown in the first aspect.
In a fourth aspect, embodiments of the present application provide a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement a data quality assessment method for a data center as described in the first aspect.
According to the data quality evaluation method and device for the data center, the electronic equipment and the computer readable storage medium, the data can be subjected to quality evaluation.
The data quality evaluation method of the data center station acquires quality inspection evaluation data to be inspected and corresponding quality inspection evaluation rules and weights; inputting the to-be-inspected evaluation data and the corresponding inspection evaluation rules and weights into a preset inspection evaluation model, and outputting a corresponding data quality evaluation result.
The quality inspection evaluation model comprises a plurality of quality inspection evaluation objects and quality inspection evaluation rules and weights which are respectively bound with the quality inspection evaluation objects, so that the data quality evaluation method of the data center can evaluate the quality of the data, can strengthen the control of the integrity, normalization, consistency, accuracy, uniqueness and timeliness of the data, and gradually realizes the improvement of the quality of the system service data.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described below, it will be obvious that the drawings in the description below are some embodiments of the present application, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a method for evaluating data quality of a data center according to an embodiment of the present application;
FIG. 2 is a schematic flow chart of creating a quality inspection object provided in one embodiment of the present application;
FIG. 3 is a flow chart illustrating creation of quality inspection rules provided by one embodiment of the present application;
FIG. 4 is a flow diagram of creating a quality model provided by one embodiment of the present application;
FIG. 5 is a flow diagram of generating a quality assessment report provided by one embodiment of the present application;
FIG. 6 is a flow diagram of generating a quality assessment report provided by one embodiment of the present application;
FIG. 7 is a schematic structural diagram of a data quality evaluation device of a data center station according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Features and exemplary embodiments of various aspects of the present application are described in detail below to make the objects, technical solutions and advantages of the present application more apparent, and to further describe the present application in conjunction with the accompanying drawings and the detailed embodiments. It should be understood that the specific embodiments described herein are intended to be illustrative of the application and are not intended to be limiting. It will be apparent to one skilled in the art that the present application may be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present application by showing examples of the present application.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
The data quality refers to the degree to which the data meets the purpose of use of data consumers and can meet the specific requirements of business scenes in a business environment.
In different business scenarios, the requirements of data consumers on the data quality are different, some people mainly pay attention to the accuracy and consistency of the data, and other people pay attention to the real-time property and correlation of the data. Therefore, as long as the data can meet the purpose of use, the data quality can be said to meet the requirements.
How to strengthen the control of the integrity, normalization, consistency, accuracy, uniqueness and timeliness of the data, gradually realize the improvement of the quality of the system business data, and need to establish a set of objective evaluation mechanism.
To solve the problems in the prior art, embodiments of the present application provide a data quality evaluation method, apparatus, device, and computer-readable storage medium for a data center. The following first describes a data quality evaluation method of the data center provided in the embodiments of the present application.
Fig. 1 is a flow chart of a data quality evaluation method of a data center station according to an embodiment of the present application. As shown in fig. 1, the data quality evaluation method of the data center station includes:
s101, acquiring evaluation data to be inspected and quality inspection evaluation rules and weights corresponding to the evaluation data.
S102, inputting the to-be-inspected evaluation data and the corresponding inspection evaluation rules and weights into a preset inspection evaluation model, and outputting a corresponding data quality evaluation result.
The quality inspection evaluation model comprises a plurality of pre-configured quality inspection evaluation objects and quality inspection evaluation rules and weights respectively bound with the quality inspection evaluation objects.
In one embodiment, before acquiring the to-be-inspected evaluation data and the corresponding quality inspection evaluation rules and weights, the method further comprises:
acquiring data quality problem information;
determining a plurality of quality inspection evaluation objects based on the data quality problem information;
based on a plurality of quality inspection evaluation objects, respectively setting corresponding quality inspection evaluation rules and weights;
based on a plurality of quality inspection evaluation objects and quality inspection evaluation rules and weights respectively bound with the quality inspection evaluation objects, establishing a quality inspection evaluation model;
wherein the quality control evaluation object comprises a data table comprising fields of different data types.
In order to perform quality evaluation on data and further strengthen control on integrity, normalization, consistency, accuracy, uniqueness and timeliness of the data, the embodiment determines a plurality of quality inspection evaluation objects based on data quality problem information, sets corresponding quality inspection evaluation rules and weights respectively, and can more accurately establish a quality inspection evaluation model meeting the requirements.
In one embodiment, the method further comprises:
and changing the quality inspection evaluation rule and the weight corresponding to any quality inspection evaluation object.
Because the user demand can change from time to time, the embodiment can change the quality inspection evaluation rule and the weight corresponding to any quality inspection evaluation object, and the data quality can be evaluated more accurately later.
Specifically, an object to be subjected to quality monitoring, namely a quality inspection evaluation object, is defined for quality problems in data, and is a foothold point operated by rules, and generally corresponds to a data table in a database, wherein the data table comprises fields, each field has different data types, and when the object is defined, the quality inspection evaluation rule is automatically matched for each field.
Then, setting quality inspection evaluation rules and establishing a quality inspection evaluation model. The quality inspection evaluation rule is the basis of table field quality judgment, and table or field level is temporarily not considered by library level. The quality inspection evaluation model is a bearing surface of the quality report, and one quality inspection evaluation model comprises a plurality of quality inspection evaluation objects, a plurality of quality inspection evaluation rules bound with the quality inspection evaluation objects and weight percentages of the quality inspection evaluation rules in the whole model.
In one embodiment, the quality control assessment model includes:
O i a quality score for the ith monitored object;
w i the weight of the ith monitoring object;
R j quality score of the jth monitoring rule in the monitoring object;
w j the weight of the jth monitoring rule in the monitoring object;
C j the number of data pieces meeting the quality requirement;
S j total number of quality tests;
for example, the number of the cells to be processed,
the quality inspection evaluation model s= [ normal line count/total line count weight (rule 1) +normal line count/total line count weight (rule 2) ], TABLE weight (TABLE a) + [ normal line count/total line count weight (rule 1) +normal line count/total line count weight (rule 2) ], TABLE weight (TABLE B) + [ normal line count/total line count weight (rule 1) +normal line count/total line count weight (rule 2) ], TABLE weight (TABLE C).
The quality inspection evaluation model can be understood as: and configuring different quality inspection evaluation objects through the model, setting different weights for the objects and the rules, and finally obtaining corresponding quality inspection results by running.
Fig. 2 is a schematic flow chart of creating quality inspection objects according to an embodiment of the present application, as shown in fig. 2, a quality inspection object classification is created first, then a quality inspection object is created, and finally a quality inspection rule is set.
Fig. 3 is a schematic flow chart of creating quality inspection rules according to an embodiment of the present application, and as shown in fig. 3, the quality inspection rules are added to the quality inspection object according to steps, and then the rule state is set to be the enabled state.
Fig. 4 is a schematic flow chart of creating a quality model according to an embodiment of the present application, as shown in fig. 4, where a quality model is created first, then a quality inspection object is added, a quality inspection rule is set, and finally a range is selected.
In one embodiment, inputting the to-be-inspected evaluation data and the corresponding inspection evaluation rule and weight thereof into a preset inspection evaluation model, and outputting a corresponding data quality evaluation result, including:
inputting the to-be-inspected evaluation data, the corresponding inspection evaluation rules and weights into a preset inspection evaluation model, and outputting a corresponding data quality evaluation report;
the data quality assessment report comprises quality system related scores and trends of the quality assessment data to be inspected.
The data quality assessment report in the embodiment comprises the quality system related scores and trends of the quality assessment data to be inspected, so that the user can know the data quality more intuitively and conveniently.
FIG. 5 is a flow chart of generating a quality assessment report according to one embodiment of the present application, where the quality assessment report is a report of measurement data such as quality system related scores, trends, etc. generated according to a quality model setting. As shown in fig. 5, the quality model is selected first, then weights are set for each quality inspection object in the model, quality inspection rules in each quality inspection object are selected and weights are set, and finally click operation is performed.
FIG. 6 is a schematic flow chart of generating a quality assessment report according to an embodiment of the present application, as shown in FIG. 6, where a quality model is selected first, then a task under the model is selected (each quality model runs once to generate a task), and report information related to the task can be displayed.
In the embodiment, an object needing quality monitoring is defined aiming at quality problems in data, a quality inspection rule is set, a quality inspection evaluation model is established, a quality evaluation report is generated, the control of the integrity, normalization, consistency, accuracy, uniqueness and timeliness of the data is enhanced, and the improvement of the quality of system service data is gradually realized.
Fig. 7 is a schematic structural diagram of a data quality evaluation device of a data center, according to an embodiment of the present application, as shown in fig. 7, where the data quality evaluation device of the data center includes:
the first obtaining module 701 is configured to obtain to-be-inspected evaluation data and corresponding quality inspection evaluation rules and weights thereof;
the quality evaluation result output module 702 is configured to input the to-be-inspected quality evaluation data, the quality inspection evaluation rule and the weight corresponding to the to-be-inspected quality evaluation data into a preset quality inspection evaluation model, and output a corresponding data quality evaluation result;
the quality inspection evaluation model comprises a plurality of pre-configured quality inspection evaluation objects and quality inspection evaluation rules and weights respectively bound with the quality inspection evaluation objects.
In one embodiment, the apparatus further comprises:
the second acquisition module is used for acquiring data quality problem information;
the determining module is used for determining a plurality of quality inspection evaluation objects based on the data quality problem information;
the setting module is used for setting corresponding quality inspection evaluation rules and weights respectively based on the plurality of quality inspection evaluation objects;
the quality inspection evaluation model building module is used for building a quality inspection evaluation model based on a plurality of quality inspection evaluation objects and quality inspection evaluation rules and weights respectively bound with the quality inspection evaluation objects;
wherein the quality control evaluation object comprises a data table comprising fields of different data types.
In one embodiment, the apparatus further comprises:
and the changing module is used for changing the quality inspection evaluation rule and the weight corresponding to any quality inspection evaluation object.
In one embodiment, the quality evaluation result output module 702 is configured to input the to-be-inspected quality evaluation data and the quality inspection evaluation rule and weight corresponding thereto into a preset quality inspection evaluation model, and output a corresponding data quality evaluation report;
the data quality assessment report comprises quality system related scores and trends of the quality assessment data to be inspected.
The modules/units in the apparatus shown in fig. 7 have functions of implementing the steps in fig. 1, and achieve corresponding technical effects, which are not described herein for brevity.
Fig. 8 shows a schematic structural diagram of an electronic device according to an embodiment of the present application.
The electronic device may include a processor 801 and a memory 802 storing computer program instructions.
In particular, the processor 801 may include a Central Processing Unit (CPU), or an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or may be configured to implement one or more integrated circuits of embodiments of the present application.
In one embodiment, memory 802 may be Read Only Memory (ROM). In one embodiment, the ROM may be mask-programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically Erasable PROM (EEPROM), electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.
The processor 801 implements the data quality assessment method of any of the data-in-process embodiments described above by reading and executing computer program instructions stored in the memory 802.
In one example, the electronic device may also include a communication interface 803 and a bus 810. As shown in fig. 8, the processor 801, the memory 802, and the communication interface 803 are connected to each other via a bus 810 and perform communication with each other.
The communication interface 803 is mainly used to implement communication between each module, apparatus, unit and/or device in the embodiments of the present application.
In addition, in combination with the data quality evaluation method of the data center in the above embodiment, the embodiment of the application may be implemented by providing a computer readable storage medium. The computer readable storage medium has stored thereon computer program instructions; the computer program instructions, when executed by a processor, implement a data quality assessment method for any of the data-center stations in the above embodiments.
It should be clear that the present application is not limited to the particular arrangements and processes described above and illustrated in the drawings. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present application are not limited to the specific steps described and illustrated, and those skilled in the art can make various changes, modifications, and additions, or change the order between steps, after appreciating the spirit of the present application.
The functional blocks shown in the above-described structural block diagrams may be implemented in hardware, software, firmware, or a combination thereof. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, a plug-in, a function card, or the like. When implemented in software, the elements of the present application are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine readable medium or transmitted over transmission media or communication links by a data signal carried in a carrier wave. A "machine-readable medium" may include any medium that can store or transfer information. Examples of machine-readable media include electronic circuitry, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio Frequency (RF) links, and the like. The code segments may be downloaded via computer networks such as the internet, intranets, etc.
It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the above-described steps, that is, the steps may be performed in the order mentioned in the embodiments, may be different from the order in the embodiments, or several steps may be performed simultaneously.
Aspects of the present application are described above with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions/acts specified in the flowchart and/or block diagram block or blocks. Such a processor may be, but is not limited to being, a general purpose processor, a special purpose processor, an application specific processor, or a field programmable logic circuit. It will also be understood that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware which performs the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In the foregoing, only the specific embodiments of the present application are described, and it will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the systems, modules and units described above may refer to the corresponding processes in the foregoing method embodiments, which are not repeated herein. It should be understood that the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive various equivalent modifications or substitutions within the technical scope of the present application, which are intended to be included in the scope of the present application.
Claims (8)
1. A method for evaluating data quality of a data center, comprising:
acquiring data quality problem information;
determining a plurality of quality inspection evaluation objects based on the data quality problem information;
based on a plurality of quality inspection evaluation objects, setting corresponding quality inspection evaluation rules and weights respectively;
establishing a quality inspection evaluation model based on a plurality of quality inspection evaluation objects and quality inspection evaluation rules and weights respectively bound with the quality inspection evaluation objects;
the quality inspection evaluation object comprises a data table, wherein the data table comprises fields of different data types;
acquiring evaluation data to be inspected and quality inspection evaluation rules and weights corresponding to the evaluation data;
inputting the to-be-inspected evaluation data and the corresponding inspection evaluation rules and weights thereof into a preset inspection evaluation model, and outputting a corresponding data quality evaluation result;
the quality inspection evaluation model comprises a plurality of pre-configured quality inspection evaluation objects and quality inspection evaluation rules and weights respectively bound with the quality inspection evaluation objects;
the quality inspection evaluation model comprises:
O i quality score for the ith monitored object, w i Weight of the ith monitored object, R j Quality score, w, for the jth monitoring rule in a monitored object j For the weight of the j-th monitoring rule in the monitored object, C j For the number of data strips meeting the quality requirement of the jth monitoring rule, S j The total number of rules is monitored for the j-th of quality inspection.
2. The method for evaluating the data quality of a station in data according to claim 1, further comprising:
and changing the quality inspection evaluation rule and the weight corresponding to any quality inspection evaluation object.
3. The method for evaluating the data quality of the data center according to claim 1, wherein inputting the data to be evaluated and the corresponding quality inspection evaluation rules and weights thereof into a preset quality inspection evaluation model, and outputting the corresponding data quality evaluation result, comprises:
inputting the to-be-inspected evaluation data and the corresponding inspection evaluation rules and weights thereof into a preset inspection evaluation model, and outputting a corresponding data quality evaluation report;
wherein the data quality assessment report includes quality system related scores and trends of the quality assessment data to be inspected.
4. A data quality evaluation apparatus of a station in data, comprising:
the second acquisition module is used for acquiring data quality problem information;
the determining module is used for determining a plurality of quality inspection evaluation objects based on the data quality problem information;
the setting module is used for setting corresponding quality inspection evaluation rules and weights respectively based on a plurality of quality inspection evaluation objects;
the quality inspection evaluation model building module is used for building a quality inspection evaluation model based on a plurality of quality inspection evaluation objects and quality inspection evaluation rules and weights respectively bound with the quality inspection evaluation objects;
the quality inspection evaluation object comprises a data table, wherein the data table comprises fields of different data types;
the first acquisition module is used for acquiring the to-be-inspected evaluation data and the corresponding quality inspection evaluation rules and weights;
the quality evaluation result output module is used for inputting the to-be-inspected evaluation data, the quality inspection evaluation rule and the weight corresponding to the to-be-inspected evaluation data into a preset quality inspection evaluation model and outputting a corresponding data quality evaluation result;
the quality inspection evaluation model comprises a plurality of pre-configured quality inspection evaluation objects and quality inspection evaluation rules and weights respectively bound with the quality inspection evaluation objects;
the quality inspection evaluation model comprises:
O i quality score for the ith monitored object, w i Weight of the ith monitored object, R j Quality score, w, for the jth monitoring rule in a monitored object j For the weight of the j-th monitoring rule in the monitored object, C j For the number of data strips meeting the quality requirement of the jth monitoring rule, S j The total number of rules is monitored for the j-th of quality inspection.
5. The apparatus for evaluating the quality of data of a station in data according to claim 4, further comprising:
and the changing module is used for changing the quality inspection evaluation rule and the weight corresponding to any quality inspection evaluation object.
6. The data quality evaluation device of the data center according to claim 4, wherein the quality evaluation result output module is configured to input the to-be-inspected quality evaluation data and the quality inspection evaluation rule and the weight corresponding thereto into a preset quality inspection evaluation model, and output a corresponding data quality evaluation report;
wherein the data quality assessment report includes quality system related scores and trends of the quality assessment data to be inspected.
7. An electronic device, the electronic device comprising: a processor and a memory storing computer program instructions;
the processor, when executing the computer program instructions, implements a data quality assessment method for a data center as claimed in any one of claims 1-3.
8. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon computer program instructions, which when executed by a processor, implement a data quality assessment method of a data center according to any of claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210037302.0A CN114547101B (en) | 2022-01-13 | 2022-01-13 | Data quality evaluation method, device, equipment and storage medium for data center |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210037302.0A CN114547101B (en) | 2022-01-13 | 2022-01-13 | Data quality evaluation method, device, equipment and storage medium for data center |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114547101A CN114547101A (en) | 2022-05-27 |
| CN114547101B true CN114547101B (en) | 2023-04-28 |
Family
ID=81672023
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210037302.0A Active CN114547101B (en) | 2022-01-13 | 2022-01-13 | Data quality evaluation method, device, equipment and storage medium for data center |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114547101B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109684851A (en) * | 2018-12-27 | 2019-04-26 | 中国移动通信集团江苏有限公司 | Evaluation of Software Quality, device, equipment and computer storage medium |
| CN113468034A (en) * | 2021-07-07 | 2021-10-01 | 浙江大华技术股份有限公司 | Data quality evaluation method and device, storage medium and electronic equipment |
| CN113886373A (en) * | 2021-09-09 | 2022-01-04 | 浙江大华技术股份有限公司 | Data processing method and device and electronic equipment |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110276499A1 (en) * | 2010-05-05 | 2011-11-10 | ValueAppeal LLC | Collecting data from multiple sources and automatically generating a report using the collected data, such as collecting data related to property taxes from multiple sources and automatically generating a property tax assessment appeal |
| CN106776261B (en) * | 2015-11-23 | 2021-03-12 | 腾讯科技（深圳）有限公司 | Object testing method and device |
| CN107506888B (en) * | 2017-07-07 | 2020-10-20 | 中国科学院计算机网络信息中心 | Universal online service platform customized evaluation method and system |
| CN111639829A (en) * | 2020-04-09 | 2020-09-08 | 广东电网有限责任公司佛山供电局 | Lean item analysis and optimization method and device and electronic equipment |
| CN113806336B (en) * | 2021-09-30 | 2024-07-30 | 北京蓝海医信科技有限公司 | Data quality assessment method and system |
-
2022
- 2022-01-13 CN CN202210037302.0A patent/CN114547101B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109684851A (en) * | 2018-12-27 | 2019-04-26 | 中国移动通信集团江苏有限公司 | Evaluation of Software Quality, device, equipment and computer storage medium |
| CN113468034A (en) * | 2021-07-07 | 2021-10-01 | 浙江大华技术股份有限公司 | Data quality evaluation method and device, storage medium and electronic equipment |
| CN113886373A (en) * | 2021-09-09 | 2022-01-04 | 浙江大华技术股份有限公司 | Data processing method and device and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114547101A (en) | 2022-05-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108280542B (en) | User portrait model optimization method, medium and equipment | |
| CN115222361A (en) | Business process flow transfer method, device, equipment and computer storage medium | |
| CN111565174A (en) | Vehicle quality information determination model training method and vehicle quality information determination method | |
| CN114091898A (en) | Information determination method, device, equipment and storage medium | |
| CN108377508B (en) | User perception classification method and device based on measurement report data | |
| CN114547101B (en) | Data quality evaluation method, device, equipment and storage medium for data center | |
| CN114417830A (en) | Risk evaluation method, device, equipment and computer readable storage medium | |
| CN112749998A (en) | Income information output method and device, electronic equipment and computer storage medium | |
| CN112685610B (en) | False registration account identification method and related device | |
| CN114677139A (en) | Method and device for determining loan amount, equipment, product and readable storage medium | |
| CN111461118B (en) | Interest feature determining method, device, equipment and storage medium | |
| CN114638503A (en) | Asset risk pressure testing method, device, equipment and storage medium | |
| CN114092219A (en) | Model verification method and device, electronic equipment and storage medium | |
| CN110458707B (en) | Behavior evaluation method and device based on classification model and terminal equipment | |
| CN115310735A (en) | Method, device, equipment and storage medium for identifying business district boundary | |
| CN113112102A (en) | Priority determination method, device, equipment and storage medium | |
| CN114547133B (en) | Multi-dimensional dataset-based conversational attribution analysis method, device and equipment | |
| CN113496288B (en) | User stability determining method, device, equipment and storage medium | |
| CN113704035B (en) | Time delay detection method and device and related equipment | |
| CN116910340A (en) | Data processing method, device, equipment, medium and product | |
| CN117007083B (en) | Vehicle ranging capability assessment method, system, equipment and medium thereof | |
| JPH1075218A (en) | Method and device for identifying characteristic boundary of data groups | |
| CN117371858A (en) | Evaluation method, device, equipment, medium and product for data management capability | |
| CN115906189A (en) | Model verification method, device, equipment and computer readable storage medium | |
| CN116186063A (en) | Data query method, device, equipment and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |