CN114266485A - Construction method and construction system of power information communication data quality detection model - Google Patents
Construction method and construction system of power information communication data quality detection model Download PDFInfo
- Publication number
- CN114266485A CN114266485A CN202111596943.1A CN202111596943A CN114266485A CN 114266485 A CN114266485 A CN 114266485A CN 202111596943 A CN202111596943 A CN 202111596943A CN 114266485 A CN114266485 A CN 114266485A
- Authority
- CN
- China
- Prior art keywords
- quality detection
- data quality
- data
- information communication
- power information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 246
- 238000004891 communication Methods 0.000 title claims abstract description 57
- 238000010276 construction Methods 0.000 title claims abstract description 14
- 238000000034 method Methods 0.000 claims abstract description 43
- 239000011159 matrix material Substances 0.000 claims abstract description 39
- 238000012545 processing Methods 0.000 claims abstract description 8
- 238000005516 engineering process Methods 0.000 claims abstract description 7
- 238000013441 quality evaluation Methods 0.000 claims description 10
- 238000004519 manufacturing process Methods 0.000 claims description 4
- 230000005540 biological transmission Effects 0.000 claims description 3
- 230000005484 gravity Effects 0.000 claims description 3
- 238000012360 testing method Methods 0.000 description 6
- 238000012797 qualification Methods 0.000 description 4
- 230000005611 electricity Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000003908 quality control method Methods 0.000 description 2
- 238000004445 quantitative analysis Methods 0.000 description 2
- 241000764238 Isis Species 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000013479 data entry Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000004451 qualitative analysis Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
Images
Abstract
The invention provides a construction method and a construction system of a power information communication data quality detection model, which comprises the steps of constructing a data source model based on multi-source power information communication data by adopting an integrated storage and parallel collection processing technology; determining a data quality detection object based on the data source model; selecting a data quality detection index based on actual service requirements; designing data quality detection rules of different dimensionality indexes based on the selected data quality detection indexes; constructing a judgment matrix by using an analytic hierarchy process, and determining the weight and the expected value of each data quality detection index; evaluating based on a data quality detection rule, and calculating to obtain the qualified percentage of the data quality detection index; and constructing a power information communication data quality detection model based on the data quality detection object, the data quality detection index, the weight, the expected value and the qualified percentage. And the hidden danger of data quality reduction is discovered in time, and the management level of the data quality is improved.
Description
Technical Field
The invention relates to the technical field of communication data sharing, in particular to a construction method and a construction system of a power information communication data quality detection model.
Background
The quantity and the variety of service data in the power information communication system are gradually increased, and the data sharing requirement is urgent. The data quality and the data sharing utilization level are not high, and firstly, the data pair analysis decision support degree is low, and the same data has the problems of multiple sources and inconsistent statistical calibers; secondly, the support degree of the data on operation management needs to be improved, the data quality is uneven, part of the data has no service system support, and unified specification, standard and definite data accountability are lacked; thirdly, the data entry workload of front-line personnel is huge, the data are repeatedly entered, and the service function is repeated; and fourthly, the data quality control is lagged, the control work is one-sidedness, an integral data quality control system and a comprehensive and effective data quality guarantee mechanism are not formed, and the deep mining of the data value is standardized.
Therefore, it is necessary to develop a method and a system for constructing a quality detection model of power information communication data, which can accurately and effectively detect power information communication data of multi-source information, timely find out hidden troubles of power data quality degradation, and improve the management level of a power grid company on the power data quality.
Disclosure of Invention
The present invention is directed to solving one of the technical problems of the prior art or the related art.
Therefore, the invention provides a construction method and a construction system of a power information communication data quality detection model.
In view of the above, an aspect of the present invention provides a method for constructing a power information communication data quality detection model, where the method includes:
on the basis of multi-source power information communication data, a data source model is constructed by adopting an integrated storage and parallel collection processing technology;
determining a data quality detection object based on the data source model;
selecting a data quality detection index based on actual service requirements;
designing data quality detection rules of different dimensionality indexes based on the selected data quality detection indexes;
evaluating based on the data quality detection rule, and calculating to obtain the qualified percentage of the data quality detection index;
constructing a judgment matrix by using an analytic hierarchy process, and determining the weight and the expected value of each data quality detection index;
and constructing a power information communication data quality detection model based on the data quality detection object, the data quality detection index, the data quality detection rule, the weight and expected value and the qualified percentage.
Further, the multi-source power information communication network data comprises operation data, production data, ERP data and marketing data.
Further, the data quality detection object is a data item or a data set, wherein the data set is a collection of data items, and the data set can be divided into subclasses of the data quality detection object.
Further, the data quality detection indexes comprise correctness, completeness, uniqueness, accuracy, effectiveness, consistency and timeliness.
Further, the correctness refers to whether the data quality detection object conforms to objective facts or not, and whether errors occur in the data quality detection object acquisition, transmission, dump and other processes or not; the integrity refers to whether the data quality detection object has missing records; the uniqueness refers to whether similar repeated records exist in the data quality detection object or not; the accuracy refers to whether the precision of the data quality detection object meets the requirement or not; the timeliness refers to whether the data quality detection object is still effective under the current condition; the consistency refers to whether the expression formats of the data quality detection objects are consistent or not; the validity refers to whether the expression format and the numerical value of the data quality detection object are valid or not.
Further, designing the data quality detection rule includes:
designing a data quality detection rule R by referring to the data quality detection indexi(Ii) Each data quality detection index corresponds to a data quality detection rule.
Further, the step of constructing the judgment matrix by using an analytic hierarchy process, and the step of determining the weight and the expected value of each data quality detection index comprises the following steps:
layering the problems to form a hierarchical structure consisting of a target layer, a criterion layer and a measure layer;
forming a judgment matrix by adopting a 1-9 scaling method;
checking the consistency of the judgment matrix based on the analytic hierarchy process;
and calculating layer by layer to obtain the weight of the scheme layer to the target layer, wherein the scheme with the maximum specific gravity is the optimal scheme for solving the problem.
Further, the calculating the qualified percentage of the data quality detection index includes:
according to the data quality detection rule Ri(Ii) Detecting and analyzing the data quality detection object, and calculating to meet the data quality detection rule Ri(Ii) Data percentage of (S)i(Ri(Ii))；
Then, an expected value E of each data quality detection index is giveniAnd respectively calculating a quality evaluation result SA, a total expected value SE and a relative quantized value SR of the data quality detection object.
Further, the communication data quality detection model for constructing the power information is as follows:
M＝<D,I,R,W,E,S>
in the formula: d is a data quality detection object; i is a data quality detection index set needing to be detected on D; r is a data quality detection rule set corresponding to the data quality detection index; w is the weight of the data quality detection index; e is the expected value given for R; and S is data quality scoring for performing data quality evaluation on the data quality detection object based on the data quality detection rule.
According to another aspect of the invention, a system for constructing a power information communication data quality detection model is provided, wherein a data source model module is used for constructing a data source model by adopting an integrated storage and parallel collection processing technology based on multi-source power information communication data;
the data quality detection object module is used for determining a data quality detection object based on the data source model;
the data quality detection index module selects a data quality detection index based on actual service requirements;
the data quality detection rule module is used for designing data quality detection rules of different dimensionality indexes based on the selected data quality detection indexes;
the qualified percentage module is used for evaluating based on the data quality detection rule and calculating to obtain the qualified percentage of the data quality detection index;
the weight and expected value module is used for constructing a judgment matrix by using an analytic hierarchy process and determining the weight and the expected value of each data quality detection index;
the power information communication data quality detection model system is used for constructing a power information communication data quality detection model based on the data quality detection object module, the data quality detection index module, the data quality detection rule module, the weight and expected value module and the qualified percentage module.
The technical scheme provided by the invention can have the following beneficial effects:
the method comprises the steps of establishing a data quality detection object, a data quality detection index, a weight, an expected value and a qualified percentage, establishing a power information communication data quality detection model, accurately and effectively detecting power information communication data of multi-source information, finding out hidden dangers of power data quality reduction in time, and improving the management level of a power grid company on the power data quality.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the embodiments or technical solutions in the prior art of the present invention, the drawings used in the description of the embodiments or prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without creative efforts.
FIG. 1 is a flow chart illustrating steps of a method for constructing a power information communication data quality detection model according to an embodiment of the invention;
FIG. 2 is a flow chart illustrating the steps of determining weights for various data quality detection indicators according to one embodiment of the present invention;
FIG. 3 shows a relationship diagram of a target layer, a criteria layer, and a measure layer, according to one embodiment of the invention.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the invention, as detailed in the appended claims.
Example 1
Fig. 1 is a flowchart illustrating steps of a method for constructing a power information communication data quality detection model according to an embodiment of the present invention.
As shown in fig. 1, the present embodiment provides a method for constructing a power information communication data quality detection model, where the method includes:
step 1, constructing a data source model by adopting an integrated storage and parallel collection processing technology based on multi-source electric power information communication data;
step 2, determining a data quality detection object based on the data source model;
step 3, selecting a data quality detection index based on the actual service requirement;
step 4, designing data quality detection rules of different dimensionality indexes based on the selected data quality detection indexes;
step 5, evaluating based on a data quality detection rule, and calculating to obtain the qualified percentage of the data quality detection index;
step 6, constructing a judgment matrix by using an analytic hierarchy process, and determining the weight and the expected value of each data quality detection index;
and 7, constructing a power information communication data quality detection model based on the data quality detection object, the data quality detection index, the data quality detection rule, the weight, the expected value and the qualified percentage.
According to the data quality detection object, the data quality detection index, the data quality detection rule, the weight, the expected value and the qualified percentage, the electric power information communication data quality detection model is constructed, so that the electric power information communication data of the multi-source information can be accurately and effectively detected, the hidden danger of electric power data quality reduction is timely found, and the management level of a power grid company on the electric power data quality is improved.
Further, the multi-source power information communication network data comprises operation data, production data, ERP data and marketing data.
The operation data comprises optical fiber rate, scheduling level, OTN overview time slot allocation ratio and the like; the production data comprises equipment information, equipment states, equipment use conditions and the like; ERP data financial data, asset data, logistics data and the like; the marketing data includes a transaction electricity price, an electricity sales amount, electricity consumption customer information, and the like.
Further, a data quality detection object is a data item or a data set, wherein a data set is a collection of data items, and the data set can be divided into subclasses of data quality detection objects.
It should be noted that the data quality detection object (indicated by D) may be a data item or a data set. A data set is a collection of data items that contains a plurality of data items, and the data set may be divided into data quality test object subclasses according to the data items in the data set. For example, a computer room is taken as a quality detection object, wherein environmental data, network security data, equipment operation data and the like are subclasses of the data quality detection object.
Further, the data quality detection indexes comprise correctness, completeness, uniqueness, accuracy, effectiveness, consistency and timeliness.
Wherein, the correctness refers to whether the data quality detection object accords with objective facts or not, and whether errors occur in the processes of data quality detection object acquisition, transmission, dump and the like; the integrity refers to whether a data quality detection object has a missing record; uniqueness refers to whether similar repeated records exist in a data quality detection object; the accuracy refers to whether the precision of a data quality detection object meets the requirement or not; timeliness refers to whether a data quality detection object is still valid under current conditions; consistency refers to whether the expression formats of the data quality detection objects are consistent or not; the validity refers to whether the expression format and the numerical value of the data quality detection object are valid, wherein a data quality detection index set needing to be detected on the data quality detection object D is represented by I.
It should be noted that the accuracy is not the only standard for measuring the data quality, and the data quality standard needs to be from the perspective of the user, and different users have different standards for the data quality at different times.
Further, designing the data quality detection rule includes:
designing a data quality detection rule R by referring to the data quality detection indexi(Ii) Each data quality detection index corresponds to a data quality detection rule.
The different dimension indexes refer to 7 dimensions of correctness, completeness, uniqueness, accuracy, effectiveness, consistency and timeliness of the data quality detection indexes, and the data quality detection rule set corresponding to the data quality detection indexes is represented by R.
It should be noted that, taking the switch as an example, the switch capacity R is referred to1Selecting data quality detection index integrity I1Design integrity rule R1(I1) The integrity of the switch capacity data cannot be null according to the actual service requirements, so the switch capacity is non-null, R1(I1) | A This is the integrity rule for the switch capacity.
Fig. 2 is a flow chart illustrating the steps of determining weights for various data quality detection indicators according to one embodiment of the present invention.
As shown in fig. 2, the determining the weight and the expected value of each data quality detection index by using the analytic hierarchy process to construct the judgment matrix includes:
601, layering the problem to form a hierarchical structure consisting of a target layer, a criterion layer and a measure layer;
step 602, forming a judgment matrix by using a 1-9 scaling method;
step 603, checking and judging the consistency of the matrix based on an analytic hierarchy process;
and step 604, obtaining the weight of the scheme layer to the target layer through layer-by-layer calculation, wherein the scheme with the largest specific gravity is the optimal scheme for solving the problem.
The analytic hierarchy process is a simple, flexible and practical multi-criterion decision-making method for quantitative analysis of qualitative problems. The method combines quantitative analysis and qualitative analysis, judges the relative importance degree between standards whether each measurement target can be realized or not by using the experience of a decision maker, reasonably gives the weight of each standard of each decision scheme, and utilizes the weight to calculate the quality sequence of each scheme.
FIG. 3 shows a relationship diagram of a target layer, a criteria layer, and a measure layer, according to one embodiment of the invention.
It should be noted that, as shown in fig. 3, the problem of the present invention is to perform quality detection on multi-source power information communication network data, where an element of a target layer a is a target, that is, to reasonably detect data quality, and to improve the overall level of data quality, an element of a criterion layer B is a criterion layer B of data quality detection indexes, and includes correctness B1, completeness B2, uniqueness B3, accuracy B4, and validity B5, and an element of a measure layer C is a data quality detection rule, which includes correctness rule C1, completeness rule C2, uniqueness rule C3, accuracy rule C4, and validity rule C5.
More specifically, in step 601, to establish a hierarchical structure, a problem to be analyzed is first defined, and the problem is organized, organized and layered to form the hierarchical structure, which generally comprises the following three layers: target layer (highest layer), which refers to the intended target of the question; criterion layer (middle layer) refers to criteria that affect the goal achievement; measure layer (lowest layer): means measures to promote the achievement of the goal. The decision-making target is made clear, the target is used as an element of a target layer (highest layer), the target requirement is unique, namely the target layer only has one element, then a criterion influencing the target realization is found out and is used as a criterion layer factor under the target layer, finally, the final solutions (measures) for solving the decision problem (realizing the decision-making target) under the criterion are analyzed and used as the measure layer factor to be placed at the lowest part (lowest layer) of the hierarchical structure, the factors and the positions of all the layers are made clear, and the relationship between the factors and the positions is connected through a connecting line, so that the hierarchical structure is formed.
For example: for multi-source power information communication data, data quality is expected to be improved through data quality detection, namely, a decision target is 'reasonably detecting data quality and improving the integral level of data quality', and in order to achieve the target, five main criteria are considered, namely, correctness, completeness, uniqueness, accuracy and effectiveness. Then, the factors of each layer are put in place according to the upper and lower relations, and the relations between the factors are connected by connecting lines, thus forming a hierarchical structure.
Further, a judgment matrix is constructed according to a hierarchical structure (this is the purpose of layering the problem and forming a hierarchical structure composed of a target layer, a criterion layer, and a measure layer).
Step 602 is constructing a decision matrix and assigning a value, wherein the method for constructing the decision matrix comprises: each element with downward membership (called criteria) is used as the first element (positioned at the upper left corner) of the judgment matrix, the elements which belong to the element are sequentially arranged at the first row and the first column, and the expert is asked repeatedly, and the criteria of the judgment matrix, wherein two elements are compared with each other in terms of which importance is important and how much importance is important, the importance scale is assigned according to 1-9, and the assignment table of the importance scale 1-9 is shown in Table 1.
TABLE 1 importance Scale 1-9 assignment Table
The obtained judgment matrix is:
wherein n is the number of rows; m is the number of columns.
Calculating a weight vector after obtaining the matrix, and solving an approximate solution and a maximum eigenvalue of the eigenvector of the judgment matrix A by adopting a square root, wherein the solving process firstly calculates the square root of the product of each row of elements of the judgment matrix (formula 1) for n times as follows:
wherein n is the number of rows, i and j are elements, and the complaint judgment matrix A is a matrix with n rows and m columns. a isijIs any one element in the judgment matrix a, when i is 1, j is 1, aijIs a11And so on.
In formula 1, when i is equal to 1,
become into
Represents the multiplication of each item, wherein j ═ 1 is an initial value, n is a final value,
represents a in the decision matrix A11*a12*…*a1nThat is, the product of the first row in the determination matrix a is further divided by the root of the nth power, the weight vector of the first row is obtained, when i is 2, the weight vector of the second row is obtained, and until i is n, the weight vector of the nth row is obtained.
And carrying out normalization processing to obtain the weight of each index. Calculating the quotient of the square root and the square root sum of each row of products for n times,form a feature vector Wi＝(w1,w2,…,wn)T：
Calculating the maximum eigenvalue lambda max of a judgment matrix A, wherein A is the judgment matrix, (AW)iThe ith element representing the vector AW:
step 603 is to check the consistency of the judgment matrix, and the calculation formula of the consistency check index CR is:
where n represents the number of rows, and RI is a random consistency index, which can be obtained by looking up a table, as shown in table 2, for example: for the decision matrix of order 5, the table lookup yields RI 1.12.
The analytic hierarchy process requires consistency check after obtaining the weight vector of the judgment matrix to ensure the validity of each layer of the judgment matrix, and reflects the non-consistency severity of the judgment matrix A through the value of CR.
TABLE 2 average random consistency index RI Table
| Order of matrix | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
| RI | 0 | 0 | 0.52 | 0.89 | 1.12 | 1.26 | 1.36 | 1.41 |
| Order of matrix | 9 | 10 | 11 | 12 | 13 | 14 | 15 | |
| RI | 1.46 | 1.49 | 1.52 | 1.54 | 1.56 | 1.58 | 1.59 |
If CR is less than 0.1, the A passes the consistency test; if not, the original judgment matrix is required to be corrected until the consistency check is passed, and at the moment, the approximate solution of the feature vector is the weight coefficient of each element.
Specifically, step 604 is to perform weighting according to the weight vector
And sequentially calculating the weight of the importance of all elements of the layer to the target layer from the most standard layer to the measure layer, wherein the scheme with the largest weight value is the optimal scheme for solving the problem, and the scheme with the largest weight value is W.
Further, the calculating the qualified percentage of the data quality detection index includes:
detection rule R according to data qualityi(Ii) Detecting and analyzing the data quality detection object, and calculating to satisfy the data quality detection rule Ri(Ii) Data percentage of (S)i(Ri(Ii))；
Then, the expected value E of each data quality detection index is giveniThe quality evaluation result SA, the overall expected value SE, and the relative quantized value SR of the data quality detection target are calculated, respectively.
In addition, S isi(Ri(Ii) Is the percent pass of the data quality detection indicator, Si(Ri(Ii) (Di/Dt) × 100%, where Di represents satisfaction of the data quality detection rule Ri(Ii) Dt represents the total number of data quality detection objects; setting an expected value E of a data quality detection index according to the qualified percentage and combining with actual detection requirementsi(for example, the actual detection requirement is that the data integrity reaches 95%, and the expected value of the data quality detection index isIs 95).
In addition, the feature vector W is used as a basis for the feature vectoriPercent of pass Si[Si(Ri(Ii))]And obtaining a quality evaluation result SA of the data quality detection object:
wherein, WiRepresenting a feature vector, SiRepresenting percent qualification, i represents the ith element, and n represents the number of rows.
According to the feature vector WiExpectation value EiObtaining the overall expected value SE of the data quality detection object:
obtaining a relative quantization value of the data quality detection object according to the quality evaluation result SA of the data quality detection object and the overall expected value SE of the data quality detection object: SR-SA-SE.
Where SA reflects the true data quality condition of the data quality test object, SE reflects the overall expected value for the data quality test object, and SR reflects the data quality condition of the data quality test object relative to its expected value SE. It is considered that the data quality level of the data quality test object is evaluated as "good" if SA e (98, 100), as "good" if SA e (95, 98), as "medium" if SA e (85, 95), as "poor" if SA e (0, 85), as "good" if SR sign is positive, as the numerical value is larger, as the data quality is worse than expected, as the SR sign is negative, as the data quality is worse than expected.
Note that SA, SE, and SR reflect data quality conditions, where SA is a quality evaluation result, SE is a total expected value, and SR is a relative difference. Assuming that the calculated overall expectation value SE is 98 and the integrated evaluation value SA is 97.8, the calculated relative difference SR is-0.2. It is concluded that SA is 97.8 and meets SA epsilon (95, 98) to judge that the data quality is excellent, and SR is minus 0.2 sign, which indicates that the data quality is worse than expected, but the difference is not large, otherwise, the opposite is true.
Further, constructing a power information communication data quality detection model as follows:
M＝<D,I,R,W,Ei,SA>
in the formula: d is a data quality detection object; i is a data quality detection index set needing to be detected on D; r is a data quality detection rule set corresponding to the data quality detection index; w is a scheme E with the maximum weight value of the data quality detection indexesiIs the expected value given for R; SA is a data quality score indicating that a data quality detection object performs data quality evaluation based on a data quality detection rule.
It should be noted that the data quality detection object D, the data quality detection index set I, the data quality detection rule set R, and one of the schemes with the largest weight value are W, the expected value Ei, and the quality evaluation result SA of the data quality detection object form an electric power information communication data quality detection model.
Example 2
The embodiment provides a construction system of a power information communication data quality detection model, which comprises:
the data source model module is used for constructing a data source model by adopting an integrated storage and parallel collection processing technology based on multi-source power information communication data;
the data quality detection object module is used for determining a data quality detection object based on the data source model;
the data quality detection index module selects a data quality detection index based on actual service requirements;
the data quality detection rule module is used for designing data quality detection rules of different dimensionality indexes based on the selected data quality detection indexes;
the qualification percentage module is used for evaluating based on the data quality detection rule and calculating to obtain the qualification percentage of the data quality detection index;
the weight and expected value module is used for constructing a judgment matrix by using an analytic hierarchy process and determining the weight and the expected value of each data quality detection index;
the power information communication data quality detection model system is used for constructing a power information communication data quality detection model based on a data quality detection object module, a data quality detection index module, a data quality detection rule module, a weight and expected value module and a qualified percentage module.
According to the data quality detection object module, the data quality detection index module, the data quality detection rule module, the weight and expected value module and the qualification percentage module, the construction system for constructing the power information communication data quality detection model can accurately and effectively detect the power information communication data of multi-source information, discover the hidden danger of power data quality reduction in time and improve the management level of a power grid company on the power data quality.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.
Claims (10)
1. A construction method of a power information communication data quality detection model is characterized by comprising the following steps:
on the basis of multi-source power information communication data, a data source model is constructed by adopting an integrated storage and parallel collection processing technology;
determining a data quality detection object based on the data source model;
selecting a data quality detection index based on actual service requirements;
designing data quality detection rules of different dimensionality indexes based on the selected data quality detection indexes;
evaluating based on the data quality detection rule, and calculating to obtain the qualified percentage of the data quality detection index;
constructing a judgment matrix by using an analytic hierarchy process, and determining the weight and the expected value of each data quality detection index;
and constructing a power information communication data quality detection model based on the data quality detection object, the data quality detection index, the data quality detection rule, the weight and expected value and the qualified percentage.
2. The method for constructing a power information communication data quality detection model according to claim 1, wherein the multi-source power information communication network data comprises operation data, production data, ERP data and marketing data.
3. The method for constructing the power information communication data quality detection model according to claim 1, wherein the data quality detection object is a data item or a data set, wherein the data set is a collection of data items and can be divided into subclasses of the data quality detection object.
4. The method for constructing the power information communication data quality detection model according to claim 1, wherein the data quality detection indexes comprise correctness, completeness, uniqueness, accuracy, validity, consistency and timeliness.
5. The method for constructing the power information communication data quality detection model according to claim 4, wherein the correctness refers to whether the data quality detection object conforms to objective facts or not, and whether errors occur in the data quality detection object collection, transmission, dump and other processes; the integrity refers to whether the data quality detection object has missing records; the uniqueness refers to whether similar repeated records exist in the data quality detection object or not; the accuracy refers to whether the precision of the data quality detection object meets the requirement or not; the timeliness refers to whether the data quality detection object is still effective under the current condition; the consistency refers to whether the expression formats of the data quality detection objects are consistent or not; the validity refers to whether the expression format and the numerical value of the data quality detection object are valid or not.
6. The method for constructing the power information communication data quality detection model according to claim 1, wherein designing the data quality detection rule comprises:
designing a data quality detection rule R by referring to the data quality detection indexi(Ii) Each data quality detection index corresponds to a data quality detection rule.
7. The method for constructing the power information communication data quality detection model according to claim 1, wherein the determining the weight and the expected value of each data quality detection index by using an analytic hierarchy process comprises:
layering the problems to form a hierarchical structure consisting of a target layer, a criterion layer and a measure layer;
forming a judgment matrix by adopting a 1-9 scaling method;
checking the consistency of the judgment matrix based on the analytic hierarchy process;
and calculating layer by layer to obtain the weight of the scheme layer to the target layer, wherein the scheme with the maximum specific gravity is the optimal scheme for solving the problem.
8. The method for constructing the power information communication data quality detection model according to claim 6, wherein calculating the qualified percentage of the data quality detection index comprises:
according to the data quality detection rule Ri(Ii) Detecting and analyzing the data quality detection object, and calculating to meet the data quality detection rule Ri(Ii) Data percentage of (S)i(Ri(Ii))；
Then, an expected value E of each data quality detection index is giveniAnd respectively calculating a quality evaluation result SA, a total expected value SE and a relative quantized value SR of the data quality detection object.
9. The method for constructing the power information communication data quality detection model according to claim 8, wherein the method for constructing the power information communication data quality detection model comprises the following steps:
M＝<D,I,R,W,E,S>
in the formula: d is a data quality detection object; i is a data quality detection index set needing to be detected on D; r is a data quality detection rule set corresponding to the data quality detection index; w is the weight of the data quality detection index; e is the expected value given for R; and S is data quality scoring for performing data quality evaluation on the data quality detection object based on the data quality detection rule.
10. A construction system of a power information communication data quality detection model is characterized by comprising the following components:
the data source model module is used for constructing a data source model by adopting an integrated storage and parallel collection processing technology based on multi-source power information communication data;
the data quality detection object module is used for determining a data quality detection object based on the data source model;
the data quality detection index module selects a data quality detection index based on actual service requirements;
the data quality detection rule module is used for designing data quality detection rules of different dimensionality indexes based on the selected data quality detection indexes;
the qualified percentage module is used for evaluating based on the data quality detection rule and calculating to obtain the qualified percentage of the data quality detection index;
the weight and expected value module is used for constructing a judgment matrix by using an analytic hierarchy process and determining the weight and the expected value of each data quality detection index;
the power information communication data quality detection model system is used for constructing a power information communication data quality detection model based on the data quality detection object module, the data quality detection index module, the data quality detection rule module, the weight and expected value module and the qualified percentage module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111596943.1A CN114266485A (en) | 2021-12-24 | 2021-12-24 | Construction method and construction system of power information communication data quality detection model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111596943.1A CN114266485A (en) | 2021-12-24 | 2021-12-24 | Construction method and construction system of power information communication data quality detection model |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114266485A true CN114266485A (en) | 2022-04-01 |
Family
ID=80829547
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111596943.1A Pending CN114266485A (en) | 2021-12-24 | 2021-12-24 | Construction method and construction system of power information communication data quality detection model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114266485A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115077618A (en) * | 2022-06-27 | 2022-09-20 | 扬州市管件厂有限公司 | Quality detection method and system for nuclear-grade alloy steel elbow |
-
2021
- 2021-12-24 CN CN202111596943.1A patent/CN114266485A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115077618A (en) * | 2022-06-27 | 2022-09-20 | 扬州市管件厂有限公司 | Quality detection method and system for nuclear-grade alloy steel elbow |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103247008A (en) | Quality evaluation method of electricity statistical index data | |
| CN106845846A (en) | Big data asset evaluation method | |
| CN103971023B (en) | R＆D process quality automatic evaluation system and method | |
| CN111260198A (en) | Method and system for judging degree of rationality of line loss in transformer area synchronization and terminal equipment | |
| CN111882198A (en) | Project performance evaluation method and system | |
| CN115660170A (en) | Multidimensional index weight collaborative optimization data asset management effect differentiation evaluation method and system | |
| CN114266485A (en) | Construction method and construction system of power information communication data quality detection model | |
| CN108805471A (en) | Evaluation method for water resources carrying capacity based on the analysis of hybrid system interactively | |
| CN114881485A (en) | Enterprise fund risk assessment method based on analytic hierarchy process and cloud model | |
| CN107463532A (en) | A kind of mass analysis method of electric power statistics | |
| CN102468997A (en) | Method for assessing stability of safety index system of multidimension network | |
| CN114091908A (en) | Power distribution network comprehensive evaluation method, device and equipment considering multi-mode energy storage station | |
| CN111832854A (en) | Maturity quantitative evaluation method and system for automobile research and development quality management system and readable medium | |
| Datta | Industrial sickness in India–An empirical analysis | |
| CN114913033A (en) | Power grid diversity data hierarchical storage balancing and transaction safety method and system | |
| CN108694527B (en) | Power distribution network evaluation method | |
| JAIN | JOSY GEORGE, SANDEEP KUMAR TEMBHARE, and | |
| CN114428730A (en) | Industry innovation application replacement sequence evaluation method and system based on analytic hierarchy process | |
| CN116109175A (en) | Power system risk analysis method and device | |
| CN117040020A (en) | Solving method for regional integral photovoltaic new energy source dissipatable capacity index distribution | |
| CN117853225A (en) | Credit evaluation method for debt subject | |
| CN115270610A (en) | Auxiliary terminal acquisition strategy optimization method | |
| CN114565307A (en) | Evaluation method, device and equipment based on measurement technology and readable storage medium | |
| Fikri et al. | Trend Analysis in Sales Forecasting and Decision Support Systems AHP Method on the Selection of Types of Motorcycles PT. AHM | |
| CN114240603A (en) | Data processing method and device for constructing trust model and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |