CN112463807A - Data processing method, device, server and storage medium - Google Patents
Data processing method, device, server and storage medium Download PDFInfo
- Publication number
- CN112463807A CN112463807A CN202011313950.1A CN202011313950A CN112463807A CN 112463807 A CN112463807 A CN 112463807A CN 202011313950 A CN202011313950 A CN 202011313950A CN 112463807 A CN112463807 A CN 112463807A
- Authority
- CN
- China
- Prior art keywords
- data
- business rule
- transformer
- determining
- consistency
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 22
- 238000003860 storage Methods 0.000 title claims description 22
- 238000004519 manufacturing process Methods 0.000 claims abstract description 57
- 238000000034 method Methods 0.000 claims abstract description 24
- 230000002159 abnormal effect Effects 0.000 claims description 22
- 238000012545 processing Methods 0.000 claims description 16
- 238000004891 communication Methods 0.000 claims description 7
- 238000013500 data storage Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 4
- 238000012544 monitoring process Methods 0.000 abstract description 13
- 238000013523 data management Methods 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 7
- 230000008859 change Effects 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 238000009826 distribution Methods 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000035515 penetration Effects 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/29—Geographical information databases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Remote Sensing (AREA)
- Computational Linguistics (AREA)
- Computer Security & Cryptography (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The embodiment of the invention discloses a data processing method. The method comprises the following steps: acquiring first data in a marketing system database, second data in a GIS system database and third data in a production system database; calling a target business rule from a business rule base, carrying out consistency comparison on the first data, the second data and the third data, and determining problem data according to a comparison result; and storing and visually displaying the problem data. The technical scheme of the embodiment of the invention realizes more comprehensive, rapid and accurate monitoring of problem data, thereby improving the data management efficiency.
Description
Technical Field
The embodiment of the invention relates to the technical field of power system operation and distribution penetration, in particular to a data processing method, a data processing device, a server and a storage medium.
Background
The power informatization System relates to a plurality of sub-business systems, including a marketing management System, a production management System, a metering automation System, a Geographic Information System (GIS) and the like, data are circulated in each sub-System, and therefore data consistency among the sub-systems faces a huge challenge.
At present, management of marketing and distribution data quality mostly carries out problem location on problem data through knowledge and experience of technicians so as to detect whether data inconsistency exists among subsystems.
When the problem data are manually monitored, the problems of difficulty in comprehensive monitoring, poor problem positioning real-time performance and unclear positioning exist.
Disclosure of Invention
The invention provides a data processing method, a data processing device, a server and a storage medium, which are used for realizing more comprehensive, rapid and accurate monitoring of problem data, thereby improving the data management efficiency.
In a first aspect, an embodiment of the present invention provides a data processing method, including:
acquiring first data in a marketing system database, second data in a GIS system database and third data in a production system database;
calling a target business rule from a business rule base, carrying out consistency comparison on the first data, the second data and the third data, and determining problem data according to a comparison result;
and storing and visually displaying the problem data.
In a second aspect, an embodiment of the present invention further provides a data processing apparatus, where the apparatus includes:
the data acquisition module is used for acquiring first data in a marketing system database, second data in a GIS system database and third data in a production system database;
the problem data determining module is used for calling a target business rule from a business rule base, comparing the consistency of the first data, the second data and the third data and determining problem data according to a comparison result;
and the problem data storage and display module is used for storing and visually displaying the problem data.
In a third aspect, an embodiment of the present invention further provides a server, where the server includes:
one or more processors;
a storage device for storing one or more programs which, when executed by the processor, cause the processor to implement a data processing method as provided by any of the embodiments of the invention.
In a fourth aspect, the embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the data extracting method provided in any of the embodiments of the present invention.
According to the technical scheme of the embodiment of the invention, the first data in the marketing system database, the second data in the GIS system database and the third data in the production system database are acquired, the target business rule is called from the business rule base, the first data, the second data and the third data are compared in a consistency mode, the problem data are determined according to the comparison result, the problem data in the marketing system database, the GIS system database and the production system database are effectively analyzed through the business rule, not only is the manpower saved, but also the analysis efficiency of the problem data is improved, the problem data are stored and visually displayed, the problem data can be more accurately, visually and timely positioned through storage and visual display of the problem data, and compared with a method for manually monitoring the problem data in the related technology, the problem data in the marketing system database, the GIS system database and the production system database are monitored in real time, The problem that the data of the GIS system database and the data of the production system database are inconsistent is solved, real-time monitoring of problem data and dynamic display of the problem data are achieved, and the treatment efficiency of the data is greatly improved.
Drawings
In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, a brief description is given below of the drawings used in describing the embodiments. It should be clear that the described figures are only views of some of the embodiments of the invention to be described, not all, and that for a person skilled in the art, other figures can be derived from these figures without inventive effort.
Fig. 1 is a schematic flow chart of a data processing method according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a data processing method according to a second embodiment of the present invention;
fig. 3 is a schematic flow chart of a data processing method according to a third embodiment of the present invention;
fig. 4 is a block diagram of a data processing apparatus according to a fourth embodiment of the present invention;
fig. 5 is a schematic structural diagram of a server according to a fifth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1 is a flowchart of a data processing method according to an embodiment of the present invention, where the embodiment is applicable to a case where problem data is obtained by performing consistency comparison on data by invoking a business rule, and the method may be executed by a data processing apparatus, where the data processing apparatus may be implemented by software and/or hardware, and the data processing apparatus may be integrated in an electronic device such as a computer or a server.
The method comprises the following steps:
s110, acquiring first data in a marketing system database, second data in a GIS system database and third data in a production system database.
The first data, the second data and the third data may be data in which the same data flows in different databases, where the first data is data acquired from a marketing system database, the second data is data acquired from a GIS system database, and the third data is data acquired from a production system database, where the field names of the first data, the second data and the third data in the marketing system database, the GIS system database and the production system database respectively may be the same or different, and are not limited herein, for example, the field name of the transformer number in the marketing system database may be YB _ ID, the field name in the GIS system database may be GB _ ID, and the field name in the production system database may be SB _ ID.
The same data flows in different databases, the data are not changed, in one embodiment, one piece of data with the transformer number of 1 in the marketing system database is obtained, namely the transformer number is 1, the rated capacity of the transformer is 1, the running state is 1, and the user number is 1; acquiring a piece of data with a transformer number of 1 in a GIS system database, namely the rated capacity of the transformer is 1, the running state is 1 and the user number is 1; acquiring a piece of data with the transformer number of 1 in a production system database, namely the transformer number of 1, the rated capacity of the transformer of 1, the running state of 1 and the user number of 1.
The same data circulates in different databases, the data changes, and in another embodiment, one piece of data with the transformer number of 1 in a marketing system database is obtained, namely the transformer number is 1, the rated capacity of the transformer is 1, the running state is 2, and the user number is 1; acquiring a piece of data with a transformer number of 1 in a GIS system database, namely the rated capacity of the transformer is 1, the running state is 1 and the user number is 1; acquiring a piece of data with the transformer number of 1 in a production system database, namely the transformer number of 1, the rated capacity of the transformer of 2, the running state of 1 and the user number of 1.
It should be noted that the first data, the second data, and the third data are not in sequence here, and are used to distinguish specific data in different databases.
And S120, calling a target business rule from the business rule base, performing consistency comparison on the first data, the second data and the third data, and determining problem data according to a comparison result.
The business rule base comprises a plurality of business rules belonging to different business types, and the business rules can be added, modified and deleted in the business rule base. Business rules can be understood as descriptions of business definitions and constraints for maintaining business structure or controlling and affecting business behavior. Each business rule defines a judgment. The business rules may be one or more rules that are pre-configured, where different business rules are used to create rules that meet different requirements. The business rule of this embodiment may be a business rule that detects whether data in different databases of the same data are consistent. The target business rules can be, but are not limited to, business rules for detecting transformer data, business rules for detecting GIS system topology relation data, business rules for monitoring the metering points and power point data of marketing system users, and business rules for detecting cooperative data. The problem data may be data in which the first data, the second data, and the third data meet the business rule, wherein the problem data may be one data or a plurality of data.
Specifically, a target business rule is selected from a plurality of business rules in a business rule base, data consistency comparison is carried out on first data in a marketing system database, second data in a GIS system database and third data in a production system database through the target business rule to obtain a comparison result, and problem data are determined according to the comparison result.
Exemplarily, when the number of the transformer is 1, the rated capacity of the transformer is 1, the operation state is 1 and the number of the user is 1 in the first data in the marketing system database; the number of a transformer in second data in a GIS system database is 1, the rated capacity of the transformer is 1, the running state is 2 and the number of a user is 1; when the number of a third data transformer in a production system database is 1, the rated capacity of the transformer is 1, the running state is 2 and the user number is 1, determining that the data type is transformer data, the called target business rule is a business rule of transformer data consistency, comparing the first data, the second data and the third data by taking the business rule of transformer data consistency as the target business rule, determining that the running state in the first data is 1, the running state in the second data is 2 and the running state in the third data is 2 through comparison, obtaining a comparison result of inconsistent running states in the three data, and determining that the first data, the second data and the state data of the transformer 1 in the third data are problem data according to the comparison result of inconsistent states.
And S130, storing and visually displaying the problem data.
Specifically, based on the determined problem data, the problem data is stored and visually displayed, so that the worker can timely position the problem data and solve the problem data. Optionally, the problem data is determined by comparing the business rules, and the problem data may be stored in a database, acquired from the database and displayed. Optionally, the problem data is stored, and the problem data is visually displayed at the same time.
In one embodiment, the state data in the first data, the state data in the second data and the state data in the third data are compared through the target business rule, when the state data have inconsistent data values, the state data of the first data, the state data of the second data and the state data of the third data are problem data, the state data of the first data, the state data of the second data and the state data of the third data are stored in a database, and the state data are displayed.
In another embodiment, when it is determined that problem data exists through business rule comparison, a data line in which the problem data exists is stored as the problem data, and for example, when it is determined that an inconsistency problem exists in the state data, a data line in which the state data in the first data, the second data, and the third data exists is determined as the problem data, for example, the data line may further include a transformer number and a user number. Optionally, the problem data is labeled according to the comparison result, so that relevant personnel can process the problem data in time.
Optionally, storing and visually displaying the problem data includes: and counting the continuous historical days of each question data, and displaying the question data and the corresponding historical days in a visual mode.
Specifically, the problem data are stored in a database, historical days of each problem data in the database are counted, and the problem data and the historical days of the corresponding problem data are visually displayed, so that relevant personnel can count the problem data and plan subsequent work arrangement conveniently. Optionally, the visual display is divided into two parts, one part may be a list of problem data of the current day, and the other part may be statistics of the number of the problem data of each day in the aspects of addition, reduction, modification and non-modification, so that service personnel can sort or screen the problem data by the continuous occurrence days of the problem data, effectively perform batch processing on the problem data according to the occurrence history days, perform cause analysis on the newly added problems first, perform major leak stoppage, and appropriately adjust the priority of the problem data for the problem data accumulated for a long time, so as to avoid unnecessary repeated labor brought to the service personnel by the daily change of the problem data list.
Further, in order to facilitate the workers to know the quantity change condition of the problem data in time and quickly process the problem data, optionally, the change condition of the data quantity of the problem data in a preset time period is counted, for example, the problem data quantity is newly increased, the problem data quantity is reduced, the problem data quantity is modified, the problem data quantity is not modified, and the like, so that the change condition is used as reference data for performance assessment of the related workers.
According to the technical scheme of the embodiment of the invention, the first data in the marketing system database, the second data in the GIS system database and the third data in the production system database are acquired, the target business rule is called from the business rule base, the first data, the second data and the third data are compared in a consistency mode, the problem data are determined according to the comparison result, the problem data in the marketing system database, the GIS system database and the production system database are effectively analyzed through the business rule, not only is the manpower saved, but also the analysis efficiency of the problem data is improved, the problem data are stored and visually displayed, the problem data can be more accurately, visually and timely positioned through storage and visual display of the problem data, and compared with a method for manually monitoring the problem data in the related technology, the problem data in the marketing system database, the GIS system database and the production system database are monitored in real time, The problem that the data of the GIS system database and the data of the production system database are inconsistent is solved, real-time monitoring of problem data and dynamic display of the problem data are achieved, and the treatment efficiency of the data is greatly improved.
Example two
Fig. 2 is a schematic flow chart of a data processing method according to a second embodiment of the present invention, and on the basis of the foregoing embodiment, the present embodiment details a method for comparing data consistency.
The technical terms that are the same as or corresponding to the above embodiments are not repeated herein.
As shown in fig. 2, the method of the embodiment may specifically include:
s210, acquiring first data in a marketing system database, second data in a GIS system database and third data in a production system database.
S220, calling a target business rule corresponding to the data type from a business rule base according to the data type, determining to-be-detected data in the first data, the second data and the third data based on the data type, and comparing the to-be-detected data based on the target business rule.
The data types of the first data, the second data and the third data comprise at least one of transformer data, GIS system topological relation data, marketing system user metering points, power point data and cooperation data. The data types may include, but are not limited to, transformer data types, GIS system topology relationship data types, marketing system user metering points and power point data types, and collaborative data types. The data to be measured may be understood as data obtained from the first data, the second data, and the third data according to the data type, and the data to be measured may be detected data.
Specifically, the business rule corresponding to the data type is obtained from the business rule base according to the data type and serves as a target business rule, the data to be detected corresponding to the data type in the first data, the second data and the third data are determined according to the data type, and the data to be detected in the first data, the data to be detected in the second data and the data to be detected in the third data are compared through the target business data.
Illustratively, based on the obtained first data being the transformer number 1, the rated capacity of the transformer being the capacity 1, the operating state being the state 1, and the user number being 1, the second data being the transformer number 1, the rated capacity of the transformer being the capacity 1, the operating state being the state 1, and the user number being 2, and the third data being the transformer number 1, the rated capacity of the transformer being the capacity 1, the operating state being the state 1, and the user number being 1, the service rule of the transformer data type is called from the service database, and the transformer number, the rated capacity of the transformer, the operating state being the state, and the user number in the first data are compared.
In order to comprehensively monitor the condition of the same data in different databases, the data can be detected by setting service rules of multiple service types, wherein the target service rules can include but are not limited to a target service rule corresponding to transformer data, a target service rule corresponding to GIS system topological relation data, a target service rule corresponding to a marketing system user metering point and power supply point data, and a target service rule corresponding to cooperative data.
Optionally, the target business rule corresponding to the transformer data includes: one or more of abnormal rated capacity, abnormal user number, abnormal transformer number and abnormal running state.
The abnormal rated capacity can be a transformer of the same GISID, and the rated capacities in the GIS system, the marketing system and the production system are not completely the same, namely, the rated capacities of the GIS system and the marketing system are inconsistent, the rated capacities of the production system and the marketing system are inconsistent, and the GIS function position and the rated capacity of the production system equipment ledger are inconsistent.
The abnormal user numbers can be transformers of the same GISID, and the user numbers in the GIS system, the marketing system and the production system are not completely the same, namely, the user numbers of the GIS system and the marketing system are not consistent, the user numbers of the production system and the marketing system are not consistent, and the GIS function position is not consistent with the user numbers in the production system equipment ledger.
The abnormal number of the transformer can be the number of the transformer of the marketing system, and is not completely consistent in the marketing system, the GIS system and the production system, namely, the number of the transformer of the GIS system is inconsistent with the number of the transformer of the marketing system, the number of the transformer of the production system is inconsistent with the number of the transformer of the marketing system, and the position of the GIS function is inconsistent with the equipment account of the marketing system.
The abnormal operation state can be the transformer of the same GISID, and the operation state of the transformer is not completely consistent in the marketing system, the production system and the marketing system, that is, the operation state of the transformer in the GIS system is inconsistent with the operation state of the transformer in the production system, the operation state of the transformer in the GIS system is inconsistent with the operation state of the transformer in the marketing system, and the GIS function position is inconsistent with the equipment account of the marketing system.
The target business rule corresponding to the GIS system topological relation data comprises: one or more of the medium-voltage topological communication rate, the low-voltage topological communication rate, the fire point inconsistency with the affiliated platform area of the hitching line. The target business rule corresponding to the marketing system user metering point and the power supply point data comprises the following steps: the number of the low-voltage user volume point belonging station area is abnormal, the number of the low-voltage user volume point belonging station area is empty, the number of the medium-voltage user power supply information power supply line is abnormal, the low-voltage user volume point is inconsistent with the station area information of the power supply information, the medium-voltage user volume point is inconsistent with the power supply information power supply line information, a transformer with a missing line change relationship, and a transformer with a plurality of line change relationships.
The target business rule corresponding to the collaborative data comprises the following steps: any one of the station line relation, the line-to-line relation and the household variable relation is inconsistent, and/or any one of a transformer substation, a feeder line, a transformer, a low-voltage user and a medium-voltage user is inconsistent.
The inconsistency of the transformer substation can include the existence of a GIS system, the absence of a production system, the existence of the GIS system, the existence of the production system, the existence of a marketing system, the absence of the production system and the existence of the marketing system.
The feeder line inconsistency can include that the GIS system is available, the production system is unavailable, the GIS system is unavailable, the production system is available, the marketing system is unavailable, the production system is unavailable, the marketing system is available, the marketing system line GISID is empty, and the marketing system line GISID is repeated.
The transformer inconsistency comprises that the GIS system is available, the production system is unavailable, the GIS system is unavailable, the production system is available, the marketing system is unavailable, the production system is unavailable, the marketing system is available, the marketing system transformer GISID is empty, and the marketing system transformer GISID is repeated.
The low-voltage user inconsistency comprises the existence of a GIS system, the absence of a marketing system, the existence of the GIS system and the existence of the marketing system, and the existence of the GIS system but the marketing system sells the user.
The inconsistency of the medium-voltage users comprises the inconsistency of the transformer which the GIS system and the users of the marketing system belong to.
On the basis, the data type further comprises an account card object data type, wherein the business rule corresponding to the account card object data comprises production, financial and distribution network, production, financial and main network, financial and production and distribution network, and financial and production are respectively not and main network.
And S230, when the data to be detected in any two items of the first data, the second data and the third data are inconsistent, determining that the data to be detected is problem data.
Specifically, the data to be detected in the first data, the data to be detected in the second data and the data to be detected in the third data are compared in pairs, that is, the data to be detected in the first data is compared with the data to be detected in the second data, the data to be detected in the first data is compared with the data to be detected in the third data, the data to be detected in the second data is compared with the data to be detected in the third data, when the data to be detected in pairs are consistent, the data to be detected is determined to be normal data, and when the data to be detected in pairs are inconsistent, the data to be detected is determined to be problem data.
And S240, storing and visually displaying the problem data.
According to the technical scheme, a first data in a marketing system database, a second data in a GIS system database and a third data in a production system database are obtained, a target business rule corresponding to a data type is called from a business rule base according to the data type, data to be detected in the first data, the second data and the third data are determined based on the data type, the data to be detected are compared based on the target business rule, when the data to be detected in any two items of the first data, the second data and the third data are inconsistent, the data to be detected are determined to be problem data, and the problem data are stored and visually displayed. The problem of inconsistent data of a real-time monitoring marketing system database, a GIS system database and a production system database is solved, real-time monitoring of problem data and dynamic display of the problem data are achieved, and the treatment efficiency of the data is greatly improved.
EXAMPLE III
Fig. 3 is a schematic flow chart of a data processing method according to a second embodiment of the present invention, and on the basis of the foregoing embodiment, optionally, the data processing method according to this embodiment may further include: and obtaining the data consistency rate of the database according to the comparison result. Specifically, according to the comparison result, data consistency rates of any two databases are determined, wherein the data consistency rates include an object consistency rate and a topological relation consistency rate.
The technical terms that are the same as or corresponding to the above embodiments are not repeated herein.
As shown in fig. 3, the method of the present embodiment may specifically include:
s310, acquiring first data in a marketing system database, second data in a GIS system database and third data in a production system database.
S320, calling the target business rule from the business rule base, and comparing the consistency of the first data, the second data and the third data.
And S330, determining problem data according to the comparison result.
And S340, determining the data consistency rates of any two databases according to the comparison result, wherein the data consistency rates comprise an object consistency rate and a topological relation consistency rate.
The data consistency rate can be the data consistency rate in any two databases of the marketing system database, the GIS system database and the production system database, and is used for counting the data consistency rate between the two databases so as to further improve the system by workers in the later period. The data consistency rate may be an object consistency rate or a topological relation consistency rate, wherein the object consistency rate may be a substation consistency rate, a feeder line consistency rate, a transformer consistency rate, a low-voltage user consistency rate, and a medium-voltage user consistency rate, and the topological relation consistency rate may be a station-line relation consistency rate, a line-change relation consistency rate, and a user-change relation consistency rate.
It should be noted that there is no precedence order between steps S330 and S340, and the steps may be executed sequentially or synchronously.
Optionally, the determining the data consistency rate of any two databases according to the comparison result includes: determining the total number of comparison data volumes in any two databases and a first data volume of objects which are successfully compared continuously in comparison results, and determining the object consistency rate of any two databases based on the first data volume and the total number of the comparison data volumes; and in the data successfully compared with the object continuously, determining a second data volume successfully compared with the topological relation, and determining the consistency rate of the topological relation of any two databases based on the second data volume and the first data volume.
The method comprises the steps of determining the total number of comparison data volumes in any two databases and a first data volume of objects which are successfully compared continuously in comparison results, and determining the object consistency rate of any two databases based on the first data volume and the total number of the comparison data volumes.
Specifically, the object matching rate of the two databases may be a ratio of 2 times of the data amount of the two databases in which the object is successfully compared in succession to the total number of the compared data amounts of the two databases.
Illustratively, the data number of the transformer data in the marketing system database is from 1 to 150, the data number of the transformer data in the GIS system database is from 1 to 150, wherein the comparison result of the transformer numbers of the transformer data from 1 to 100 is consistent, and the data number of 101 has data comparison abnormality, the data volume of the object with continuous comparison success is 100, and the transformer object consistency ratio of the marketing system database and the GIS system database is (100 × 2)/(150+150) ═ 2/3.
And in the data successfully compared with the object continuously, determining a second data volume successfully compared with the topological relation, and determining the consistency rate of the topological relation of any two databases based on the second data volume and the first data volume.
The calculation of the topological relation consistency rate is obtained on the basis of object comparison, wherein the topological relation is the dependency relation between the objects, for example, the transformer substation and the feeder line are in one-to-many relationship.
Specifically, for the data volume of the object successfully compared, the topological relation in the two databases is compared, and the data volume of the successfully compared topological relation is determined, for example, the topological relation comparison may be the comparison between the feeder line corresponding to the transformer number 1 in the marketing system database and the feeder line corresponding to the transformer number 1 in the GIS system database.
The consistency rate of the topological relations of the two databases can be a ratio of 2 times of the data quantity of the successfully compared topological relations to the sum of the data quantities of the successfully compared objects in the two databases.
Illustratively, the comparison result of the number of the transformers in the marketing system database and the GIS system database from 1 to 100 is consistent, wherein the number of the transformers in the marketing system database and the GIS system database, which are successfully compared in the topological relation, is 90, and the topological consistency ratio of the marketing system database and the GIS system database is (90 × 2)/(100+100) ═ 9/10.
And S350, storing and visually displaying the problem data.
In the technical scheme of this embodiment, by obtaining first data in a marketing system database, second data in a GIS system database, and third data in a production system database, calling a target business rule from a business rule base, performing consistency comparison on the first data, the second data, and the third data, determining problem data according to a comparison result, or determining a data consistency rate of any two databases according to the comparison result, wherein the data consistency rate includes an object consistency rate and a topological relation consistency rate, determining an object consistency rate of any two databases by determining a total number of comparison data amounts in any two databases and a first data amount of an object in the comparison result, determining the object consistency rate of any two databases based on the first data amount and the total number of the comparison data amounts, and determining, in the data of the object continuous comparison success, and determining a second data volume for which the comparison of the topological relations is successful, and determining the consistency rate of the topological relations of any two databases based on the second data volume and the first data volume, so that the problem of database data quality detection is solved, and the treatment efficiency of problem data is improved.
Example four
Fig. 4 is a schematic block diagram of a data processing apparatus according to a fourth embodiment of the present invention, where the data processing apparatus includes:
the data acquisition module 410 is used for acquiring first data in a marketing system database, second data in a GIS system database and third data in a production system database;
a problem data determining module 420, configured to invoke a target business rule from a business rule base, perform consistency comparison on the first data, the second data, and the third data, and determine problem data according to a comparison result;
and the question data storage and display module 430 is used for storing and visually displaying the question data.
According to the technical scheme of the embodiment, the first data in the marketing system database, the second data in the GIS system database and the third data in the production system database are acquired through the data acquisition module, the target business rule is called from the business rule base through the problem data determination module, the first data, the second data and the third data are compared in a consistency mode, the problem data are determined according to the comparison result, the problem data of the marketing system database, the GIS system database and the production system database are effectively analyzed through the business rule, the labor is saved, the analysis efficiency of the problem data is improved, the problem data are stored and visually displayed through the problem data storage and display module, the problem data are stored and visually displayed, and the problem data can be more accurately, visually and in real time positioned, compared with a method for manually monitoring problem data in the related technology, the method solves the problem that the data of a real-time monitoring marketing system database, a GIS system database and a production system database are inconsistent, realizes the real-time monitoring of the problem data and the dynamic display of the problem data, and greatly improves the treatment efficiency of the data.
Optionally, the problem data determining module 420 is configured to invoke a target business rule corresponding to the data type from the business rule base according to the data type;
determining data to be tested in the first data, the second data and the third data based on the data types, and comparing the data to be tested based on the target business rule, wherein the first data, the second data and the third data are respectively at least one of the following data types: transformer data, GIS system topological relation data, marketing system user metering points, power point data and cooperation data.
Optionally, the problem data storage and display module 430 is configured to determine that the data to be detected is problem data when the data to be detected in any two items of the first data, the second data, and the third data are inconsistent.
Optionally, the problem determining module 420 is configured to apply the target business rule corresponding to the transformer data to include: one or more of abnormal rated capacity, abnormal user number, abnormal transformer number and abnormal running state;
the target business rule corresponding to the GIS system topological relation data comprises: one or more of the medium-voltage topological communication rate, the low-voltage topological communication rate, the fire point inconsistency with the affiliated platform area of the hanging line;
the target business rule corresponding to the marketing system user metering point and the power supply point data comprises the following steps: one or more of the transformer with the missing line variable relationship and the transformer with the multiple line variable relationships are selected from the group consisting of abnormal numbering of a transformer area to which the low-voltage user volume point belongs, empty numbering of a power supply line for medium-voltage user power supply information, abnormal numbering of a power supply line for medium-voltage user power supply information, inconsistent information of the transformer area to which the low-voltage user volume point and the power supply information belong, inconsistent information of the transformer area to which the medium-voltage user volume point and the power supply information power supply line are;
the target business rule corresponding to the collaborative data comprises: any one of the station line relation, the line-to-line relation and the household variable relation is inconsistent, and/or any one of a transformer substation, a feeder line, a transformer, a low-voltage user and a medium-voltage user is inconsistent.
Optionally, the question data storage and display module 430 is configured to count consecutive historical days of each question data, and visually display the question data and the corresponding historical days.
Optionally, the apparatus further comprises: a consistency rate determining module 440, configured to determine a data consistency rate of any two databases according to the comparison result, where the data consistency rate includes an object consistency rate and a topological relation consistency rate.
Optionally, the consistency rate determining module 440 is configured to determine a total number of comparison data amounts in any two databases and a first data amount of the objects that are successfully compared continuously in the comparison result, and determine an object consistency rate of any two databases based on the first data amount and the total number of the comparison data amounts;
and determining a second data volume of which the topological relation comparison is successful in the data of which the object continuous comparison is successful, and determining the consistency rate of the topological relation of any two databases based on the second data volume and the first data volume.
The device can execute the data processing method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of executing the data processing method.
EXAMPLE five
Fig. 5 is a schematic structural diagram of a server according to a fifth embodiment of the present invention. FIG. 5 illustrates a block diagram of an exemplary server 12 suitable for use in implementing embodiments of the present invention. The server 12 shown in fig. 5 is only an example, and should not bring any limitation to the function and the scope of use of the embodiment of the present invention.
As shown in FIG. 5, the server 12 is in the form of a general purpose computing device. The components of the server 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 and the processing unit 16.
The server 12 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by server 12 and includes both volatile and nonvolatile media, removable and non-removable media.
The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)30 and/or cache memory 32. The server 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 5, and commonly referred to as a "hard drive"). Although not shown in FIG. 5, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally carry out the functions and/or methodologies of the described embodiments of the invention.
The server 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), with one or more devices that enable a user to interact with the server 12, and/or with any devices (e.g., network card, modem, etc.) that enable the server 12 to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface 22. Also, the server 12 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet) via the network adapter 20. As shown, the network adapter 20 communicates with the other modules of the server 12 via the bus 18. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the server 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
The processing unit 16 executes various functional applications and data processing by executing programs stored in the system memory 28, for example, implementing a data processing method provided by the embodiment of the present invention, the method including:
acquiring first data in a marketing system database, second data in a GIS system database and third data in a production system database;
calling a target business rule from a business rule base, carrying out consistency comparison on the first data, the second data and the third data, and determining problem data according to a comparison result;
and storing and visually displaying the problem data.
Of course, those skilled in the art can understand that the processor can also implement the technical solution of the data processing method provided by any embodiment of the present invention.
EXAMPLE six
An embodiment of the present invention also provides a storage medium containing computer-executable instructions, which when executed by a computer processor, perform a data processing method.
The method comprises the following steps:
acquiring first data in a marketing system database, second data in a GIS system database and third data in a production system database;
calling a target business rule from a business rule base, carrying out consistency comparison on the first data, the second data and the third data, and determining problem data according to a comparison result;
and storing and visually displaying the problem data.
Computer storage media for embodiments of the invention may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for embodiments of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. A data processing method, comprising:
acquiring first data in a marketing system database, second data in a GIS system database and third data in a production system database;
calling a target business rule from a business rule base, carrying out consistency comparison on the first data, the second data and the third data, and determining problem data according to a comparison result;
and storing and visually displaying the problem data.
2. The method of claim 1, wherein the first data, the second data, and the third data are each at least one of the following data types: transformer data, GIS system topological relation data, marketing system user metering points, power point data and cooperation data;
the calling a target business rule from a business rule base to compare the consistency of the first data, the second data and the third data comprises:
calling a target business rule corresponding to the data type from the business rule base according to the data type;
determining data to be tested in the first data, the second data and the third data based on the data types, and comparing the data to be tested based on the target business rule.
3. The method of claim 2, wherein determining the problem data based on the comparison comprises:
and when the data to be tested in any two items of the first data, the second data and the third data are inconsistent, determining that the data to be tested is problem data.
4. The method of claim 2,
the target business rule corresponding to the transformer data comprises the following steps: one or more of abnormal rated capacity, abnormal user number, abnormal transformer number and abnormal running state;
the target business rule corresponding to the GIS system topological relation data comprises: one or more of the medium-voltage topological communication rate, the low-voltage topological communication rate, the fire point inconsistency with the affiliated platform area of the hanging line;
the target business rule corresponding to the marketing system user metering point and the power supply point data comprises the following steps: one or more of the transformer with the missing line variable relationship and the transformer with the multiple line variable relationships are selected from the group consisting of abnormal numbering of a transformer area to which the low-voltage user volume point belongs, empty numbering of a power supply line for medium-voltage user power supply information, abnormal numbering of a power supply line for medium-voltage user power supply information, inconsistent information of the transformer area to which the low-voltage user volume point and the power supply information belong, inconsistent information of the transformer area to which the medium-voltage user volume point and the power supply information power supply line are;
the target business rule corresponding to the collaborative data comprises: any one of the station line relation, the line-to-line relation and the household variable relation is inconsistent, and/or any one of a transformer substation, a feeder line, a transformer, a low-voltage user and a medium-voltage user is inconsistent.
5. The method of claim 1, wherein the storing and visually presenting the issue data comprises:
and counting the continuous historical days of each question data, and visually displaying the question data and the corresponding historical days.
6. The method of claim 1, wherein after invoking a target business rule from a business rule base and performing a consistency comparison on the first data, the second data, and the third data, the method further comprises:
and determining the data consistency rates of any two databases according to the comparison result, wherein the data consistency rates comprise an object consistency rate and a topological relation consistency rate.
7. The method according to claim 6, wherein the determining the data consistency ratio of any two databases according to the comparison result comprises:
determining the total number of comparison data volumes in any two databases and a first data volume of objects which are successfully compared continuously in the comparison result, and determining the object consistency rate of any two databases based on the first data volume and the total number of the comparison data volumes;
and determining a second data volume of which the topological relation comparison is successful in the data of which the object continuous comparison is successful, and determining the consistency rate of the topological relation of any two databases based on the second data volume and the first data volume.
8. A data processing apparatus, comprising:
the data acquisition module is used for acquiring first data in a marketing system database, second data in a GIS system database and third data in a production system database;
the problem data determining module is used for calling a target business rule from a business rule base, comparing the consistency of the first data, the second data and the third data and determining problem data according to a comparison result;
and the problem data storage and display module is used for storing and visually displaying the problem data.
9. A server, characterized in that the server comprises:
one or more processors;
a storage device for storing one or more programs,
when executed by the processor, cause the processor to implement the data processing method of any of claims 1-7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the data processing method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011313950.1A CN112463807A (en) | 2020-11-20 | 2020-11-20 | Data processing method, device, server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011313950.1A CN112463807A (en) | 2020-11-20 | 2020-11-20 | Data processing method, device, server and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112463807A true CN112463807A (en) | 2021-03-09 |
Family
ID=74799307
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011313950.1A Pending CN112463807A (en) | 2020-11-20 | 2020-11-20 | Data processing method, device, server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112463807A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113205431A (en) * | 2021-07-07 | 2021-08-03 | 广东电网有限责任公司佛山供电局 | Multi-source account data abnormity monitoring system and method for distribution network line |
| CN113282594A (en) * | 2021-07-26 | 2021-08-20 | 广东电网有限责任公司佛山供电局 | Automatic extraction and analysis method and system for basic data of distribution transformer |
| CN114971938A (en) * | 2022-06-07 | 2022-08-30 | 上海山源电子科技股份有限公司 | Method and device for detecting override-proof network topology of coal mine power supply system and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103514514A (en) * | 2013-09-23 | 2014-01-15 | 广州供电局有限公司 | On-line monitoring method for electricity marketing business data |
| CN105701626A (en) * | 2016-03-03 | 2016-06-22 | 国网浙江省电力公司 | Electric marketing inception lean control multi-system integrated method |
| CN108776856A (en) * | 2018-04-20 | 2018-11-09 | 国家电网有限公司 | A kind of electric power account data verification method and device based on the relationship of tracing to the source |
| CN109460432A (en) * | 2018-11-14 | 2019-03-12 | 腾讯科技（深圳）有限公司 | A kind of data processing method and system |
| CN110427387A (en) * | 2019-08-12 | 2019-11-08 | 中国工商银行股份有限公司 | A kind of data consistency detection and device |
-
2020
- 2020-11-20 CN CN202011313950.1A patent/CN112463807A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103514514A (en) * | 2013-09-23 | 2014-01-15 | 广州供电局有限公司 | On-line monitoring method for electricity marketing business data |
| CN105701626A (en) * | 2016-03-03 | 2016-06-22 | 国网浙江省电力公司 | Electric marketing inception lean control multi-system integrated method |
| CN108776856A (en) * | 2018-04-20 | 2018-11-09 | 国家电网有限公司 | A kind of electric power account data verification method and device based on the relationship of tracing to the source |
| CN109460432A (en) * | 2018-11-14 | 2019-03-12 | 腾讯科技（深圳）有限公司 | A kind of data processing method and system |
| CN110427387A (en) * | 2019-08-12 | 2019-11-08 | 中国工商银行股份有限公司 | A kind of data consistency detection and device |
Non-Patent Citations (1)
| Title |
|---|
| 邱林;李爱民;杨李达;: "营配调贯通数据质量分析方法及自动化实现的研究和应用", 电子技术与软件工程, no. 18, pages 173 - 175 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113205431A (en) * | 2021-07-07 | 2021-08-03 | 广东电网有限责任公司佛山供电局 | Multi-source account data abnormity monitoring system and method for distribution network line |
| CN113282594A (en) * | 2021-07-26 | 2021-08-20 | 广东电网有限责任公司佛山供电局 | Automatic extraction and analysis method and system for basic data of distribution transformer |
| CN113282594B (en) * | 2021-07-26 | 2021-12-10 | 广东电网有限责任公司佛山供电局 | Automatic extraction and analysis method and system for basic data of distribution transformer |
| CN114971938A (en) * | 2022-06-07 | 2022-08-30 | 上海山源电子科技股份有限公司 | Method and device for detecting override-proof network topology of coal mine power supply system and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112463807A (en) | Data processing method, device, server and storage medium | |
| CN101876938B (en) | Message queue-based application software response time measuring method | |
| CN109254901B (en) | A kind of Monitoring Indexes method and system | |
| CN112307003B (en) | Power grid data multidimensional auxiliary analysis method, system, terminal and readable storage medium | |
| CN108197774A (en) | A kind of abnormality diagnostic method and device of distributed photovoltaic power generation amount | |
| CN110647447A (en) | Abnormal instance detection method, apparatus, device and medium for distributed system | |
| CN115049232B (en) | Method and system for judging station area abnormity | |
| CN114662953A (en) | Internet of things equipment operation and maintenance method, device, equipment and medium | |
| CN111400189A (en) | Code coverage rate monitoring method and device, electronic equipment and storage medium | |
| CN110262955B (en) | Application performance monitoring tool based on pinpoint | |
| CN113077359B (en) | Line loss abnormity investigation method, device, equipment and storage medium | |
| CN112541011A (en) | Power grid terminal fault processing method and device based on RPA and AI | |
| CN111158827A (en) | Method and device for graphic configuration tool correlation calculation value information | |
| CN115471215B (en) | Business process processing method and device | |
| CN115292202A (en) | Product test analysis method and device, electronic equipment and storage medium | |
| CN115951172A (en) | Method, device and equipment for positioning metering point of abnormal unbalanced electric quantity of bus | |
| CN114757482A (en) | Method, system, device and medium for analyzing electric quantity use trend of user group | |
| CN110457705B (en) | Method, device, equipment and storage medium for processing point of interest data | |
| CN113656452A (en) | Method and device for detecting abnormal index of call chain, electronic equipment and storage medium | |
| CN113888360A (en) | Method, device, equipment and storage medium for identifying electricity stealing users | |
| CN112085344A (en) | Equipment health analysis method and device and electronic equipment | |
| CN112003369A (en) | Cloud platform based power grid automation equipment operation monitoring and statistical analysis method | |
| CN109460497B (en) | Service bus-based calculation model visual display method and system | |
| CN115495380A (en) | Test case generation method and device, electronic equipment and storage medium | |
| CN115409381A (en) | Line loss cause determination method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |