CN112231362A - Visual monitoring application system based on power grid main data service log - Google Patents
Visual monitoring application system based on power grid main data service log Download PDFInfo
- Publication number
- CN112231362A CN112231362A CN202011179523.9A CN202011179523A CN112231362A CN 112231362 A CN112231362 A CN 112231362A CN 202011179523 A CN202011179523 A CN 202011179523A CN 112231362 A CN112231362 A CN 112231362A
- Authority
- CN
- China
- Prior art keywords
- data
- main data
- log
- service
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000000007 visual effect Effects 0.000 title claims abstract description 24
- 238000012544 monitoring process Methods 0.000 title claims abstract description 23
- 230000002159 abnormal effect Effects 0.000 claims abstract description 24
- 238000004891 communication Methods 0.000 claims abstract description 4
- 239000000284 extract Substances 0.000 claims abstract description 4
- 238000004458 analytical method Methods 0.000 claims description 15
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000012423 maintenance Methods 0.000 claims description 4
- 239000013589 supplement Substances 0.000 claims description 3
- 238000007726 management method Methods 0.000 description 6
- 238000000034 method Methods 0.000 description 6
- 238000013523 data management Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012558 master data management Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2462—Approximate or statistical queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Probability & Statistics with Applications (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Fuzzy Systems (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a visual monitoring application system based on a main data service log of a power grid, which comprises a plurality of data nodes, a data center in communication connection with each node and monitoring software connected with the data center, wherein the data center comprises a plurality of data nodes; each data node automatically extracts and analyzes new log information to a data table c at regular time, and analyzes and counts the transaction total amount, abnormal amount and detailed amount of the data node according to a plurality of data tables; transmitting the total transaction amount, the abnormal amount and the detail amount as main data to the data center; and the monitoring software displays and queries the main data of each data node through a visual interface.
Description
Technical Field
The invention relates to the technical field of management systems, in particular to a visual monitoring application system based on a power grid main data service log.
Background
At present, a part of enterprises define a batch of most basic, most stable and most important data as main data of the enterprises according to self business requirements. The conditions of information isolated islands, inconsistent information versions and non-standard multi-source management exist in the production, popularization and use processes of main data. Therefore, the enterprise plans a master data management standard based on an SOA service-oriented form, and a data producer encapsulates the operation information, the data content body and the related management attributes of the master data into an xml file by calling a service interface form and forwards the xml file to a master data manager through bus service. And the main data management party uniformly manages the unique source of the main data, and then the main data is shared and distributed in a JMS queue mode according to the service requirements of each data consumer. The whole process can meet the management requirements of main data sharing, main data unique trusted sources, data version history backtracking and the like, but abnormity occurs in the actual system flow interaction process, so that the main data flow cannot be completely executed. In the circulation process lacking visual support, abnormity is difficult to find and troubleshoot, the popularization and use effect of main data is seriously influenced, the user experience is greatly reduced, and the progress of related business handling is blocked.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a visual monitoring application system which is novel and reliable in technology and can monitor the main data state of each node of the system in a visual mode and is based on a power grid main data service log.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows: a visual monitoring application system based on a power grid main data service log comprises a plurality of data nodes, a data center and monitoring software, wherein the data center is in communication connection with the nodes; each data node automatically extracts and analyzes new log information into a data table at regular time, and analyzes and counts the transaction total amount, abnormal amount and detailed amount of the data node according to a plurality of data tables; transmitting the total transaction amount, the abnormal amount and the detail amount as main data to the data center; and the monitoring software displays and queries the main data of each data node through a visual interface.
The monitoring software comprises a main data flow conversion monitoring module, a main data flow conversion abnormity analyzing and counting module and a main data rebroadcasting module.
The main data flow-to-monitoring module stores corresponding analyzed log information according to different system sources, monitoring timeliness and business flow modes of extracted data log files, simultaneously marks management attribute fields on the data tables and is used for identifying the flow state of a main data at different nodes, finally establishes an abnormal analysis model according to error reporting codes in logs, main data business key comparison among nodes and business flow mode timeliness comparison, and application can automatically check the analyzed log information stored in a warehouse according to the abnormal analysis model to realize automatic identification of abnormal work.
The main data flow is converted into an anomaly analysis and statistics module, classification and sorting functions of a knowledge base are provided, operation and maintenance personnel can supplement remarks such as service analysis explanation, influence range and the like to the current anomaly type according to basic error reporting codes on logs and by combining service practice, and after the anomaly is automatically identified, corresponding error reporting codes are output, and the remarks in the knowledge base are output together, so that general users can conveniently understand the cause and the influence range of the anomaly; the abnormity is counted mainly by a visual graph, three data items of data volume, abnormal volume and success rate are counted, and log detail report information and log original XML message information are provided; and the dimension is divided into a data log system source, time and a main data category for carrying out subdivision statistics. The main data rebroadcasting module can be used for self-service broadcasting when a main data consumer finds that the main data in the service system is not the latest version, the application executes the broadcast state rewriting operation on the main data node selected by the user according to the set code rule, and the main data manager system re-performs the data broadcasting operation after recognizing the broadcast state and distributes the data to the consumer service system node.
The invention has the beneficial effects that: the invention provides the monitoring of the main data flow transfer process in a node link visualization mode, is convenient for operation and maintenance personnel to visually and clearly check the information of the nodes where the main data service abnormity is located, the abnormal quantity of the main data, the total data, the success rate and the like, can promote the data management work aiming at different abnormity as early as possible, and prevents the occurrence of business problems. Meanwhile, the data log system source, time and main data category are used for performing visual subdivision statistics, identifying abnormal situation trend, comparing and analyzing and the like, so that main data management personnel can know the current main data service application situation conveniently, and a basis is provided for future main data management strategies. The system user can re-transmit the main data to the consumer system by one key in the system according to the service requirement of re-broadcasting the main data without repeatedly calling related services from the database layer through the technical script for repeated broadcasting, so that the working efficiency is improved, and the database operation risk and the operation difficulty are reduced.
Detailed Description
The claimed solution will now be described in further detail with reference to specific embodiments.
In this embodiment, a visual monitoring application system based on a power grid master data service log includes a plurality of data nodes, a data center in communication connection with each node, and monitoring software connected to the data center.
Each data node automatically extracts and analyzes new log information into a data table at regular time, and analyzes and counts the transaction total amount, abnormal amount and detailed amount of the data node according to a plurality of data tables; and transmitting the total transaction amount, the abnormal amount and the detail amount as main data to the data center.
In this embodiment, the main data interactive stream is converted into an XML log file for service analysis, and relevant valuable log information is analyzed and extracted, and finally converted into a normalized data table. Based on the real-time requirement of the service, new log information is automatically extracted and analyzed into the data tables every 5 minutes, and then subsequent display and analysis work is carried out according to a plurality of data tables.
In this embodiment, the interface of the monitoring software represents the main data state of each data node by color, where red indicates that the node has more abnormal data, yellow indicates that the node has abnormal data, but the overall abnormal rate is below 60%, blue indicates that the overall abnormal rate is below 10% or there is no abnormal node, and gray nodes are nodes that cannot be monitored at present; clicking on the node may enter a details page to view details of the data. In addition, the transaction amount and the success rate can be automatically counted by taking the provider system as a unit according to the process circulation condition of the main data through a pie chart of visual statistics, so that a manager can quickly master the transaction condition of the main data, and a similar visual statistical chart can be used according to time, the type of the main data object and the like. The monitoring software displays and inquires the main data of each data node through a visual interface, wherein the monitoring software comprises a main data flow-to-flow monitoring module, a main data flow-to-flow condition analysis and statistics module and a main data rebroadcasting module.
In this embodiment, the main data flow monitoring module stores the corresponding analyzed log information in different data tables according to different system sources, monitoring timeliness and service flow modes of the extracted data log files, and marks a management attribute field on the data tables for identifying the flow state of a main data at different nodes, and finally establishes an anomaly analysis model according to an error reporting code in the log, a main key comparison of main data services between nodes and a timeliness comparison of service flow modes, so that the application can automatically check the analyzed log information in a storage according to the anomaly analysis model, and realize automatic identification of anomaly work.
The main data flow is converted into an anomaly analysis and statistics module, classification and sorting functions of a knowledge base are provided, operation and maintenance personnel can supplement remarks such as service analysis explanation, influence range and the like to the current anomaly type according to basic error reporting codes on logs and by combining service practice, and after the anomaly is automatically identified, corresponding error reporting codes are output, and the remarks in the knowledge base are output together, so that general users can conveniently understand the cause and the influence range of the anomaly; the abnormity is counted mainly by a visual graph, three data items of data volume, abnormal volume and success rate are counted, and log detail report information and log original XML message information are provided; and the dimension is divided into a data log system source, time and a main data category for carrying out subdivision statistics. The main data rebroadcasting module can be used for self-service broadcasting when a main data consumer finds that the main data in the service system is not the latest version, the application executes the broadcast state rewriting operation on the main data node selected by the user according to the set code rule, and the main data manager system re-performs the data broadcasting operation after recognizing the broadcast state and distributes the data to the consumer service system node. The above-described embodiments are merely preferred embodiments of the present invention, which is not intended to limit the present invention in any way. Those skilled in the art can make many changes and modifications to the disclosed embodiments, or modify equivalent embodiments to practice the disclosed embodiments, without departing from the scope of the disclosed embodiments. Therefore, equivalent variations made according to the idea of the present invention should be covered within the protection scope of the present invention without departing from the contents of the technical solution of the present invention.
Claims (5)
1. A visual monitoring application system based on a power grid main data service log is characterized in that: the monitoring system comprises a plurality of data nodes, a data center in communication connection with each node and monitoring software connected with the data center; each data node automatically extracts and analyzes new log information into a data table at regular time, and analyzes and counts the transaction total amount, abnormal amount and detailed amount of the data node according to a plurality of data tables; transmitting the total transaction amount, the abnormal amount and the detail amount as main data to the data center; and the monitoring software displays and queries the main data of each data node through a visual interface.
2. The visual monitoring application system based on the power grid main data service log as claimed in claim 1, wherein: the monitoring software comprises a main data flow conversion monitoring module, a main data flow conversion abnormity analyzing and counting module and a main data rebroadcasting module.
3. The visual monitoring application system based on the power grid main data service log as claimed in claim 2, wherein: the main data flow-to-monitoring module stores corresponding analyzed log information according to different system sources, monitoring timeliness and business flow modes of extracted data log files, simultaneously marks management attribute fields on the data tables and is used for identifying the flow state of a main data at different nodes, finally establishes an abnormal analysis model according to error reporting codes in logs, main data business key comparison among nodes and business flow mode timeliness comparison, and application can automatically check the analyzed log information stored in a warehouse according to the abnormal analysis model to realize automatic identification of abnormal work.
4. The visual monitoring application system based on the power grid main data service log as claimed in claim 2, wherein: the main data flow is converted into an anomaly analysis and statistics module to provide classification and sorting functions of a knowledge base, operation and maintenance personnel can supplement remarks such as service analysis explanation, influence range and the like to the current anomaly type according to basic error reporting codes on logs and by combining service practice, and after the anomaly is automatically identified, corresponding error reporting codes are output, and the remarks in the knowledge base are output together, so that general users can conveniently understand the cause and the influence range of the anomaly; the abnormity is counted mainly by a visual graph, three data items of data volume, abnormal volume and success rate are counted, and log detail report information and log original XML message information are provided; and the dimension is divided into a data log system source, time and a main data category for carrying out subdivision statistics.
5. The visual monitoring application system based on the power grid main data service log as claimed in claim 2, wherein: the main data rebroadcasting module can be used for self-service broadcasting when a main data consumer finds that the main data in the service system is not the latest version, the application executes the broadcast state rewriting operation on the main data node selected by the user according to the set code rule, and the main data manager system re-performs the data broadcasting operation after recognizing the broadcast state and distributes the data to the consumer service system node.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011179523.9A CN112231362A (en) | 2020-10-29 | 2020-10-29 | Visual monitoring application system based on power grid main data service log |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011179523.9A CN112231362A (en) | 2020-10-29 | 2020-10-29 | Visual monitoring application system based on power grid main data service log |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112231362A true CN112231362A (en) | 2021-01-15 |
Family
ID=74109781
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011179523.9A Pending CN112231362A (en) | 2020-10-29 | 2020-10-29 | Visual monitoring application system based on power grid main data service log |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112231362A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113220537A (en) * | 2021-06-02 | 2021-08-06 | 上海节卡机器人科技有限公司 | Software monitoring method, device, equipment and readable storage medium |
| CN115242349A (en) * | 2022-06-21 | 2022-10-25 | 苏州盈数智能科技有限公司 | Enterprise-level data verification method and device, computer equipment and storage medium |
| WO2023093386A1 (en) * | 2021-11-29 | 2023-06-01 | 腾讯科技（深圳）有限公司 | Data detection method and apparatus, electronic device, computer storage medium, and computer program product |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2247498A1 (en) * | 1996-02-29 | 1997-09-04 | Intermind Corporation | An automated communications system and method for transferring informations between databases in order to control and process communications |
| CN106469419A (en) * | 2016-09-09 | 2017-03-01 | 国家电网公司 | The processing method and processing device of auxiliary tone insertion data is sought in a kind of power system |
| US20190005744A1 (en) * | 2013-04-22 | 2019-01-03 | Latitude Technologies Corporation | Aircraft flight data monitoring and reporting system and use thereof |
| CN110704280A (en) * | 2019-10-10 | 2020-01-17 | 南方电网数字电网研究院有限公司 | Main data monitoring system |
-
2020
- 2020-10-29 CN CN202011179523.9A patent/CN112231362A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2247498A1 (en) * | 1996-02-29 | 1997-09-04 | Intermind Corporation | An automated communications system and method for transferring informations between databases in order to control and process communications |
| US20190005744A1 (en) * | 2013-04-22 | 2019-01-03 | Latitude Technologies Corporation | Aircraft flight data monitoring and reporting system and use thereof |
| CN106469419A (en) * | 2016-09-09 | 2017-03-01 | 国家电网公司 | The processing method and processing device of auxiliary tone insertion data is sought in a kind of power system |
| CN110704280A (en) * | 2019-10-10 | 2020-01-17 | 南方电网数字电网研究院有限公司 | Main data monitoring system |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113220537A (en) * | 2021-06-02 | 2021-08-06 | 上海节卡机器人科技有限公司 | Software monitoring method, device, equipment and readable storage medium |
| WO2023093386A1 (en) * | 2021-11-29 | 2023-06-01 | 腾讯科技（深圳）有限公司 | Data detection method and apparatus, electronic device, computer storage medium, and computer program product |
| CN115242349A (en) * | 2022-06-21 | 2022-10-25 | 苏州盈数智能科技有限公司 | Enterprise-level data verification method and device, computer equipment and storage medium |
| CN115242349B (en) * | 2022-06-21 | 2023-11-14 | 苏州盈数智能科技有限公司 | Enterprise-level data verification method, enterprise-level data verification device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112231362A (en) | Visual monitoring application system based on power grid main data service log | |
| CN112488558A (en) | Energy consumption monitoring and analyzing system based on industrial internet | |
| CN102347983A (en) | ESB (Enterprise Service Bus) system in SOA (Service-Oriented Architecture) | |
| CN112396397A (en) | Operation and maintenance affair collection and management system | |
| CN112508486A (en) | Secondary spare part inventory management system for power system enterprise | |
| CN111160789A (en) | Intelligent whole-person safety production responsibility management system | |
| CN111311132A (en) | Operation and maintenance service quality data analysis decision support system | |
| CN109858807B (en) | Enterprise operation monitoring method and system | |
| CN111768177B (en) | County-level power supply company operation analysis system and county-level power supply company operation analysis method | |
| CN115016902B (en) | Industrial flow digital management system and method | |
| CN111061716A (en) | Distributed data quality monitoring system and method based on heterogeneous data source | |
| CN112085412B (en) | Resource optimization distribution system and distribution method | |
| CN114428812A (en) | Report data monitoring method, device, equipment and storage medium | |
| CN114596162A (en) | Intelligent carbon transaction service management system and method | |
| CN112702196A (en) | Automatic fault processing method and system | |
| CN113362017A (en) | Method for instantly sharing and analyzing standardized operation management information | |
| LU505378B1 (en) | Safety management system for hydraulic engineering construction and emergency command method thereof | |
| CN213634560U (en) | Intelligent management system for logistics transportation data information | |
| Kang et al. | Research on Business Center Monitoring System Based on Business Emergency System of Power Grid Company | |
| CN110990457A (en) | Production report management system | |
| CN110908977B (en) | Steel enterprise system fault statistical analysis method based on two-in-one management platform | |
| CN116846786A (en) | Sentinel system for monitoring equipment abnormality and equipment abnormality monitoring method | |
| CN117675790A (en) | Method for realizing cross-platform file distribution | |
| CN115511654A (en) | Management system for intelligent manufacturing workshop | |
| CN112256469A (en) | Intelligent problem positioning method based on cross-system collaborative scene |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210115 |
|
| RJ01 | Rejection of invention patent application after publication |