CN112231362A - Visual monitoring application system based on power grid main data service log - Google Patents

Visual monitoring application system based on power grid main data service log Download PDF

Info

Publication number
CN112231362A
CN112231362A CN202011179523.9A CN202011179523A CN112231362A CN 112231362 A CN112231362 A CN 112231362A CN 202011179523 A CN202011179523 A CN 202011179523A CN 112231362 A CN112231362 A CN 112231362A
Authority
CN
China
Prior art keywords
data
main data
log
service
node
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011179523.9A
Other languages
Chinese (zh)
Inventor
廖嘉炜
宋强
陆宏治
徐炫东
邹时容
池燕清
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Power Supply Bureau of Guangdong Power Grid Co Ltd
Original Assignee
Guangzhou Power Supply Bureau of Guangdong Power Grid Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Power Supply Bureau of Guangdong Power Grid Co Ltd filed Critical Guangzhou Power Supply Bureau of Guangdong Power Grid Co Ltd
Priority to CN202011179523.9A priority Critical patent/CN112231362A/en
Publication of CN112231362A publication Critical patent/CN112231362A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2458Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • G06F16/2462Approximate or statistical queries
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/248Presentation of query results
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25Integrating or interfacing systems involving database management systems
    • G06F16/254Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Fuzzy Systems (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The invention discloses a visual monitoring application system based on a main data service log of a power grid, which comprises a plurality of data nodes, a data center in communication connection with each node and monitoring software connected with the data center, wherein the data center comprises a plurality of data nodes; each data node automatically extracts and analyzes new log information to a data table c at regular time, and analyzes and counts the transaction total amount, abnormal amount and detailed amount of the data node according to a plurality of data tables; transmitting the total transaction amount, the abnormal amount and the detail amount as main data to the data center; and the monitoring software displays and queries the main data of each data node through a visual interface.

Description

Visual monitoring application system based on power grid main data service log
Technical Field
The invention relates to the technical field of management systems, in particular to a visual monitoring application system based on a power grid main data service log.
Background
At present, a part of enterprises define a batch of most basic, most stable and most important data as main data of the enterprises according to self business requirements. The conditions of information isolated islands, inconsistent information versions and non-standard multi-source management exist in the production, popularization and use processes of main data. Therefore, the enterprise plans a master data management standard based on an SOA service-oriented form, and a data producer encapsulates the operation information, the data content body and the related management attributes of the master data into an xml file by calling a service interface form and forwards the xml file to a master data manager through bus service. And the main data management party uniformly manages the unique source of the main data, and then the main data is shared and distributed in a JMS queue mode according to the service requirements of each data consumer. The whole process can meet the management requirements of main data sharing, main data unique trusted sources, data version history backtracking and the like, but abnormity occurs in the actual system flow interaction process, so that the main data flow cannot be completely executed. In the circulation process lacking visual support, abnormity is difficult to find and troubleshoot, the popularization and use effect of main data is seriously influenced, the user experience is greatly reduced, and the progress of related business handling is blocked.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a visual monitoring application system which is novel and reliable in technology and can monitor the main data state of each node of the system in a visual mode and is based on a power grid main data service log.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows: a visual monitoring application system based on a power grid main data service log comprises a plurality of data nodes, a data center and monitoring software, wherein the data center is in communication connection with the nodes; each data node automatically extracts and analyzes new log information into a data table at regular time, and analyzes and counts the transaction total amount, abnormal amount and detailed amount of the data node according to a plurality of data tables; transmitting the total transaction amount, the abnormal amount and the detail amount as main data to the data center; and the monitoring software displays and queries the main data of each data node through a visual interface.
The monitoring software comprises a main data flow conversion monitoring module, a main data flow conversion abnormity analyzing and counting module and a main data rebroadcasting module.
The main data flow-to-monitoring module stores corresponding analyzed log information according to different system sources, monitoring timeliness and business flow modes of extracted data log files, simultaneously marks management attribute fields on the data tables and is used for identifying the flow state of a main data at different nodes, finally establishes an abnormal analysis model according to error reporting codes in logs, main data business key comparison among nodes and business flow mode timeliness comparison, and application can automatically check the analyzed log information stored in a warehouse according to the abnormal analysis model to realize automatic identification of abnormal work.
The main data flow is converted into an anomaly analysis and statistics module, classification and sorting functions of a knowledge base are provided, operation and maintenance personnel can supplement remarks such as service analysis explanation, influence range and the like to the current anomaly type according to basic error reporting codes on logs and by combining service practice, and after the anomaly is automatically identified, corresponding error reporting codes are output, and the remarks in the knowledge base are output together, so that general users can conveniently understand the cause and the influence range of the anomaly; the abnormity is counted mainly by a visual graph, three data items of data volume, abnormal volume and success rate are counted, and log detail report information and log original XML message information are provided; and the dimension is divided into a data log system source, time and a main data category for carrying out subdivision statistics. The main data rebroadcasting module can be used for self-service broadcasting when a main data consumer finds that the main data in the service system is not the latest version, the application executes the broadcast state rewriting operation on the main data node selected by the user according to the set code rule, and the main data manager system re-performs the data broadcasting operation after recognizing the broadcast state and distributes the data to the consumer service system node.
The invention has the beneficial effects that: the invention provides the monitoring of the main data flow transfer process in a node link visualization mode, is convenient for operation and maintenance personnel to visually and clearly check the information of the nodes where the main data service abnormity is located, the abnormal quantity of the main data, the total data, the success rate and the like, can promote the data management work aiming at different abnormity as early as possible, and prevents the occurrence of business problems. Meanwhile, the data log system source, time and main data category are used for performing visual subdivision statistics, identifying abnormal situation trend, comparing and analyzing and the like, so that main data management personnel can know the current main data service application situation conveniently, and a basis is provided for future main data management strategies. The system user can re-transmit the main data to the consumer system by one key in the system according to the service requirement of re-broadcasting the main data without repeatedly calling related services from the database layer through the technical script for repeated broadcasting, so that the working efficiency is improved, and the database operation risk and the operation difficulty are reduced.
Detailed Description
The claimed solution will now be described in further detail with reference to specific embodiments.
In this embodiment, a visual monitoring application system based on a power grid master data service log includes a plurality of data nodes, a data center in communication connection with each node, and monitoring software connected to the data center.
Each data node automatically extracts and analyzes new log information into a data table at regular time, and analyzes and counts the transaction total amount, abnormal amount and detailed amount of the data node according to a plurality of data tables; and transmitting the total transaction amount, the abnormal amount and the detail amount as main data to the data center.
In this embodiment, the main data interactive stream is converted into an XML log file for service analysis, and relevant valuable log information is analyzed and extracted, and finally converted into a normalized data table. Based on the real-time requirement of the service, new log information is automatically extracted and analyzed into the data tables every 5 minutes, and then subsequent display and analysis work is carried out according to a plurality of data tables.
In this embodiment, the interface of the monitoring software represents the main data state of each data node by color, where red indicates that the node has more abnormal data, yellow indicates that the node has abnormal data, but the overall abnormal rate is below 60%, blue indicates that the overall abnormal rate is below 10% or there is no abnormal node, and gray nodes are nodes that cannot be monitored at present; clicking on the node may enter a details page to view details of the data. In addition, the transaction amount and the success rate can be automatically counted by taking the provider system as a unit according to the process circulation condition of the main data through a pie chart of visual statistics, so that a manager can quickly master the transaction condition of the main data, and a similar visual statistical chart can be used according to time, the type of the main data object and the like. The monitoring software displays and inquires the main data of each data node through a visual interface, wherein the monitoring software comprises a main data flow-to-flow monitoring module, a main data flow-to-flow condition analysis and statistics module and a main data rebroadcasting module.
In this embodiment, the main data flow monitoring module stores the corresponding analyzed log information in different data tables according to different system sources, monitoring timeliness and service flow modes of the extracted data log files, and marks a management attribute field on the data tables for identifying the flow state of a main data at different nodes, and finally establishes an anomaly analysis model according to an error reporting code in the log, a main key comparison of main data services between nodes and a timeliness comparison of service flow modes, so that the application can automatically check the analyzed log information in a storage according to the anomaly analysis model, and realize automatic identification of anomaly work.
The main data flow is converted into an anomaly analysis and statistics module, classification and sorting functions of a knowledge base are provided, operation and maintenance personnel can supplement remarks such as service analysis explanation, influence range and the like to the current anomaly type according to basic error reporting codes on logs and by combining service practice, and after the anomaly is automatically identified, corresponding error reporting codes are output, and the remarks in the knowledge base are output together, so that general users can conveniently understand the cause and the influence range of the anomaly; the abnormity is counted mainly by a visual graph, three data items of data volume, abnormal volume and success rate are counted, and log detail report information and log original XML message information are provided; and the dimension is divided into a data log system source, time and a main data category for carrying out subdivision statistics. The main data rebroadcasting module can be used for self-service broadcasting when a main data consumer finds that the main data in the service system is not the latest version, the application executes the broadcast state rewriting operation on the main data node selected by the user according to the set code rule, and the main data manager system re-performs the data broadcasting operation after recognizing the broadcast state and distributes the data to the consumer service system node. The above-described embodiments are merely preferred embodiments of the present invention, which is not intended to limit the present invention in any way. Those skilled in the art can make many changes and modifications to the disclosed embodiments, or modify equivalent embodiments to practice the disclosed embodiments, without departing from the scope of the disclosed embodiments. Therefore, equivalent variations made according to the idea of the present invention should be covered within the protection scope of the present invention without departing from the contents of the technical solution of the present invention.

Claims (5)

1. A visual monitoring application system based on a power grid main data service log is characterized in that: the monitoring system comprises a plurality of data nodes, a data center in communication connection with each node and monitoring software connected with the data center; each data node automatically extracts and analyzes new log information into a data table at regular time, and analyzes and counts the transaction total amount, abnormal amount and detailed amount of the data node according to a plurality of data tables; transmitting the total transaction amount, the abnormal amount and the detail amount as main data to the data center; and the monitoring software displays and queries the main data of each data node through a visual interface.
2. The visual monitoring application system based on the power grid main data service log as claimed in claim 1, wherein: the monitoring software comprises a main data flow conversion monitoring module, a main data flow conversion abnormity analyzing and counting module and a main data rebroadcasting module.
3. The visual monitoring application system based on the power grid main data service log as claimed in claim 2, wherein: the main data flow-to-monitoring module stores corresponding analyzed log information according to different system sources, monitoring timeliness and business flow modes of extracted data log files, simultaneously marks management attribute fields on the data tables and is used for identifying the flow state of a main data at different nodes, finally establishes an abnormal analysis model according to error reporting codes in logs, main data business key comparison among nodes and business flow mode timeliness comparison, and application can automatically check the analyzed log information stored in a warehouse according to the abnormal analysis model to realize automatic identification of abnormal work.
4. The visual monitoring application system based on the power grid main data service log as claimed in claim 2, wherein: the main data flow is converted into an anomaly analysis and statistics module to provide classification and sorting functions of a knowledge base, operation and maintenance personnel can supplement remarks such as service analysis explanation, influence range and the like to the current anomaly type according to basic error reporting codes on logs and by combining service practice, and after the anomaly is automatically identified, corresponding error reporting codes are output, and the remarks in the knowledge base are output together, so that general users can conveniently understand the cause and the influence range of the anomaly; the abnormity is counted mainly by a visual graph, three data items of data volume, abnormal volume and success rate are counted, and log detail report information and log original XML message information are provided; and the dimension is divided into a data log system source, time and a main data category for carrying out subdivision statistics.
5. The visual monitoring application system based on the power grid main data service log as claimed in claim 2, wherein: the main data rebroadcasting module can be used for self-service broadcasting when a main data consumer finds that the main data in the service system is not the latest version, the application executes the broadcast state rewriting operation on the main data node selected by the user according to the set code rule, and the main data manager system re-performs the data broadcasting operation after recognizing the broadcast state and distributes the data to the consumer service system node.
CN202011179523.9A 2020-10-29 2020-10-29 Visual monitoring application system based on power grid main data service log Pending CN112231362A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011179523.9A CN112231362A (en) 2020-10-29 2020-10-29 Visual monitoring application system based on power grid main data service log

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011179523.9A CN112231362A (en) 2020-10-29 2020-10-29 Visual monitoring application system based on power grid main data service log

Publications (1)

Publication Number Publication Date
CN112231362A true CN112231362A (en) 2021-01-15

Family

ID=74109781

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011179523.9A Pending CN112231362A (en) 2020-10-29 2020-10-29 Visual monitoring application system based on power grid main data service log

Country Status (1)

Country Link
CN (1) CN112231362A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113220537A (en) * 2021-06-02 2021-08-06 上海节卡机器人科技有限公司 Software monitoring method, device, equipment and readable storage medium
CN115242349A (en) * 2022-06-21 2022-10-25 苏州盈数智能科技有限公司 Enterprise-level data verification method and device, computer equipment and storage medium
WO2023093386A1 (en) * 2021-11-29 2023-06-01 腾讯科技(深圳)有限公司 Data detection method and apparatus, electronic device, computer storage medium, and computer program product

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA2247498A1 (en) * 1996-02-29 1997-09-04 Intermind Corporation An automated communications system and method for transferring informations between databases in order to control and process communications
CN106469419A (en) * 2016-09-09 2017-03-01 国家电网公司 The processing method and processing device of auxiliary tone insertion data is sought in a kind of power system
US20190005744A1 (en) * 2013-04-22 2019-01-03 Latitude Technologies Corporation Aircraft flight data monitoring and reporting system and use thereof
CN110704280A (en) * 2019-10-10 2020-01-17 南方电网数字电网研究院有限公司 Main data monitoring system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA2247498A1 (en) * 1996-02-29 1997-09-04 Intermind Corporation An automated communications system and method for transferring informations between databases in order to control and process communications
US20190005744A1 (en) * 2013-04-22 2019-01-03 Latitude Technologies Corporation Aircraft flight data monitoring and reporting system and use thereof
CN106469419A (en) * 2016-09-09 2017-03-01 国家电网公司 The processing method and processing device of auxiliary tone insertion data is sought in a kind of power system
CN110704280A (en) * 2019-10-10 2020-01-17 南方电网数字电网研究院有限公司 Main data monitoring system

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113220537A (en) * 2021-06-02 2021-08-06 上海节卡机器人科技有限公司 Software monitoring method, device, equipment and readable storage medium
WO2023093386A1 (en) * 2021-11-29 2023-06-01 腾讯科技(深圳)有限公司 Data detection method and apparatus, electronic device, computer storage medium, and computer program product
CN115242349A (en) * 2022-06-21 2022-10-25 苏州盈数智能科技有限公司 Enterprise-level data verification method and device, computer equipment and storage medium
CN115242349B (en) * 2022-06-21 2023-11-14 苏州盈数智能科技有限公司 Enterprise-level data verification method, enterprise-level data verification device, computer equipment and storage medium

Similar Documents

Publication Publication Date Title
CN112231362A (en) Visual monitoring application system based on power grid main data service log
CN112488558A (en) Energy consumption monitoring and analyzing system based on industrial internet
CN102347983A (en) ESB (Enterprise Service Bus) system in SOA (Service-Oriented Architecture)
CN112396397A (en) Operation and maintenance affair collection and management system
CN112508486A (en) Secondary spare part inventory management system for power system enterprise
CN111160789A (en) Intelligent whole-person safety production responsibility management system
CN111311132A (en) Operation and maintenance service quality data analysis decision support system
CN109858807B (en) Enterprise operation monitoring method and system
CN113469385A (en) Visual monitoring method and system for equipment state
CN111768177B (en) County-level power supply company operation analysis system and county-level power supply company operation analysis method
CN115016902B (en) Industrial flow digital management system and method
CN112085412B (en) Resource optimization distribution system and distribution method
CN114428812A (en) Report data monitoring method, device, equipment and storage medium
CN115185710A (en) Transaction interface time consumption counting and early warning method
CN114596162A (en) Intelligent carbon transaction service management system and method
CN112702196A (en) Automatic fault processing method and system
CN113362017A (en) Method for instantly sharing and analyzing standardized operation management information
LU505378B1 (en) Safety management system for hydraulic engineering construction and emergency command method thereof
CN213634560U (en) Intelligent management system for logistics transportation data information
Kang et al. Research on Business Center Monitoring System Based on Business Emergency System of Power Grid Company
CN110990457A (en) Production report management system
CN110908977B (en) Steel enterprise system fault statistical analysis method based on two-in-one management platform
CN116846786A (en) Sentinel system for monitoring equipment abnormality and equipment abnormality monitoring method
CN117675790A (en) Method for realizing cross-platform file distribution
CN115511654A (en) Management system for intelligent manufacturing workshop

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20210115

RJ01 Rejection of invention patent application after publication