CN113505048A - Unified monitoring platform based on application system portrait and implementation method - Google Patents
Unified monitoring platform based on application system portrait and implementation method Download PDFInfo
- Publication number
- CN113505048A CN113505048A CN202110778488.0A CN202110778488A CN113505048A CN 113505048 A CN113505048 A CN 113505048A CN 202110778488 A CN202110778488 A CN 202110778488A CN 113505048 A CN113505048 A CN 113505048A
- Authority
- CN
- China
- Prior art keywords
- data
- application system
- monitoring
- monitoring platform
- service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 90
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000012545 processing Methods 0.000 claims abstract description 54
- 238000004458 analytical method Methods 0.000 claims abstract description 25
- 238000012423 maintenance Methods 0.000 claims abstract description 24
- 238000003860 storage Methods 0.000 claims abstract description 19
- 238000013075 data extraction Methods 0.000 claims abstract description 9
- 238000001914 filtration Methods 0.000 claims abstract description 4
- 238000012216 screening Methods 0.000 claims abstract description 4
- 238000002372 labelling Methods 0.000 claims description 32
- 238000013079 data visualisation Methods 0.000 claims description 15
- 230000002776 aggregation Effects 0.000 claims description 10
- 238000004220 aggregation Methods 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 9
- 230000000007 visual effect Effects 0.000 claims description 6
- 238000011161 development Methods 0.000 claims description 4
- 230000002452 interceptive effect Effects 0.000 claims description 4
- 238000012800 visualization Methods 0.000 claims description 4
- 238000004140 cleaning Methods 0.000 claims description 3
- 230000006835 compression Effects 0.000 claims description 3
- 238000007906 compression Methods 0.000 claims description 3
- 238000005553 drilling Methods 0.000 claims description 3
- 238000002156 mixing Methods 0.000 claims description 3
- 238000011144 upstream manufacturing Methods 0.000 claims 1
- 238000010276 construction Methods 0.000 abstract description 5
- 238000007726 management method Methods 0.000 description 15
- 230000006870 function Effects 0.000 description 10
- 230000002159 abnormal effect Effects 0.000 description 4
- 238000013024 troubleshooting Methods 0.000 description 3
- 238000007621 cluster analysis Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000012098 association analyses Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 238000007711 solidification Methods 0.000 description 1
- 230000008023 solidification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/32—Monitoring with visual or acoustical indication of the functioning of the machine
- G06F11/323—Visualisation of programs or trace data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
Abstract
The invention discloses a unified monitoring platform based on application system portrait and an implementation method, belonging to the field of monitoring management of a plurality of cloud platforms, aiming at solving the technical problems that the monitoring system accumulated due to repeated construction of historical reasons is dispersed, the monitoring data is of various types, the format is not unified, the unified view display is lacked, and operation and maintenance personnel cannot be assisted to carry out fault analysis, and the technical scheme is as follows: the monitoring platform comprises a data extraction processing module, a data processing module and a data processing module, wherein the data extraction processing module is used for directly acquiring data from an original monitoring system or a service system and processing the data; the application system portrait module is used for system portrait, and the system portrait processing data is displayed on a Web end of the monitoring platform according to the service requirement; the data acquisition module is used for covering monitoring data of manufacturers of the original cloud platforms, and newly acquiring data which is not contained in the original cloud platforms according to the standard; and the data standardization processing module is used for realizing the access, analysis, screening, filtering and storage of the log data and the index data.
Description
Technical Field
The invention relates to the field of monitoring management of a plurality of cloud platforms, in particular to a unified monitoring platform based on application system portrayal and an implementation method.
Background
Government affair cloud construction comprises external public clouds and private clouds used in internal office work and the like, in the construction process of the government affair cloud, a plurality of cloud management platforms are provided by purchased cloud service providers, corresponding resource management and monitoring platforms are provided by the cloud service provider construction, and the cloud management and monitoring platforms have general basic monitoring, such as hardware, network and storage resource monitoring, and special monitoring, such as business system monitoring, database monitoring, various middleware monitoring and the like. Each system has an independent management mode and a display mode, so that when a fault occurs, a unified view is lacked to summarize various data, and operation and maintenance personnel cannot be effectively assisted to perform fault analysis. The operation and maintenance personnel cannot intuitively judge the association relation between the abnormal indexes, and need to check each system repeatedly, so that data integration and analysis and judgment are performed artificially, and the efficiency is low.
Disclosure of Invention
The invention provides a unified monitoring platform based on application system portrait and an implementation method thereof, and aims to solve the problems that monitoring systems accumulated due to repeated construction of historical reasons are dispersed, monitoring data are various, formats are not unified, unified view display is lacked, and operation and maintenance personnel cannot be effectively assisted in fault analysis.
The technical task of the invention is achieved in that a unified monitoring platform based on application system portrayal comprises,
the data extraction processing module is used for directly acquiring data from the original monitoring system or the original business system and processing the data;
the application system portrait module is used for system portrait, and the system portrait processing data is displayed on a Web end on the monitoring platform according to the service requirement and is sent to the alarm work order system according to the service definition;
the data acquisition module is used for covering monitoring data of manufacturers of the original cloud platforms, and newly acquiring data which is not contained in the original cloud platforms according to the standard;
and the data standardization processing module is used for realizing the access, analysis, screening, filtering and storage of the log data and the index data.
Preferably, the working process of the data extraction processing module is as follows:
(1) extracting data from the original monitoring system through a data acquisition interface;
(2) carrying out data acquisition formatting processing and classified storage on the extracted data, and carrying out data cleaning, de-duplication and compression processing;
(3) signing various data;
(4) sending the processed data to an application system image module for processing;
(5) analyzing and summarizing the data according to the service dimension and the time dimension;
(6) and sending the processed data to an alarm work order system for alarm judgment and analysis, and simultaneously displaying on a unified monitoring platform page according to monitoring service requirements.
Preferably, the application system image module performs classification and labeling processing by using the collected information, specifically as follows:
and associating the data by combining the asset data and the service calling relation data of the CMDB: the method comprises the following specific steps:
acquiring a multilevel topological relation of a service system by acquiring asset configuration data and service relation data of the CMDB;
associating key indexes of different objects to specific nodes of a service topology;
various key index data on the service topological graph are finally gathered through the real-time aggregation calculation of the stream processing framework, so that the functions of report configurability and monitoring scene customization are realized, and the dependence of monitoring scene change on system research personnel is eliminated;
labeling treatment is carried out after modeling: labeling various hardware and software according to physical attribute, configuration attribute, operation data attribute, operation and maintenance data attribute and intelligent attribute; the hardware comprises a network, a cloud server and a cloud service physical host; the software includes an operating system, a database, and middleware.
Preferably, the labeling treatment comprises manual labeling and machine labeling;
the manual tagging is to set a tag dictionary according to operation and maintenance experience and specifications, and is formed by querying a data association dictionary when data are collected, for example, equipment manufacturer information is a physical and non-modifiable attribute, and an ip address of a cloud server is a configuration attribute;
the machine labeling is to perform labeling through a performance data and application access association relation, for example, extracting keywords and performing cluster analysis on equipment (software) according to log information, performance data and fault frequency, and then labeling, for example, stability and access frequency; machine labeling was corrected by hand.
Preferably, the labeled data is sorted according to label, number and value, and is aggregated and normalized according to the analysis scene to form ordered and standard data.
Preferably, the Web end utilizes a basic form, a report and an Echart component to display data in a customized scene, and provides a visual data chart which is intuitive, vivid, interactive and highly personalized;
the data visualization chart supports mixing and matching between graphs, and relevant data are deposited into analysis reports of various dimensions; the data visualization chart comprises a line chart, a bar chart, a scatter chart, a pie chart, a k-line chart, a box chart for statistics, a map for geographical data visualization, a thermodynamic chart and a line chart, a relational chart for data relation visualization, a treemap, parallel coordinates for multi-dimensional data visualization, a funnel chart for BI and a dashboard.
Preferably, the information acquired by the data acquisition module comprises basic command information, platform information and middleware information; the basic command information comprises curl, ping and snmp; the platform information comprises openstack, vmware and ceph; the middleware information includes oracle, mysql, and Nginx.
A realization method of a unified monitoring platform based on application system portrait comprises the following steps:
s1, defining a standard interface, realizing data standardization of the existing cloud platform, and showing standardization; the method specifically comprises the following steps: unified monitoring management of a plurality of platforms in the government affair cloud is realized by defining a unified interface and acquiring data; the method comprises the steps of comprehensively monitoring and managing IT resources such as load balancing equipment, network equipment, a server, storage equipment, safety equipment, a database, middleware, application software and the like;
s2, according to the customized standard, butt-jointing the data lake product of the wave and the data of the existing cloud platform, and collecting the historical data of the cloud platform needing to be supervised, wherein the collected data comprises platform basic information and software hardware performance data; data are distributed to an application system portrait module and an alarm work order system for processing;
s3, deploying an application system portrait module, and performing aggregation calculation and labeling processing on the acquired data; the data is distributed to a data standardization processing module and an alarm work order system after being labeled;
and S4, deploying a web end, and realizing the unified display of the data.
Preferably, massive call chain log information is collected by using a data lake product of an open source component sky walking plus wave and is stored in an elastic search, the upstream-downstream relation of the call chain log and the aggregation calculation of key indexes are analyzed in real time through a stream processing frame, finally, the display of the call chain topological relation is realized through an Echarts customized development front end, and the drilling to specific services is supported to check the key index trend and alarm information.
A computer readable storage medium, which stores computer executable instructions, when a processor executes the computer, the implementation method of the unified monitoring platform based on application system representation as described above is realized.
The unified monitoring platform based on the application system portrait and the realization method thereof have the following advantages:
the method is mainly applied to solving the problems that the existing system is dispersed, the monitoring index analysis is single, the business association analysis cannot be realized, and the scene analysis is insufficient, so that the purposes of unified monitoring and high-efficiency operation and maintenance are realized;
the invention forms a unified view of data dispersed in a plurality of systems, establishes index analysis of a service visual angle, solves the problem of single display of indexes, and realizes the functions of depth interactive analysis of indexes, scene customization and scene solidification; the method comprises the steps that a mature data visualization tool and a service topological graph are used for achieving a unified monitoring view of cross-system indexes, correlation between the service indexes and basic indexes is achieved through service system images, analysis capability of depth-depth interaction is achieved through calling link tracking, and customized monitoring under a special scene is achieved;
and thirdly, comprehensively displaying key indexes of different systems in a unified interface, and associating real-time alarm data to perform alarm prompt of the service node. The centralized control and management of multi-system index data are realized, a monitoring index system is enriched, end-to-end monitoring is realized from the perspective of technical architecture and business process, and all-round three-dimensional monitoring from business, application, function and system level is realized, so that operation and maintenance personnel can quickly obtain information which can be used for fault (alarm) processing decision, system faults are recovered as soon as possible, and operation and maintenance efficiency is improved;
the unified monitoring platform realizes end-to-end monitoring through unified data management and control and display, and performs all-around three-dimensional monitoring from service, application, function and system level; network resources, hardware resources, software resources, database resources and the like related in all service systems are brought into a unified operation and maintenance monitoring platform, unified management, unified specification, unified processing, unified display, unified user login and unified authority control are realized on various different data sources by eliminating the difference of management software and the difference of data acquisition means, and finally, operation and maintenance standardized, automatic and intelligent large operation and maintenance management is realized;
the unified web view realizes the display standardization and customizability, and operation and maintenance personnel do not need to switch a plurality of systems back and forth to manually acquire data, so that the problem positioning time can be shortened;
for the application system image module, realizing service system image according to service requirements, integrating service architecture topology and key service monitoring indexes, displaying data in a centralized and associated manner, carrying out context associated query on operation and maintenance data based on a system-component-server and one key, and realizing end-to-end omnibearing three-dimensional monitoring through trend comparison and checking;
the method realizes complete single-stroke calling tracking, visual calling relation statistical display and real-time dynamic association with performance indexes, fast retrieval of single-stroke calling chain logs, and can realize drill-down analysis of abnormal calling chain information by matching with front-end customization development, wherein drill-down can display calling chain detail logs with abnormal key indexes, and calling chain information displayed in a time sequence mode can intuitively find calling abnormal nodes;
the invention (eighthly) aims at the customization of the monitoring scene in the government affair cloud and the portrait analysis of the business system, realizes the centralized control and management of multi-system index data, enriches the monitoring index system, supports the aspects of technical architecture and business process, realizes end-to-end monitoring, and realizes the omnibearing three-dimensional monitoring from the business level, the application level, the function level and the system level.
Drawings
The invention is further described below with reference to the accompanying drawings.
FIG. 1 is a schematic diagram of a unified monitoring platform based on application system representation;
fig. 2 is a schematic diagram of the operation process of the data extraction processing module.
Detailed Description
The unified monitoring platform and the implementation method based on application system representation of the present invention are described in detail below with reference to the drawings and the specific embodiments of the specification.
In the description of the present invention, it should be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
Example 1:
as shown in FIG. 1, the present invention provides a unified monitoring platform based on application system representation, which comprises,
the data extraction processing module is used for directly acquiring data from the original monitoring system or the original business system and processing the data;
the application system portrait module is used for system portrait, and the system portrait processing data is displayed on a Web end on the monitoring platform according to the service requirement and is sent to the alarm work order system according to the service definition;
the data acquisition module is used for covering monitoring data of manufacturers of the original cloud platforms, and newly acquiring data which is not contained in the original cloud platforms according to the standard;
the data standardization processing module is used for realizing the access, analysis, screening, filtering and storage of log data and index data; the method has the advantages of adapting to various data sources such as various log files, system events, indexes, databases and the like, and the acquired data are formatted and then stored in the database of the unified monitoring platform. Network resources, hardware resources, software resources, database resources and the like related in all service systems are brought into a unified operation and maintenance monitoring platform, and unified management, unified specification and unified processing are realized on various data sources by eliminating the difference of management software and the difference of data acquisition means.
As shown in fig. 2, the working process of the data extraction processing module in this embodiment is as follows:
(1) extracting data from the original monitoring system through a data acquisition interface;
(2) carrying out data acquisition formatting processing and classified storage on the extracted data, and carrying out data cleaning, de-duplication and compression processing;
(3) signing various data;
(4) sending the processed data to an application system image module for processing;
(5) analyzing and summarizing the data according to the service dimension and the time dimension;
(6) and sending the processed data to an alarm work order system for alarm judgment and analysis, and simultaneously displaying on a unified monitoring platform page according to monitoring service requirements.
The application system image module in this embodiment performs classification and labeling processing using the collected information, which is specifically as follows:
performing association processing by combining the asset data and the service call relation data of the CMDB: the method comprises the following specific steps:
acquiring multi-level topological relation of a business system by acquiring asset configuration data and business relation data of a CMDB;
secondly, relating key indexes of different objects to specific nodes of service topology;
thirdly, various key index data on the service topological graph are finally gathered through real-time aggregation calculation of the stream processing framework, so that report configurability and monitoring scene customization functions are realized, and the dependence of monitoring scene changes on system research personnel is eliminated;
and (II) performing labeling treatment after modeling: labeling various hardware and software according to physical attribute, configuration attribute, operation data attribute, operation and maintenance data attribute and intelligent attribute; the hardware comprises a network, a cloud server and a cloud service physical host; the software includes an operating system, a database, and middleware.
The labeling in the embodiment comprises manual labeling and machine labeling;
the manual tagging is to set a tag dictionary according to operation and maintenance experience and specifications, and is formed by querying a data association dictionary when data are collected, for example, equipment manufacturer information is a physical and non-modifiable attribute, and an ip address of a cloud server is a configuration attribute;
the machine labeling is to perform labeling through a performance data and application access association relation, for example, extracting keywords and performing cluster analysis on equipment (software) according to log information, performance data and fault frequency, and then labeling, for example, stability and access frequency; machine labeling was corrected by hand.
And sorting the labeled data according to labels, numbers and values, and performing aggregation and normalization processing according to an analysis scene to form ordered and standard data.
The Web end in the embodiment utilizes the basic table, the report and the Echart component to display data in a customized scene, and provides a visual, vivid, interactive and highly personalized data chart;
the data visualization chart supports mixing and matching between graphs, and relevant data are deposited into analysis reports of various dimensions; the data visualization chart comprises a line chart, a bar chart, a scatter chart, a pie chart, a k-line chart, a box chart for statistics, a map for geographical data visualization, a thermodynamic chart and a line chart, a relational chart for data relation visualization, a treemap, parallel coordinates for multi-dimensional data visualization, a funnel chart for BI and a dashboard.
The information acquired by the data acquisition module in the embodiment comprises basic command information, platform information and middleware information; the basic command information comprises curl, ping and snmp; the platform information comprises openstack, vmware and ceph; the middleware information includes oracle, mysql, and Nginx.
Example 2:
the invention discloses a realization method of a unified monitoring platform based on application system portrait, which comprises the following steps:
s1, defining a standard interface, realizing data standardization of the existing cloud platform, and showing standardization; the method specifically comprises the following steps: unified monitoring management of a plurality of platforms in the government affair cloud is realized by defining a unified interface and acquiring data; the method comprises the steps of comprehensively monitoring and managing IT resources such as load balancing equipment, network equipment, a server, storage equipment, safety equipment, a database, middleware, application software and the like;
s2, according to the customized standard, butt-jointing the data lake product of the wave and the data of the existing cloud platform, and collecting the historical data of the cloud platform needing to be supervised, wherein the collected data comprises platform basic information and software hardware performance data; data are distributed to an application system portrait module and an alarm work order system for processing;
s3, deploying an application system portrait module, and performing aggregation calculation and labeling processing on the acquired data; the data is distributed to a data standardization processing module and an alarm work order system after being labeled;
and S4, deploying a web end, and realizing the unified display of the data.
The method comprises the steps of acquiring massive call chain log information by using a data lake product of an open source component skywalk and a wave, storing the information into an elastic search, analyzing the upstream-downstream relation of the call chain log and the aggregation calculation of key indexes in real time through a stream processing frame, finally realizing the display of the call chain topological relation through an Echarts customized development front end, and supporting drilling to specific services to check key index trends and alarm information.
Example 3:
there are ninety business systems and platforms managed by a provincial hall, and the business systems and platforms relate to a plurality of departments and units. The service systems are wide in user range, wide in coverage, high in importance and multiple in cooperative units, and use a plurality of cloud service providers such as the wave, the safe cloud, the purple light government affair cloud and the like, so that the service system has the characteristics of complex management, wide technical field, high safety requirement and the like. The scenes aggravate the difficulty and the complexity of the system in the aspects of unified operation and maintenance, problem positioning, problem solving, stable operation of the system, problem repair of a developer and the like. During the use of the operation and maintenance monitoring platforms, fault location usually needs a troubleshooting mode through a multi-person collaboration conference, a data visualization tool in a fault location scene optimizes a working collaboration mode during troubleshooting, and the problem range is quickly narrowed through unified and centralized visualization view display in the system, so that troubleshooting is more targeted, the time of blind dispute is reduced, the operation and maintenance efficiency is improved, and the operation and maintenance quality and satisfaction are improved.
The real-time collection of indexes such as business, application, service, equipment and the like forms all-dimensional and three-dimensional monitoring data, and the multidimensional information is displayed through a panoramic image by a visual tool, so that the efficiency of positioning problems is improved, and the time for fault treatment is shortened.
In the operation and maintenance management process, the problems encountered by the user in the application process can be really solved according to the requirement of the user, the management and control of the service platform are enhanced, and the value of the method is mainly embodied in the following aspects:
firstly, accurate positioning: based on the analysis of the multi-dimension of the topological structure, the call chain, the affairs and the like in the application, the method helps a user to quickly master the running state of the application and positions the root cause of the problem.
②, AIOps: the AI performs autonomous detection on the threshold, performs performance early warning and performance bottleneck prediction in time according to an anomaly detection algorithm and learning of baseline performance data, and provides data support for resource optimization of users.
Example 4:
the embodiment of the invention also provides a computer-readable storage medium, wherein a plurality of instructions are stored, and the instructions are loaded by the processor, so that the processor executes the implementation method of the unified monitoring platform based on the application system portrait in any embodiment of the invention. Specifically, a system or an apparatus equipped with a storage medium on which software program codes that realize the functions of any of the above-described embodiments are stored may be provided, and a computer (or a CPU or MPU) of the system or the apparatus is caused to read out and execute the program codes stored in the storage medium.
In this case, the program code itself read from the storage medium can realize the functions of any of the above-described embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.
Examples of the storage medium for supplying the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (e.g., CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD + RW), a magnetic tape, a nonvolatile memory card, and a ROM. Alternatively, the program code may be downloaded from a server computer via a communications network.
Further, it should be clear that the functions of any one of the above-described embodiments may be implemented not only by executing the program code read out by the computer, but also by causing an operating system or the like operating on the computer to perform a part or all of the actual operations based on instructions of the program code.
Further, it is to be understood that the program code read out from the storage medium is written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion unit connected to the computer, and then causes a CPU or the like mounted on the expansion board or the expansion unit to perform part or all of the actual operations based on instructions of the program code, thereby realizing the functions of any of the above-described embodiments.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A unified monitoring platform based on application system image is characterized in that the monitoring platform comprises,
the data extraction processing module is used for directly acquiring data from the original monitoring system or the original business system and processing the data;
the application system portrait module is used for system portrait, and the system portrait processing data is displayed on a Web end of the monitoring platform according to the service requirement;
the data acquisition module is used for covering monitoring data of manufacturers of the original cloud platforms, and newly acquiring data which is not contained in the original cloud platforms according to the standard;
and the data standardization processing module is used for realizing the access, analysis, screening, filtering and storage of the log data and the index data.
2. The application system representation-based unified monitoring platform of claim 1, wherein said data extraction processing module specifically operates as follows:
(1) extracting data from the original monitoring system through a data acquisition interface;
(2) carrying out data acquisition formatting processing and classified storage on the extracted data, and carrying out data cleaning, de-duplication and compression processing;
(3) signing various data;
(4) sending the processed data to an application system image module for processing;
(5) analyzing and summarizing the data according to the service dimension and the time dimension;
(6) and sending the processed data to an alarm work order system for alarm judgment and analysis, and simultaneously displaying on a unified monitoring platform page according to monitoring service requirements.
3. The unified monitoring platform for application system portrayal as claimed in claim 1, wherein said application system portrayal module uses the collected information for classification and labeling, specifically as follows:
and associating the data by combining the asset data and the service calling relation data of the CMDB: the method comprises the following specific steps:
acquiring a multilevel topological relation of a service system by acquiring asset configuration data and service relation data of the CMDB;
associating key indexes of different objects to specific nodes of a service topology;
finally, various key index data on the service topological graph are gathered through the real-time aggregation calculation of the stream processing framework;
labeling treatment is carried out after modeling: labeling various hardware and software according to physical attribute, configuration attribute, operation data attribute, operation and maintenance data attribute and intelligent attribute; the hardware comprises a network, a cloud server and a cloud service physical host; the software includes an operating system, a database, and middleware.
4. The application system representation-based unified monitoring platform of claim 3, wherein said labeling process is divided into manual labeling and machine labeling;
the manual labeling is formed by setting a label dictionary according to operation and maintenance experience and specifications and inquiring a data association dictionary when data are collected;
the machine labeling is performed through the association relationship between performance data and application access, and the machine labeling is corrected manually.
5. The application system portrait-based unified monitoring platform of claim 4, wherein tagged data is sorted according to tags, numbers and values, aggregated and normalized according to analysis scenarios to form ordered and normative data.
6. The application system representation-based unified monitoring platform of claim 1, wherein said Web-side utilizes basic forms, statements and Echart components for customized scene display data, providing visual, vivid, interactive, highly personalized and customizable data visualization charts;
the data visualization chart supports mixing and matching between graphs, and relevant data are deposited into analysis reports of various dimensions; the data visualization chart comprises a line chart, a bar chart, a scatter chart, a pie chart, a k-line chart, a box chart for statistics, a map for geographical data visualization, a thermodynamic chart and a line chart, a relational chart for data relation visualization, a treemap, parallel coordinates for multi-dimensional data visualization, a funnel chart for BI and a dashboard.
7. The application system representation-based unified monitoring platform of claim 1, wherein said data acquisition module acquires information comprising basic command information, platform information and middleware information; the basic command information comprises curl, ping and snmp; the platform information comprises openstack, vmware and ceph; the middleware information includes oracle, mysql, and Nginx.
8. A realization method of a unified monitoring platform based on application system portrait is characterized by comprising the following steps:
s1, defining a standard interface, realizing data standardization of the existing cloud platform, and showing standardization;
s2, according to the customized standard, butt-jointing the data lake product of the wave and the data of the existing cloud platform, and collecting the historical data of the cloud platform needing to be supervised, wherein the collected data comprises platform basic information and software hardware performance data; data are distributed to an application system portrait module and an alarm work order system for processing;
s3, deploying an application system portrait module, and performing aggregation calculation and labeling processing on the acquired data; the data is distributed to a data standardization processing module and an alarm work order system after being labeled;
and S4, deploying a web end, and realizing the unified display of the data.
9. The implementation method of the unified monitoring platform based on the application system portrait is characterized in that massive call chain log information is collected by using data lake products of open source components skywalk and wave tides and stored in an elastic search, the upstream and downstream relation of the call chain log and the aggregation calculation of key indexes are analyzed in real time through a stream processing framework, finally, the display of the call chain topological relation is realized through an Echarts customized development front end, and the downward drilling to specific services is supported to check the key index trend and the alarm information.
10. A computer readable storage medium, having stored thereon computer executable instructions, which when executed by a processor implement a method for implementing an application system representation-based unified monitoring platform according to any of claims 8 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110778488.0A CN113505048A (en) | 2021-07-09 | 2021-07-09 | Unified monitoring platform based on application system portrait and implementation method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110778488.0A CN113505048A (en) | 2021-07-09 | 2021-07-09 | Unified monitoring platform based on application system portrait and implementation method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113505048A true CN113505048A (en) | 2021-10-15 |
Family
ID=78012477
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110778488.0A Pending CN113505048A (en) | 2021-07-09 | 2021-07-09 | Unified monitoring platform based on application system portrait and implementation method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113505048A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114492877A (en) * | 2022-04-13 | 2022-05-13 | 北京蒙帕信创科技有限公司 | Operation and maintenance analysis method and device of business system |
| CN114500293A (en) * | 2022-01-12 | 2022-05-13 | 浙江鸿程计算机系统有限公司 | Method for processing communication business data of government, enterprise and business buildings |
| CN114528452A (en) * | 2022-02-18 | 2022-05-24 | 浪潮卓数大数据产业发展有限公司 | Data tag implementation method and system based on tobacco and wine sales |
| US11921704B2 (en) | 2021-12-28 | 2024-03-05 | Vmware, Inc. | Version control interface for accessing data lakes |
-
2021
- 2021-07-09 CN CN202110778488.0A patent/CN113505048A/en active Pending
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11921704B2 (en) | 2021-12-28 | 2024-03-05 | Vmware, Inc. | Version control interface for accessing data lakes |
| CN114500293A (en) * | 2022-01-12 | 2022-05-13 | 浙江鸿程计算机系统有限公司 | Method for processing communication business data of government, enterprise and business buildings |
| CN114528452A (en) * | 2022-02-18 | 2022-05-24 | 浪潮卓数大数据产业发展有限公司 | Data tag implementation method and system based on tobacco and wine sales |
| CN114492877A (en) * | 2022-04-13 | 2022-05-13 | 北京蒙帕信创科技有限公司 | Operation and maintenance analysis method and device of business system |
| CN114492877B (en) * | 2022-04-13 | 2022-07-05 | 北京蒙帕信创科技有限公司 | Operation and maintenance analysis method and device of business system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113505048A (en) | Unified monitoring platform based on application system portrait and implementation method | |
| CN110493348B (en) | Intelligent monitoring alarm system based on Internet of things | |
| CN104407964B (en) | A kind of centralized monitoring system and method based on data center | |
| CN108039957B (en) | Intelligent analysis system for complex network traffic packet | |
| US20180129579A1 (en) | Systems and Methods with a Realtime Log Analysis Framework | |
| CN110659173A (en) | Operation and maintenance system and method | |
| CN108197261A (en) | A kind of wisdom traffic operating system | |
| CN107958337A (en) | A kind of information resources visualize mobile management system | |
| EP3278243A1 (en) | Method and system for implementing a log parser in a log analytics system | |
| CN108763957A (en) | A kind of safety auditing system of database, method and server | |
| CN104243236A (en) | Method, system and servers for analyzing monitoring system operation and maintenance alarm data | |
| CN102567531B (en) | General method for monitoring status of light database | |
| WO2016161381A1 (en) | Method and system for implementing a log parser in a log analytics system | |
| CN103295155A (en) | Security core service system monitoring method | |
| CN111858713A (en) | Object-based government information asset management method and system | |
| CN113448812A (en) | Monitoring alarm method and device under micro-service scene | |
| CN112559237B (en) | Operation and maintenance system troubleshooting method and device, server and storage medium | |
| CN108390782A (en) | A kind of centralization application system performance question synthesis analysis method | |
| CN112769605A (en) | Heterogeneous multi-cloud operation and maintenance management method and hybrid cloud platform | |
| US10089167B2 (en) | Log file reduction according to problem-space network topology | |
| CN111884853A (en) | Cloud environment automatic resource management method and system | |
| CN106951360B (en) | Data statistical integrity calculation method and system | |
| CN110968479A (en) | Business-level full-link monitoring method for application program and server | |
| CN112134719A (en) | Method and system for analyzing base station security log | |
| CN110138583B (en) | Display method for intelligent alarm analysis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |