CN110784377A - Method for uniformly managing cloud monitoring data in multi-cloud environment - Google Patents
Method for uniformly managing cloud monitoring data in multi-cloud environment Download PDFInfo
- Publication number
- CN110784377A CN110784377A CN201911041092.7A CN201911041092A CN110784377A CN 110784377 A CN110784377 A CN 110784377A CN 201911041092 A CN201911041092 A CN 201911041092A CN 110784377 A CN110784377 A CN 110784377A
- Authority
- CN
- China
- Prior art keywords
- cloud
- monitoring
- monitoring data
- platform
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 124
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000013523 data management Methods 0.000 abstract description 4
- 230000006978 adaptation Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/04—Processing captured monitoring data, e.g. for logfile generation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45583—Memory management, e.g. access or allocation
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention relates to the technical field of monitoring data management, in particular to a method for uniformly managing cloud monitoring data in a multi-cloud environment. The method comprises the steps of collecting cloud platform monitoring data, adding a special identification of a platform to the collected monitoring data, and uniformly converting collection frequency; for subsequent display and use. The invention solves the problem that the monitoring data of all cloud platforms accessed by a multi-cloud system are uniformly processed and displayed; the cloud monitoring data management method can be used for cloud monitoring data management in a multi-cloud environment.
Description
Technical Field
The invention relates to the technical field of monitoring data management, in particular to a method for uniformly managing cloud monitoring data in a multi-cloud environment.
Background
With the increasingly wide cloud demand scenarios, multi-cloud deployment has become the primary solution for enterprise cost and complex scenarios. In a multi-cloud system, resource monitoring data of each public cloud platform is particularly important. Thousands of monitoring data can be generated at one time point in a cloud platform; in a cloudy background, the amount of monitoring data is more immeasurable; how to efficiently process and use such monitoring data is a problem faced in the art.
The mainstream monitoring means of the existing multi-cloud product in the market is to acquire real-time monitoring data by sending a request or calling an SDK (software development kit) when the monitoring data needs to be inquired; and then directly returns to be displayed for the user to view. The method can cause that the monitoring data is too one-sided and inflexible, the monitoring data structures of different types of cloud platforms are inconsistent, the monitoring frequencies are inconsistent, and the monitoring data is difficult to be uniformly displayed and processed.
Disclosure of Invention
The invention solves the technical problem of providing a method for uniformly managing cloud monitoring data in a multi-cloud environment; the monitoring data of all cloud platforms accessed by the multi-cloud system are processed and displayed uniformly.
The technical scheme for solving the technical problems is as follows:
the method comprises the steps of collecting cloud platform monitoring data, adding a platform special identifier to the collected monitoring data, and uniformly converting collection frequency; for subsequent display and use.
The method comprises the following specific steps:
s1, regularly pulling cloud platform monitoring data;
s2, recalculating the monitoring data according to the time frequency;
s3, adding a special identifier of the platform for each piece of monitoring data;
s4, writing into a unified monitoring time sequence database;
s5, outputting display monitoring data;
and S6, monitoring triggering an alarm and informing a user.
In the step S1, a cloud platform that needs to pull monitoring data is configured by a configuration file; the pulled monitoring data specifically includes:
1) the data related to the virtual machine resources comprise a virtual machine cpu, a virtual machine memory, a virtual machine disk io and a virtual machine network rate;
2) the data related to the virtual machine user comprises personal information and alarm configuration information of the virtual machine user;
3) data related to the cloud platform, including a platform type, an area id and the like;
4) and monitoring the acquisition time and the acquisition frequency of the data.
The time frequency in the step S2 is the uniform collection frequency of the multi-cloud system; the generation mode of the time frequency is as follows:
acquiring the minimum intersection of the monitoring frequencies of the cloud platforms according to the monitoring frequencies of the cloud platforms acquired in the step S1, and ensuring that the monitoring frequencies can act on all accessed platforms; then, comparing the acquisition frequency with the monitoring data frequency acquired by the cloud platform, converting the monitoring data of the cloud platform according to the uniform acquisition frequency of the multi-cloud system, and converting the monitoring data into data of uniform time frequency;
and writing the uniform acquisition frequency of the multi-cloud system into a monitoring configuration file, and changing the configuration according to the monitoring frequency of the new platform type when the new cloud platform type is accessed.
The special identifier of step S3 specifically includes:
1) type of public cloud platform;
2) and a platform area where the monitoring data is located.
The data structure constructed in step S4 is:
distinguishing each table by a monitoring item, wherein monitoring data of the monitoring item of all cloud platforms are stored in each monitoring item table;
adding special identification into the monitoring item table, and distinguishing monitoring data of different areas of different public cloud platforms by adopting two dimensions;
the monitoring data is sorted in time and the acquisition time is converted to a uniform standard.
The monitoring data of step S5 includes: a graph of virtual machine instance information and virtual machine monitoring item real-time data.
The alarm information of step 6S6 includes: the method comprises the following steps of virtual machine instance information triggering alarm, virtual machine monitoring item alarm values and threshold values triggering alarm, and platform types and areas of the platform triggering alarm.
The method comprises the steps of determining platform uniqueness of monitoring data in a multi-cloud system through two-dimensional identifiers, generating the identifiers when the multi-cloud system is connected to a cloud platform, enabling a public cloud type to be in a fixed format, randomly generating an area id through the multi-cloud platform, corresponding to the area id of an original platform, and checking the area id when the area monitoring data is inquired.
According to the cloud platform monitoring data acquisition method and device, the special platform area identification is added when the cloud platform monitoring data is acquired, and the acquisition frequency is uniformly converted, so that the effect of uniformly processing and displaying the monitoring data of all cloud platforms accessed by a multi-cloud system is achieved, and the requirement of uniformly monitoring the multi-cloud system is met.
Drawings
The invention is further described below with reference to the accompanying drawings:
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a block diagram of the process of the present invention.
Detailed Description
Referring to fig. 1, the method for implementing unified management of cloud monitoring data according to the present invention includes the following steps S1-S6.
And S1, regularly pulling the cloud platform monitoring data.
In this step, what the cloud platform that needs to be pulled has can be configured by the configuration file. The pulled monitoring data specifically includes:
1) data related to virtual machine resources, such as a virtual machine cpu, a virtual machine memory, a virtual machine disk io, and a virtual machine network rate;
2) the data related to the virtual machine user comprises personal information and alarm configuration information of the virtual machine user;
3) data related to the cloud platform, including a platform type, an area id and the like;
4) and monitoring the acquisition time and the acquisition frequency of the data.
And S2, recalculating the monitoring data according to the time frequency.
The time frequency refers to the uniform acquisition frequency of a multi-cloud system, and the frequency is generated in the following mode: acquiring the minimum intersection of the monitoring frequencies of the cloud platforms according to the monitoring frequencies of the cloud platforms acquired in the step S1, and ensuring that the monitoring frequencies can act on all accessed platforms; and then, comparing the acquisition frequency with the monitoring data frequency acquired by the cloud platform for calculation, converting the monitoring data of the cloud platform according to the uniform acquisition frequency of the multi-cloud system, and converting the monitoring data into data of uniform time frequency.
The unified acquisition frequency of the multi-cloud system is written into a monitoring configuration file, and when a new cloud platform type is accessed, the configuration can be flexibly changed according to the monitoring frequency of the new platform type.
And S3, adding a platform special identifier for each piece of monitoring data.
The added platform specific identifier specifically includes two dimensions:
1. the types of public cloud platforms, such as various cloud platforms of different types, including Ali cloud, Huashi cloud, Tencent cloud and the like, are defined and generated by a multi-cloud system;
2. the identification of the platform area where the monitoring data is located, such as Guangzhou area of Tencent cloud: cn-source-1, such identification is converted into a unified format, such as concept-regionxxx, by the multi-cloud system adaptation layer;
the platform uniqueness of the monitoring data is determined in the multi-cloud system through the identification of two dimensions, the identification is generated when the multi-cloud system is connected to a cloud platform, the public cloud type is in a fixed format, the area id is randomly generated through the multi-cloud platform and corresponds to the area id of the original platform, and the id is verified when the area monitoring data is inquired.
Fig. 2 shows a relationship structure between a multi-cloud platform and types and areas of the cloud platform.
And S4, writing the data into a unified monitoring time sequence database.
When the unified monitoring time sequence database is written, the constructed data structure is as follows:
distinguishing each table by a monitoring item, wherein monitoring data of the monitoring item of all cloud platforms are stored in each monitoring item table;
adding special identification into the monitoring item table, and distinguishing monitoring data of different areas of different public cloud platforms by adopting two dimensions;
the monitoring data is time-ordered and the acquisition time is translated into a uniform standard, i.e. the acquisition frequency used by all platforms is uniform.
And S5, outputting the display monitoring data.
The displayed monitoring data specifically includes:
a graph of virtual machine instance information and virtual machine monitoring item real-time data.
And S6, monitoring triggering an alarm and informing a user.
The content of the alarm information informing the user comprises the following steps:
triggering alarm virtual machine instance information;
triggering alarm virtual machine monitoring item alarm value and threshold;
the type of platform and the area where the alarm is triggered.
Claims (9)
1. A method for uniformly managing cloud monitoring data in a multi-cloud environment is characterized by comprising the following steps: the method comprises the steps of collecting cloud platform monitoring data, adding a platform special identifier to the collected monitoring data, and uniformly converting collection frequency; for subsequent display and use.
2. The method of claim 1, wherein: the method comprises the following specific steps:
s1, regularly pulling cloud platform monitoring data;
s2, recalculating the monitoring data according to the time frequency;
s3, adding a special identifier of the platform for each piece of monitoring data;
s4, writing into a unified monitoring time sequence database;
s5, outputting display monitoring data;
and S6, monitoring triggering an alarm and informing a user.
3. The method of claim 2, wherein: in the step S1, a cloud platform that needs to pull monitoring data is configured by a configuration file; the pulled monitoring data specifically includes:
1) the data related to the virtual machine resources comprise a virtual machine cpu, a virtual machine memory, a virtual machine disk io and a virtual machine network rate;
2) the data related to the virtual machine user comprises personal information and alarm configuration information of the virtual machine user;
3) cloud platform related data comprising a platform type and a region id;
4) and monitoring the acquisition time and the acquisition frequency of the data.
4. The method of claim 2, wherein: the time frequency in the step S2 is the uniform collection frequency of the multi-cloud system; the generation mode of the time frequency is as follows:
acquiring the minimum intersection of the monitoring frequencies of the cloud platforms according to the monitoring frequencies of the cloud platforms acquired in the step S1, and ensuring that the monitoring frequencies can act on all accessed platforms; then, comparing the acquisition frequency with the monitoring data frequency acquired by the cloud platform, converting the monitoring data of the cloud platform according to the uniform acquisition frequency of the multi-cloud system, and converting the monitoring data into data of uniform time frequency;
and writing the uniform acquisition frequency of the multi-cloud system into a monitoring configuration file, and changing the configuration according to the monitoring frequency of the new platform type when the new cloud platform type is accessed.
5. The method of claim 2, wherein: the special identifier of step S3 specifically includes:
1) type of public cloud platform;
2) and a platform area where the monitoring data is located.
6. The method of claim 2, wherein: the data structure constructed in step S4 is:
distinguishing each table by a monitoring item, wherein monitoring data of the monitoring item of all cloud platforms are stored in each monitoring item table;
adding special identification into the monitoring item table, and distinguishing monitoring data of different areas of different public cloud platforms by adopting two dimensions;
the monitoring data is sorted in time and the acquisition time is converted to a uniform standard.
7. The method of claim 2, wherein: the monitoring data of step S5 includes: a graph of virtual machine instance information and virtual machine monitoring item real-time data.
8. The method of claim 2, wherein: the alarm information of step 6S6 includes: the method comprises the following steps of virtual machine instance information triggering alarm, virtual machine monitoring item alarm values and threshold values triggering alarm, and platform types and areas of the platform triggering alarm.
9. The method according to claim 5 or 6, characterized in that: the method comprises the steps of determining platform uniqueness of monitoring data in a multi-cloud system through two-dimensional identifiers, generating the identifiers when the multi-cloud system is connected to a cloud platform, enabling a public cloud type to be in a fixed format, randomly generating an area id through the multi-cloud platform, corresponding to the area id of an original platform, and checking the area id when the area monitoring data is inquired.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911041092.7A CN110784377A (en) | 2019-10-30 | 2019-10-30 | Method for uniformly managing cloud monitoring data in multi-cloud environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911041092.7A CN110784377A (en) | 2019-10-30 | 2019-10-30 | Method for uniformly managing cloud monitoring data in multi-cloud environment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110784377A true CN110784377A (en) | 2020-02-11 |
Family
ID=69387501
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911041092.7A Withdrawn CN110784377A (en) | 2019-10-30 | 2019-10-30 | Method for uniformly managing cloud monitoring data in multi-cloud environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110784377A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112306801A (en) * | 2020-10-28 | 2021-02-02 | 联通(广东)产业互联网有限公司 | Monitoring data processing method, system and medium based on heterogeneous data source |
| CN113037828A (en) * | 2021-03-03 | 2021-06-25 | 浪潮云信息技术股份公司 | Data acquisition and control method and system in cloud environment |
| CN113127292A (en) * | 2021-03-17 | 2021-07-16 | 上海慧典软件科技有限公司 | Operation, maintenance and monitoring method suitable for multi-cloud management |
| CN113923406A (en) * | 2021-09-29 | 2022-01-11 | 四川警察学院 | Method, device, equipment and storage medium for adjusting video monitoring coverage area |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140047099A1 (en) * | 2012-08-08 | 2014-02-13 | International Business Machines Corporation | Performance monitor for multiple cloud computing environments |
| US20140365434A1 (en) * | 2012-02-24 | 2014-12-11 | Huawei Device Co., Ltd. | Data Collecting Method and Apparatus, and Mobile Terminal |
| WO2016025321A1 (en) * | 2014-08-13 | 2016-02-18 | OneCloud Labs, Inc. | Replication of virtualized infrastructure within distributed computing environments |
| US20160294666A1 (en) * | 2014-05-30 | 2016-10-06 | Boe Technology Group Co., Ltd. | Method and system for monitoring server cluster |
| CN108881460A (en) * | 2018-07-02 | 2018-11-23 | 郑州云海信息技术有限公司 | A kind of implementation method and realization device of cloud platform unified monitoring |
| CN108965056A (en) * | 2018-07-19 | 2018-12-07 | 郑州云海信息技术有限公司 | A kind of implementation method that cloud platform monitoring data summarizes and device |
| WO2019130009A1 (en) * | 2017-12-29 | 2019-07-04 | Agarik Sas | Orchestrated hybrid cloud platform for multi-cloud environment |
-
2019
- 2019-10-30 CN CN201911041092.7A patent/CN110784377A/en not_active Withdrawn
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140365434A1 (en) * | 2012-02-24 | 2014-12-11 | Huawei Device Co., Ltd. | Data Collecting Method and Apparatus, and Mobile Terminal |
| US20140047099A1 (en) * | 2012-08-08 | 2014-02-13 | International Business Machines Corporation | Performance monitor for multiple cloud computing environments |
| US20160294666A1 (en) * | 2014-05-30 | 2016-10-06 | Boe Technology Group Co., Ltd. | Method and system for monitoring server cluster |
| WO2016025321A1 (en) * | 2014-08-13 | 2016-02-18 | OneCloud Labs, Inc. | Replication of virtualized infrastructure within distributed computing environments |
| WO2019130009A1 (en) * | 2017-12-29 | 2019-07-04 | Agarik Sas | Orchestrated hybrid cloud platform for multi-cloud environment |
| CN108881460A (en) * | 2018-07-02 | 2018-11-23 | 郑州云海信息技术有限公司 | A kind of implementation method and realization device of cloud platform unified monitoring |
| CN108965056A (en) * | 2018-07-19 | 2018-12-07 | 郑州云海信息技术有限公司 | A kind of implementation method that cloud platform monitoring data summarizes and device |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112306801A (en) * | 2020-10-28 | 2021-02-02 | 联通(广东)产业互联网有限公司 | Monitoring data processing method, system and medium based on heterogeneous data source |
| CN112306801B (en) * | 2020-10-28 | 2023-01-03 | 联通(广东)产业互联网有限公司 | Monitoring data processing method, system and medium based on heterogeneous data source |
| CN113037828A (en) * | 2021-03-03 | 2021-06-25 | 浪潮云信息技术股份公司 | Data acquisition and control method and system in cloud environment |
| CN113127292A (en) * | 2021-03-17 | 2021-07-16 | 上海慧典软件科技有限公司 | Operation, maintenance and monitoring method suitable for multi-cloud management |
| CN113923406A (en) * | 2021-09-29 | 2022-01-11 | 四川警察学院 | Method, device, equipment and storage medium for adjusting video monitoring coverage area |
| CN113923406B (en) * | 2021-09-29 | 2023-05-12 | 四川警察学院 | Method, device, equipment and storage medium for adjusting video monitoring coverage area |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110784377A (en) | Method for uniformly managing cloud monitoring data in multi-cloud environment | |
| CN107145489B (en) | Information statistics method and device for client application based on cloud platform | |
| WO2018103590A1 (en) | Method, device, and system for assigning pickup tasks | |
| CN108874558B (en) | Message subscription method of distributed transaction, electronic device and readable storage medium | |
| CN113987074A (en) | Distributed service full-link monitoring method and device, electronic equipment and storage medium | |
| CN107133309B (en) | Method and device for storing and querying process example, storage medium and electronic equipment | |
| EP3916584A1 (en) | Information processing method and apparatus, electronic device and storage medium | |
| US8959051B2 (en) | Offloading collection of application monitoring data | |
| CN108595316A (en) | Life cycle management method, manager, equipment and the medium of Distributed Application | |
| CN108319539B (en) | Method and system for generating GPU card slot position information | |
| CN110147470B (en) | Cross-machine-room data comparison system and method | |
| CN114116065B (en) | Method and device for acquiring topological graph data object and electronic equipment | |
| CN112559538A (en) | Incidence relation generation method and device, computer equipment and storage medium | |
| CN115237857A (en) | Log processing method and device, computer equipment and storage medium | |
| WO2022237506A1 (en) | Method, apparatus, and device for monitoring online diagnosis service, and storage medium | |
| CN112711398A (en) | Method, device and equipment for generating buried point file and storage medium | |
| CN112433757A (en) | Method and device for determining interface calling relationship | |
| CN111597091A (en) | Data monitoring method and system, electronic equipment and computer storage medium | |
| CN113672497B (en) | Method, device and equipment for generating non-buried point event and storage medium | |
| CN112363774B (en) | Method and device for configuring Storm real-time task | |
| CN110020166A (en) | A kind of data analysing method and relevant device | |
| CN113626869A (en) | Data processing method, system, electronic device and storage medium | |
| US20170161359A1 (en) | Pattern-driven data generator | |
| CN113760568A (en) | Data processing method and device | |
| US10133997B2 (en) | Object lifecycle analysis tool |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20200211 |