CN112350898A - Micro-service application full-link performance real-time monitoring system and detection method thereof - Google Patents
Micro-service application full-link performance real-time monitoring system and detection method thereof Download PDFInfo
- Publication number
- CN112350898A CN112350898A CN202011244882.8A CN202011244882A CN112350898A CN 112350898 A CN112350898 A CN 112350898A CN 202011244882 A CN202011244882 A CN 202011244882A CN 112350898 A CN112350898 A CN 112350898A
- Authority
- CN
- China
- Prior art keywords
- micro
- performance
- service
- data
- tool
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/04—Processing captured monitoring data, e.g. for logfile generation
- H04L43/045—Processing captured monitoring data, e.g. for logfile generation for graphical visualisation of monitoring data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/12—Network monitoring probes
Abstract
The invention relates to the technical field of micro-server performance monitoring, in particular to a micro-service application full-link performance real-time monitoring system and a detection method thereof. The micro-service performance is monitored and analyzed in real time through the coordinated operation of modules such as data collection, data storage, elastic telescopic configuration and integrated scheduling. Meanwhile, an elastic expansion mode is provided by combining scene driving and time sliding window technology and calculating and predicting performance indexes aiming at the elastic expansion configuration of the cloud service in the framework. Experiments prove that the mode can effectively predict resource expansion and reduce resources. The framework is beneficial to real-time monitoring and continuous optimization of micro-service application, and promotes the micro-service-oriented research, development, test, operation and maintenance integrated process of national power grid companies.
Description
Technical Field
The invention relates to the technical field of micro-server performance monitoring, in particular to a micro-service application full-link performance real-time monitoring system and a detection method thereof.
Background
The micro-service architecture can split different modules in a traditional software system in a micro-service mode, and a huge single application is decomposed into mutually independent micro-services. Each service is dedicated to a single business, running in separate processes and possibly with separate databases. The services adopt a lightweight communication mechanism to carry out communication call among each other. Due to the characteristics of low coupling and splitting, a distributed architecture and a communication mechanism of microservice application, higher requirements are put forward on the performance monitoring of the microservice application. Generally, in a microservice application, a user completes a business operation, and may need to complete the business operation through several microservice sequential calls, that is, the output data of one microservice operation will be the input data of the next microservice. The dependency relationship between services may cause the performance of the whole service invocation to be affected due to the performance of a certain micro-service. In order to effectively ensure that a large server sequence runs well and provide reliable information service, enterprises must have a certain server performance monitoring means. Therefore, the key to ensure the high-efficiency operation of the software is to realize the real-time monitoring of the micro-service performance and to make effective analysis and intervention in time.
The existing micro-service room lacks real-time performance monitoring based on a real scene, and cannot perform high-efficiency elastic expansion management on the micro-service based on a real service time window.
Disclosure of Invention
Technical problem to be solved
In order to solve the above problems in the prior art, the present invention provides a micro-service application full link performance real-time monitoring system and a detection method thereof, which realize real-time monitoring and analysis of micro-service performance through the coordinated operation of modules such as data collection, data storage, elastic flexible configuration, integrated scheduling, and the like. Meanwhile, an elastic expansion mode is provided by combining scene driving and time sliding window technology and calculating and predicting performance indexes aiming at the elastic expansion configuration of the cloud service in the framework. Experiments prove that the mode can effectively predict resource expansion and reduce resources. The framework is beneficial to real-time monitoring and continuous optimization of micro-service application, and promotes the micro-service-oriented research, development, test, operation and maintenance integrated process of national power grid companies.
(II) technical scheme
In order to achieve the purpose, the invention adopts the main technical scheme that: a micro-service application full-link performance real-time monitoring system comprises an integrated scheduling tool, a performance testing tool, a data collector, a big data memory and an elastic expansion manager;
the integrated scheduling tool is used for initiating and scheduling tasks for performance monitoring;
the performance testing tool performs performance testing on the micro-service by executing a testing script;
the data collector is used for collecting response time and resource indexes of the server;
the big data storage device collects and stores information data transmitted by the data collector;
the elastic expansion manager carries out relative smoothing processing on the time window data, applies the time window data to an elastic expansion algorithm, and carries out resource management configuration on the micro-service by matching different preset corresponding threshold values.
Preferably, the detection system further comprises a decision support platform, the decision support platform is used for displaying the annual, seasonal and monthly multidimensional performance indexes and related reports, and the manual adjustment resource management tool is used for manually allocating the micro service resources.
Preferably, the detection system further comprises a manual adjustment resource management tool, and the manual adjustment resource management tool is used for manually allocating the micro service resources.
Preferably, the indexes collected by the data collector are collected by performance probes embedded in the servers.
A method for detecting full link performance of micro service application in real time includes carrying out performance test on micro service by performance test tool through executing test script, carrying out collection of response time and resource index by starting data collector, carrying out big data storage on data and starting up automatic elastic expansion mode according to calculation of performance index and relevant algorithm.
(III) advantageous effects
The invention provides a micro-service application full-link performance real-time monitoring system and a detection method thereof. The method has the following beneficial effects:
(1) all indexes are displayed from the whole dimensionality to the local dimensionality in the full link performance monitoring, performance information of all cross-application calling chains is displayed in a centralized mode, the whole performance and the local performance can be measured conveniently, a source of fault generation can be found conveniently, and fault removal time can be shortened greatly in production.
(2) The full link tool can realize the tracking of the request link, quickly locate the fault and quickly locate the error information by calling the link combination service log. Each phase is time consuming and a visual performance analysis is performed. And (4) combing the service dependence relationship and optimizing according to the availability of each calling link. Behavior paths of users can be obtained, and summarizing analysis is applied to a plurality of service scenes.
Drawings
FIG. 1 is a block diagram of the system of the present invention
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, the present invention provides a technical solution: a micro-service application full-link performance real-time monitoring system comprises an integrated scheduling tool, a performance testing tool, a data collector, a big data memory and an elastic expansion manager;
the integrated scheduling tool is used for initiating and scheduling tasks for performance monitoring;
the performance testing tool performs performance testing on the micro-service by executing the testing script;
the data collector is used for collecting response time and resource indexes of the server;
the big data storage device collects and stores the information data transmitted by the data collector;
the elastic expansion manager carries out relative smoothing processing on the time window data, applies the time window data to an elastic expansion algorithm, and carries out resource management configuration on the micro-service by matching different preset corresponding threshold values.
As a specific implementation manner of the present invention, the detection system further includes a decision support platform, the decision support platform is used for displaying the multi-dimensional performance indexes of year, quarter and month and related reports, and the manual adjustment resource management tool is used for manually allocating the micro service resources.
As a specific embodiment of the present invention, the detection system further includes a manual resource adjustment management tool, and the manual resource adjustment management tool is used for manually allocating the micro service resources.
In one embodiment of the present invention, the data collector collects the index by using a performance probe embedded in each server.
A method for detecting full link performance of micro service application in real time includes carrying out performance test on micro service by performance test tool through executing test script, carrying out collection of response time and resource index by starting data collector, carrying out big data storage on data and starting up automatic elastic expansion mode according to calculation of performance index and relevant algorithm.
The working principle is as follows: first, the performance testing tool performs a performance test on the microservice by executing a test script. And then, collecting response time and resource indexes by starting a data collector, storing big data of the data, and finally starting an automatic elastic expansion mode according to calculation of performance indexes and a related algorithm. The integrated scheduling tool is used for initiating and scheduling tasks for performance monitoring, and monitoring and controlling the performance of the micro-services are completed through data collection, big data storage, elastic flexible configuration, integrated scheduling and the like. An elastic expansion mode is provided by combining scene driving and time sliding window technology. The system efficiency is improved by the configuration in advance through the prediction of the performance index during the peak period of the service. And the threshold value is calculated by smoothing the performance index in the service low peak period, so that the utilization rate of resources is effectively improved. Experiments show that the elastic stretching mode can effectively predict the stretching of resources and save the resources.
During the low peak period of traffic, five time segments were selected for testing. Based on the mode described herein, the performance index is smoothed and not smoothed, and the start timing of elastic expansion and contraction is controlled by the same low peak period threshold TL. The experimental results are shown by comparing the smoothing processing modes of the low peak periods of the service. The result shows that after the performance index is subjected to smoothing treatment, the number of times of elastic expansion is reduced by 6.7% -27.3% under the environment that the service response time is increased by 2.9% -6.9%. In the period of low peak of the service, it is feasible to increase a very small amount of service response time to replace a part of running resources, and server resources and cost are saved to a certain extent.
Comparison of low peak period smoothing mode of service
During peak traffic hours, five time periods were chosen for testing, each time period containing 30 time windows. The performance index is predicted by the mode described herein, and the predicted effect is shown in table 2. The result shows that the predicted average error of the CPU utilization rate is between 9 and 14, and the minimum error is 0. The prediction method can effectively predict the performance index to a certain extent and prepare elastic expansion in advance. The problem that the processing is carried out after the traffic is sunk into the bottleneck of resources due to the sharp increase of the traffic in the peak period of the traffic is avoided.
Traffic rush hour window CPU usage prediction
The system efficiency is improved by the configuration in advance through the prediction of the performance index during the peak period of the service. And the threshold value is calculated by smoothing the performance index in the service low peak period, so that the utilization rate of resources is effectively improved. Experiments show that the elastic stretching mode can effectively predict the stretching of resources and save the resources.
Claims (5)
1. A micro-service application full link performance real-time monitoring system is characterized in that: the system comprises an integrated scheduling tool, a performance testing tool, a data collector, a big data memory and an elastic expansion manager;
the integrated scheduling tool is used for initiating and scheduling tasks for performance monitoring;
the performance testing tool performs performance testing on the micro-service by executing a testing script;
the data collector is used for collecting response time and resource indexes of the server;
the big data storage device collects and stores information data transmitted by the data collector;
the elastic expansion manager carries out relative smoothing processing on the time window data, applies the time window data to an elastic expansion algorithm, and carries out resource management configuration on the micro-service by matching different preset corresponding threshold values.
2. The system according to claim 1, wherein the system comprises: the detection system further comprises a decision support platform, the decision support platform is used for displaying the multi-dimensional performance indexes of year, quarter and month and related reports, and the manual adjustment resource management tool is used for manually allocating the micro-service resources.
3. The system according to claim 1, wherein the system comprises: the detection system further comprises a manual adjustment resource management tool, and the manual adjustment resource management tool is used for manually allocating the micro service resources.
4. The system according to claim 1, wherein the system comprises: the indexes collected by the data collector are collected through performance probes pre-embedded in the servers.
5. A method for detecting full link performance of micro-service application in real time is characterized in that: firstly, a performance testing tool performs performance testing on micro services by executing a testing script, then collects response time and resource indexes by starting a data collector, stores data in a big data mode, and finally starts an automatic elastic expansion mode according to calculation of performance indexes and related algorithms.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011244882.8A CN112350898A (en) | 2020-11-10 | 2020-11-10 | Micro-service application full-link performance real-time monitoring system and detection method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011244882.8A CN112350898A (en) | 2020-11-10 | 2020-11-10 | Micro-service application full-link performance real-time monitoring system and detection method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112350898A true CN112350898A (en) | 2021-02-09 |
Family
ID=74362549
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011244882.8A Pending CN112350898A (en) | 2020-11-10 | 2020-11-10 | Micro-service application full-link performance real-time monitoring system and detection method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112350898A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116961241A (en) * | 2023-09-20 | 2023-10-27 | 国网江苏省电力有限公司信息通信分公司 | Unified application monitoring platform based on power grid business |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030120502A1 (en) * | 2001-12-20 | 2003-06-26 | Robb Terence Alan | Application infrastructure platform (AIP) |
| CN108984269A (en) * | 2018-07-16 | 2018-12-11 | 中山大学 | Container resource provision method and system based on random regression forest model |
| CN109144724A (en) * | 2018-07-27 | 2019-01-04 | 众安信息技术服务有限公司 | A kind of micro services resource scheduling system and method |
| CN111045820A (en) * | 2019-12-03 | 2020-04-21 | 浩鲸云计算科技股份有限公司 | Container scheduling method based on time sequence prediction |
-
2020
- 2020-11-10 CN CN202011244882.8A patent/CN112350898A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030120502A1 (en) * | 2001-12-20 | 2003-06-26 | Robb Terence Alan | Application infrastructure platform (AIP) |
| CN108984269A (en) * | 2018-07-16 | 2018-12-11 | 中山大学 | Container resource provision method and system based on random regression forest model |
| CN109144724A (en) * | 2018-07-27 | 2019-01-04 | 众安信息技术服务有限公司 | A kind of micro services resource scheduling system and method |
| CN111045820A (en) * | 2019-12-03 | 2020-04-21 | 浩鲸云计算科技股份有限公司 | Container scheduling method based on time sequence prediction |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116961241A (en) * | 2023-09-20 | 2023-10-27 | 国网江苏省电力有限公司信息通信分公司 | Unified application monitoring platform based on power grid business |
| CN116961241B (en) * | 2023-09-20 | 2023-12-05 | 国网江苏省电力有限公司信息通信分公司 | Unified application monitoring platform based on power grid business |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110069348B (en) | Method for efficiently utilizing cloud center quantum computer resources | |
| CN102075384A (en) | Performance test system and method | |
| CN101808351B (en) | Method and system for business impact analysis | |
| CN107861796B (en) | Virtual machine scheduling method supporting energy consumption optimization of cloud data center | |
| CN104216782A (en) | Dynamic resource management method for high-performance computing and cloud computing hybrid environment | |
| CN103425564B (en) | A kind of smartphone software uses Forecasting Methodology | |
| CN103186603B (en) | Determine that SQL statement is on the method for the impact of the performance of key business, system and equipment | |
| CN112685153A (en) | Micro-service scheduling method and device and electronic equipment | |
| Cheng et al. | Elastic power-aware resource provisioning of heterogeneous workloads in self-sustainable datacenters | |
| CN112084016B (en) | Stream computing performance optimization system and method based on flink | |
| CN108898239A (en) | A kind of site selection method for distribution transformer based on data analysis | |
| CN115061809A (en) | Android-based RPA multi-task scheduling method and system | |
| He | Novel container cloud elastic scaling strategy based on Kubernetes | |
| CN112350898A (en) | Micro-service application full-link performance real-time monitoring system and detection method thereof | |
| CN109086816A (en) | A kind of user behavior analysis system based on Bayesian Classification Arithmetic | |
| Wang et al. | Cloud workload analytics for real-time prediction of user request patterns | |
| CN114297290A (en) | Electric power marketing data sharing application platform | |
| CN117076882A (en) | Dynamic prediction management method for cloud service resources | |
| CN116526553A (en) | Management method and system for improving power generation capacity of photovoltaic power station | |
| CN115840656A (en) | Automatic operation and maintenance method and system for application program based on fault self-healing | |
| CN113918636B (en) | ETL-based data throughput analysis method | |
| CN112052087B (en) | Deep learning training system and method for dynamic resource adjustment and migration | |
| Huang et al. | A novel compression algorithm decision method for spark shuffle process | |
| CN114240395A (en) | Resource delivery method and device, computer equipment and storage medium | |
| Zhou et al. | AHPA: adaptive horizontal pod autoscaling systems on alibaba cloud container service for kubernetes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210209 |
|
| RJ01 | Rejection of invention patent application after publication |