CN104079662A - Measuring system and method - Google Patents
Measuring system and method Download PDFInfo
- Publication number
- CN104079662A CN104079662A CN201410334475.4A CN201410334475A CN104079662A CN 104079662 A CN104079662 A CN 104079662A CN 201410334475 A CN201410334475 A CN 201410334475A CN 104079662 A CN104079662 A CN 104079662A
- Authority
- CN
- China
- Prior art keywords
- daily record
- record data
- data
- log
- continuous data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Automatic Analysis And Handling Materials Therefor (AREA)
Abstract
The invention discloses a measuring system and method. According to the measuring method based on logs, various performance indexes of the system are measured and counted through a log adapter, a log data generator, a log analyzer, a measuring analysis controller and the like. A push mode is adopted for data acquisition, an existing log framework in a measured system is utilized, new execution codes do not need to be implanted into the system, therefore, the workload for developing the measured application system cannot be increased, the measuring complexity is reduced, and the influence on the operating performance of the system is little. Moreover, huge log data of the large application system is fully utilized, and thus the measuring accuracy is guaranteed.
Description
Technical field
The present invention relates to cloud computing treatment technology, metering system and method in espespecially a kind of cloud computing platform.
Background technology
Important role is being played the part of in metering in cloud computing system.By metering, can collect the data target of cloud computing platform in operation process as CPU usage, memory usage, the network bandwidth etc., to understand the ruuning situation of system, for other service on upper strata provides basic data as monitor service, billing of services etc.
At present, it is many that cloud computing resources relates to resource kind, comprises virtual machine, piece storage, object storage, network, middleware services, database service etc., the metering form of every kind of resource and require difference very large; Cloud computing platform is except providing resource service, the deployment that can also support various application types serves (SaaS as software, Software-as-a-service) application, and various dissimilar application, the form to metering and require difference also very large; Application on cloud computing platform can be carried out elastic telescopic according to the conversion of load, and therefore the scope of continuous data can be very large; Because cloud computing platform support is used as required, usage-based billing, metering is will be directly relevant with monitoring, charging, therefore requires continuous data accurate; On cloud computing platform, tend to dispose and widely apply, each application meeting produces mass data, therefore, in cloud environment, the continuous data rapid development of output; Cloud computing platform requires fast response time, high to requirement of real-time, obtains sooner measuring result, and it is better that user experiences.Therefore, efficiently, metered scheme is that cloud computing and large-scale application system are the bases of actual operation accurately, is the indispensable function of system.
In existing metering system, for the data target in operation process to cloud computing platform, such as CPU usage, memory usage etc. are collected, need to be in system code implant, affected the performance of system.The general framework of existing metering system is the mode of agency's (agent)+server (server), conventionally adopt the mode of drawing (pull) data, under this mode, need to agent be installed in measured system to collect data, and data are moved on server.This collecting method has reduced the performance of measured system, thereby can not ensure well the accuracy of metering; And data capacity is larger, efficiency is lower.In addition, due to the data format disunity that each measured system is used, also make the implantation of agent become difficulty.
Summary of the invention
In order to solve the problems of the technologies described above, the invention provides a kind of metering system and method, can reduce the complexity of metering, ensure the accuracy of metering simultaneously.
In order to reach the object of the invention, the invention provides a kind of metering system, at least comprise daily record adapter, daily record data maker, one or more continuous data containers, log analyzer, and quantitative analysis controller, wherein,
Daily record adapter, for the corresponding relation between storing daily record data and propelling movement target;
Daily record data maker, for according to the corresponding relation of daily record adapter, is pushed to corresponding continuous data container by the daily record data of the application system generation from each node of cloud computing platform;
Continuous data container, the daily record data pushing for storing daily record data maker, and from the log analysis result of log analyzer;
Log analyzer, under the control at quantitative analysis controller, carries out statistical analysis to the daily record data in continuous data container;
Quantitative analysis controller, for notifying log analyzer to carry out statistical analysis to the daily record data of continuous data container.
Described log analyzer also for, before described daily record data is analyzed, according to the attribute of daily record data, described daily record data is classified.
Also comprise chart display module, for showing daily record data and the log analysis result from described continuous data container;
Described quantitative analysis controller, also exports to chart display module for controlling described continuous data container by the daily record data of storage and log analysis result.
The present invention also provides a kind of metering method, having comprised: daily record data maker is according to the measuring parameter setting in advance, and the daily record data that the application system from each node of cloud computing platform is produced, pushes to continuous data container;
Log analyzer, according to the analytical parameters setting in advance, carries out statistical analysis to the daily record data in continuous data container.
Described daily record data in continuous data container is carried out to statistical analysis before, the method also comprises: according to the attribute of daily record data, described daily record data is classified.
The method also comprises: the log analysis result store that described log analyzer produces statistical analysis is in described continuous data container.
Compared with prior art, the present invention is based on the metering method of daily record, by daily record adapter, daily record data maker, log analyzer, quantitative analysis controller etc., the property indices of system is measured and added up.Data Collection adopts (push) mode that pushes away, and utilize existing daily record framework in measured system, need in system, not implant new run time version, therefore can not increase the development amount of measured application system, reduce the complexity of metering, and very little on the runnability impact of system.And the present invention makes full use of the huge daily record data of large-scale application system, ensure the accuracy of metering.
Other features and advantages of the present invention will be set forth in the following description, and, partly from specification, become apparent, or understand by implementing the present invention.Object of the present invention and other advantages can be realized and be obtained by specifically noted structure in specification, claims and accompanying drawing.
Brief description of the drawings
Accompanying drawing is used to provide the further understanding to technical solution of the present invention, and forms a part for specification, is used from and explains technical scheme of the present invention with the application's embodiment mono-, does not form the restriction to technical solution of the present invention.
Fig. 1 is the composition structural representation of metering system of the present invention;
Fig. 2 is the schematic flow sheet of the embodiment of metering method of the present invention.
Embodiment
For making the object, technical solutions and advantages of the present invention clearer, hereinafter in connection with accompanying drawing, embodiments of the invention are elaborated.It should be noted that, in the situation that not conflicting, the combination in any mutually of the feature in embodiment and embodiment in the application.
Can in the computer system such as one group of computer executable instructions, carry out in the step shown in the flow chart of accompanying drawing.And, although there is shown logical order in flow process, in some cases, can carry out shown or described step with the order being different from herein.
Fig. 1 is the composition structural representation of metering system of the present invention, and as shown in Figure 1, metering system of the present invention at least comprises daily record adapter, daily record data maker, one or more continuous data containers, log analyzer, and quantitative analysis controller, wherein,
Daily record adapter, for the corresponding relation between storing daily record data and propelling movement target, has shown that it is daily record data-pushing target that daily record data is outputed to which output medium.
Daily record data maker, for according to the corresponding relation of daily record adapter, is pushed to corresponding continuous data container by the daily record data of the application system generation from each node of cloud computing platform.
Continuous data container, the daily record data pushing for storing daily record data maker, and be that data target is as CPU usage, memory usage, the network bandwidth etc. from the log analysis result of log analyzer.In continuous data container, canned data can be referred to as metering result.
According to the feature of daily record data, continuous data container can adopt non-relational database MongoDB to realize.Wherein, MongoDB is a non-relational database, there is good scalability, it combines the advantage of relevant database and the storage of key-value, there is the features such as use is simple, efficiency is high, scalability good, the storage of support unstructured data, greatly improve the efficiency of log analysis and data query, can hold a large amount of daily record datas of quick generation simultaneously.
Log analyzer, for under the control of quantitative analysis controller, daily record data in continuous data container is carried out to statistical analysis, such as, according to the source of daily record, generation date, quantity, daily record rank etc., each data target of statistics application system, comprises the various performance index such as CPU usage, memory usage, the network bandwidth; And the log analysis result of generation is exported to continuous data container.To calculate CPU usage as example: if while supposing to produce 15000 daily records within the scope of Preset Time, in cloud computing platform, the CPU usage of equipment reaches 100%, so, according to the source of daily record, if device A has produced 10000 daily records in cloud computing platform, show that by statistical analysis the CPU usage of device A is 67%; And if equipment B has produced 1000 records, show that by statistical analysis the CPU usage of device A is 7%.
It should be noted that; here be only a kind of situation of simple example explanation statistical analysis; be not intended to limit the present invention the specific implementation of statistical analysis; specifically how statistical analysis is also not intended to limit the scope of the invention; be those skilled in the art's conventional techniques means, repeat no more here.What the present invention emphasized is that daily record data maker adopts the form pushing to realize the collection of daily record data, Data Collection adopts (push) mode that pushes away, and utilize existing daily record framework in measured system, need in system, not implant new run time version, therefore can not increase the development amount of measured application system, reduce the complexity of metering, and very little on the runnability impact of system.
Quantitative analysis controller, for notifying log analyzer to carry out statistical analysis to the daily record data of continuous data container.Such as can regularly saying the word, or adopt Artificial Control continuous data container to carry out statistical analysis.
Described log analyzer also for, before daily record data is analyzed, according to the attribute of daily record data, daily record data is classified.
Quantitative analysis controller is further used for controlling continuous data container and sends to chart display module to show metering result.Chart display module, shows metering result for the graph mode with directly perceived, understandable.
Fig. 2 is the schematic flow sheet of the embodiment of metering method of the present invention, as shown in Figure 2, comprising:
Step 200: daily record data maker is according to the measuring parameter setting in advance, and the daily record data that utilizes daily record adapter that the application system from each node of cloud computing platform is produced, pushes to continuous data container.
Wherein, measuring parameter comprises metering chronomere and value (as second, minute, hour, day etc.), such as metering, chronomere is minute that value is 1 minute, pushes exactly the daily record data that the application system of moving on each node of cloud computing platform produced in 1 minute; Or metering chronomere is hour that value is 24 hours, pushes exactly the daily record data that the application system of moving on each node of cloud computing platform produced in 24 hours; Etc..
Step 201: log analyzer is classified to data according to the attribute of daily record data.
Wherein, the attribute of daily record data include but not limited to daily record data node source, daily record rank, produce date and time, carry out the attributes such as source module.
Step 202: log analyzer, according to the analytical parameters setting in advance, carries out statistical analysis to the daily record data in continuous data container.
Suppose to calculate the value of each metering dimension of daily record data between each unit, as maximum, mean value, minimum value.Give an example, by calculating in 10 minutes, the daily record number of generation per minute, can obtain maximum, mean value and the minimum value of generation daily record number per minute in these 10 minutes.
The result relatively calculating and initial value, each performance index of the application system of each node in acquisition cloud computing platform, as log analysis results such as cpu busy percentage, memory usage, the network bandwidths.
The quantity that produces daily record in unit interval is relevant with cpu busy percentage, and the daily record of generation is more, and cpu busy percentage is higher.
This step also comprises: the data target that the log analysis result that log analyzer produces statistical analysis is collected, as CPU usage, memory usage, the network bandwidth etc., stores in continuous data container.
Step 203: show and also measure result by the log analysis result obtaining after metering method of the present invention with directly perceived, understandable graph mode.The specific implementation of this step belongs to those skilled in the art's conventional techniques means, repeats no more here.
Although the disclosed execution mode of the present invention as above, the execution mode that described content only adopts for ease of understanding the present invention, not in order to limit the present invention.Those of skill in the art under any the present invention; do not departing under the prerequisite of the disclosed spirit and scope of the present invention; can in the form of implementing and details, carry out any amendment and variation; but scope of patent protection of the present invention, still must be as the criterion with the scope that appending claims was defined.
Claims (6)
1. a metering system, is characterized in that, at least comprises daily record adapter, daily record data maker, one or more continuous data containers, log analyzer, and quantitative analysis controller, wherein,
Daily record adapter, for the corresponding relation between storing daily record data and propelling movement target;
Daily record data maker, for according to the corresponding relation of daily record adapter, is pushed to corresponding continuous data container by the daily record data of the application system generation from each node of cloud computing platform;
Continuous data container, the daily record data pushing for storing daily record data maker, and from the log analysis result of log analyzer;
Log analyzer, under the control at quantitative analysis controller, carries out statistical analysis to the daily record data in continuous data container;
Quantitative analysis controller, for notifying log analyzer to carry out statistical analysis to the daily record data of continuous data container.
2. metering system according to claim 1, is characterized in that, described log analyzer also for, before described daily record data is analyzed, according to the attribute of daily record data, described daily record data is classified.
3. metering system according to claim 1 and 2, is characterized in that, also comprises chart display module, for showing daily record data and the log analysis result from described continuous data container;
Described quantitative analysis controller, also exports to chart display module for controlling described continuous data container by the daily record data of storage and log analysis result.
A metering method, it is characterized in that, comprising: daily record data maker is according to the measuring parameter setting in advance, and the daily record data that the application system from each node of cloud computing platform is produced, pushes to continuous data container;
Log analyzer, according to the analytical parameters setting in advance, carries out statistical analysis to the daily record data in continuous data container.
5. metering method according to claim 4, is characterized in that, described daily record data in continuous data container is carried out to statistical analysis before, the method also comprises: according to the attribute of daily record data, described daily record data is classified.
6. according to the metering method described in claim 4 or 5, it is characterized in that, the method also comprises: the log analysis result store that described log analyzer produces statistical analysis is in described continuous data container.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410334475.4A CN104079662A (en) | 2014-07-14 | 2014-07-14 | Measuring system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410334475.4A CN104079662A (en) | 2014-07-14 | 2014-07-14 | Measuring system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104079662A true CN104079662A (en) | 2014-10-01 |
Family
ID=51600702
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410334475.4A Pending CN104079662A (en) | 2014-07-14 | 2014-07-14 | Measuring system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104079662A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109672735A (en) * | 2018-12-20 | 2019-04-23 | 中国移动通信集团江苏有限公司 | A kind of method, device and equipment of Message Processing |
| CN111881161A (en) * | 2020-07-27 | 2020-11-03 | 新华智云科技有限公司 | Index measurement calculation method, system, equipment and storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103581336A (en) * | 2013-11-20 | 2014-02-12 | 中国联合网络通信集团有限公司 | Service flow scheduling method and system based on cloud computing platform |
-
2014
- 2014-07-14 CN CN201410334475.4A patent/CN104079662A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103581336A (en) * | 2013-11-20 | 2014-02-12 | 中国联合网络通信集团有限公司 | Service flow scheduling method and system based on cloud computing platform |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109672735A (en) * | 2018-12-20 | 2019-04-23 | 中国移动通信集团江苏有限公司 | A kind of method, device and equipment of Message Processing |
| CN111881161A (en) * | 2020-07-27 | 2020-11-03 | 新华智云科技有限公司 | Index measurement calculation method, system, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Bird et al. | Update of the Computing Models of the WLCG and the LHC Experiments | |
| US9521052B1 (en) | Methods, systems, and computer readable mediums for utilizing application programming interfaces for accessing key performance indicator information | |
| JP2019511054A (en) | Distributed cluster training method and apparatus | |
| CN111459698A (en) | Database cluster fault self-healing method and device | |
| CN103955509A (en) | Quick search method for massive electric power metering data | |
| CN104407688A (en) | Virtualized cloud platform energy consumption measurement method and system based on tree regression | |
| US10701213B2 (en) | Dynamically generating an aggregation routine | |
| Tribastone et al. | Fluid rewards for a stochastic process algebra | |
| Jones et al. | The evolution of the ATLAS computing model | |
| CN113268403B (en) | Time series analysis and prediction method, device, equipment and storage medium | |
| CN108038036A (en) | Application system performance monitoring method, device, equipment and computer-readable storage medium | |
| CN104079662A (en) | Measuring system and method | |
| CN105279226A (en) | Data monitoring method and equipment based on big data | |
| CN102612058B (en) | Method and device for determining performance index statistical result | |
| CN116166655B (en) | Big data cleaning system | |
| CN110543509A (en) | Monitoring system, method and device for user access data and electronic equipment | |
| CN110750582A (en) | Data processing method, device and system | |
| CN115248782A (en) | Automatic testing method and device and computer equipment | |
| CN105490871B (en) | A kind of method and system for testing Hadoop cluster stability | |
| CN110728838A (en) | Meter reading method and device and power acquisition terminal | |
| CN105119742A (en) | Performance data analog generator and performance data analog generation method based on Java Swing component | |
| CN115242870B (en) | Power consumption data pushing method, device, server and computer readable storage medium | |
| KR20160031807A (en) | The spreadsheet-based big data analysis system and the method | |
| CN109857655A (en) | A kind of method and system detecting fine arts resource dynamic property | |
| CN114282919B (en) | Cloud sea resource charging dynamic cost analysis optimization method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20141001 |
|
| RJ01 | Rejection of invention patent application after publication |