CN101800654B - Instant collection method of network resource accounting information based on event trigger mechanism - Google Patents
Instant collection method of network resource accounting information based on event trigger mechanism Download PDFInfo
- Publication number
- CN101800654B CN101800654B CN2010101264174A CN201010126417A CN101800654B CN 101800654 B CN101800654 B CN 101800654B CN 2010101264174 A CN2010101264174 A CN 2010101264174A CN 201010126417 A CN201010126417 A CN 201010126417A CN 101800654 B CN101800654 B CN 101800654B
- Authority
- CN
- China
- Prior art keywords
- state
- changed
- text
- network resource
- accounting information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Abstract
The invention discloses an instant collection method of network resource accounting information based on an event trigger mechanism. When a thread job Supervisor monitors the operation of finish execution in a trigger state, a network resource use record collector assembly in a network accounting system is activated, an operation inquire command supplied by an operation management system is called, wherein a return value of the command is a text containing resource amount information used in the operation execution process, the text is analyzed, accounting information needed to be collected is matched based on a special regular expression, and the collection of the resource amount used in the operation execution process can be finished; the accounting information is stored in a database and persistently stored; and finally, a state of the operation is converted to a completion state. If text analysis fails, an analysis task is executed again on the test returned by the operation inquire command.
Description
Technical field
The present invention relates to the resource billing technology under a kind of grid environment, particularly a kind of instant collection method of network resource accounting information based on event trigger mechanism.
Background technology
Grid be one on the geographical position, extensively distribute, open distributed system.It is the resource consolidation that disperses by the grid middleware handle on the basis of existing the Internet, realizes sharing and collaborative work of various resources.Along with the development of society, the progress of science and technology, application such as science calculating, enterprise calculation constantly increase the demand of computing capability, and grid is exactly in order to satisfy the infrastructure of this large-scale calculations demand.
For this large-scale grid, all there is every day a large amount of operations to put in the grid and calculates.If can add up and write down the consumption data of these operations effectively to the grid computing resource, not only can help to understand contribution and the consumption situation of grid user, also help simultaneously and formulate relative measures and reduce waste and irrational utilization the grid computing resource to gridding resource.The grid charge system studies how to carry out the system that gridding resource charges just under grid environment.Are piths in the grid charge system and how to add up and write down grid work efficiently to the consumption data of various computational resources.
In the middle of traditional grid charge system, the scheme of taking at the collection of network resource accounting information all is based on the parsing to the charging journal file in the job management system.Therefore all there is a kind of tightly coupled relation in traditional charging information collecting scheme with the charging journal file.There are following two deficiencies in this traditional charging information collecting scheme:
1) time delay of charge information is oversize
Here the time delay of mentioning is meant and is finished from operation that this operation institute consumed resources amount is resolved comes out and deposit this time period of database in.Have much several job management systems of representative and influence power now, produce an operation charging journal file every day, stored the consumption information of performed each operation on the same day in this document resource as systems such as PBS, CONDOR, LSF.And the content in this journal file is that the execution along with each operation is written into gradually.Traditional charging information collecting scheme is exactly to utilize the once measure of this journal file of resolving every day to come collecting charging information.Grid system will receive a large amount of operations in one day, and parsing work just can be carried out after will waiting until this day end, and therefore along with the difference of operation deadline, the charge information of operation may have 24 hours time delay at most nearly.The benefit of doing like this is to avoid load summit, but big like this time delay is intolerable for some grid user.
2) charge information is omitted and is repeated
Traditional charging information collecting scheme is to start a finger daemon, when every day, system was the most idle, such as carrying out the parsing task 0 of morning.Because the charging journal file is with the name of date on the same day, thus date of only containing in the name according to journal file of the daily record task of resolving determine the file that resolve.If the parsing task is made mistakes in the process of implementation, and the parsing task has little time timely recovery, and the charge information of some operations of carrying out on the same day will can not get resolving so, will cause the omission of charge information.If in time recover the parsing task, and the progress of resolving before not having record to make mistakes, it is repetition that the charge information of Partial Jobs will be arranged so again.These mistakes definitely can not take place in resource is chargeed the field that requires the information pin-point accuracy like this.In order to tackle the problem that the parsing task is made mistakes, some charging information collecting schemes adopts carries out method by row labels to the file of resolving, if the parsing task is made mistakes, the this document of lining by line scan after the task of then resolving is in time recovered, begin to resolve from unlabelled row, though can solve the problem that charge information repeats like this, this has increased keeper's the burden and the performance cost of system undoubtedly, this processing procedure also is quite complicated.
Summary of the invention
At the defective that the described traditional charging information collecting scheme of background technology exists, the object of the present invention is to provide a kind ofly realize simply, efficiently based on the instant collection method of network resource accounting information of event trigger mechanism.
For reaching above purpose, the present invention takes following technical scheme to be achieved:
A kind of instant collection method of network resource accounting information based on event trigger mechanism comprises the steps:
(1) monitoring thread jobSupervisor of operation in the grid charge system, this thread is monitored the state of All Jobs in real time.In case monitor the state of operation when being triggering state finishexecution, the gridding resource service recorder collector assembly that then activates immediately in the grid charge system is collected the charge information of this operation;
(2) at first carry out job management system order tracejob, a text that comprises the employed in the process of implementation stock number information of operation is returned in this order;
(3) when this command execution failure, then the state of operation still is finishexecution, waits for that monitoring thread jobSupervisor carries out the collection of step (1) to the employed stock number of this operation again in next monitor procedure;
(4) when this command execution success, then the text that returns in the step (2) is resolved, mate according to the charge information of specific regular expression, and the information after will resolving is stored required collection;
(5) make mistakes when resolving text in the step (4), then the text that again job management system order tracejob is returned is carried out the parsing task;
(6) state of at last step (4) being resolved the back operation is changed to completion, and the charging information collecting of operation so far finishes.
In the such scheme, the triggering state finishexecution of described operation obtains according to following steps:
(1) job management system at first is submitted to operation in the Global Queue, and its state is changed to submitted, waits for scheduling;
(2) state is a be scheduled algorithm process and produced scheduling decision of the operation of submitted, then its state is changed to scheduled, waits for and assigning;
The scheduled operation that the submitted operation of above-mentioned wait scheduling or wait are assigned if do not have to dispatch or assign successfully, then is changed to waiting_g with this two states, waits for next time and dispatching;
(3) state is that the operation of scheduled is dispatched to local operating system, then its state is changed to dispatched, waits for that local operating system starts to carry out;
(4) state is that the operation of dispatched is started successfully by local operating system, then its state is changed to running; Start failure, then its state is changed to waiting_l, requeue and wait for startup;
(5) state is that the operation of running is finished by local operating system, then its state is changed to finishexecution, is the triggering state of waiting for charging information collecting.
Instant collection method of network resource accounting information based on event trigger mechanism of the present invention, removed the close coupling relation of the charging journal file in network resource accounting information and the job management system, made the collection of network resource accounting information and the charging journal file in the job management system irrelevant.In case operation is finished, to collect with regard to charge information immediately operation, this mechanism has reduced the time delay of charge information to a great extent.Simultaneously, utilize this mechanism when collecting the charge information of operation, if the text resolution failure, then the text that again the operation querying command is returned is carried out the parsing task.If the operation querying command is carried out failure, then the state of operation still is the triggering state of finishexecution, waits for that monitoring thread jobSupervisor collects the employed stock number of operation in next monitor procedure again.So promptly get rid of the omission of charge information, can not occur the replication problem of charge information again.This method has realized the collection of network resource accounting information simply, efficiently.
Description of drawings
Fig. 1 is a charging information collecting workflow diagram of the present invention.
Fig. 2 is the follow-up work flow chart of the operation trigger event finishexecution among Fig. 1.
Fig. 3 is actual effect figure of the present invention.
Embodiment
The present invention is described in further detail below in conjunction with accompanying drawing.
As shown in Figure 1, a kind of instant collection method of network resource accounting information based on event trigger mechanism comprises the steps:
(1) monitoring thread jobSupervisor of operation in the grid charge system, this thread is monitored the state of All Jobs in real time.In case monitor the state of operation when being triggering state finishexecution, the gridding resource service recorder collector assembly that then activates immediately in the grid charge system is collected the charge information of this operation;
(2) at first carry out job management system order tracejob, a text that comprises the employed in the process of implementation resource information amount of operation is returned in this order;
(3) if this command execution is failed, then the state of operation still is finishexecution, waits for that monitoring thread jobSupervisor carries out the collection of step (1) to the employed stock number of this operation again in next monitor procedure;
(4) if this command execution success is then resolved the text that returns in the step (2), mate according to the charge information of specific regular expression, and the information after will resolving is stored required collection;
(5) make mistakes when resolving text in the step (4), then return step (4), the text that job management system order tracejob is returned is carried out the parsing task again;
(6) state of at last step (4) being resolved the back operation is changed to completion, and the charging information collecting of operation so far finishes.Like this with regard to charge information omission that has prevented from effectively might exist in traditional resource accounting information collection scheme and the problem that repeats.
As shown in Figure 2, the operation that the present invention is based on event trigger mechanism triggers state finishexecution and obtains according to following steps:
(1) job management system at first is submitted to operation in the Global Queue, and its state is changed to submitted, waits for scheduling;
(2) state be the operation of submitted if be scheduled manager processes and produce scheduling decision, its state is changed to scheduled, wait for and assigning;
Above in (1), (2) step, state is the operation of submitted or scheduled state, if scheduling or assign not success then is changed to waiting_g with corresponding state, waits for scheduling next time;
(3) state be the operation of scheduled if be dispatched to local operating system, its state is changed to dispatched, wait for that local operating system starts to carry out;
(4) state be the operation of dispatched if started successfully by local operating system, then its state is changed to running; If start failure, then its state is changed to waiting_l, requeue and wait for startup;
(5) state be the operation of running if be finished by local operating system, then its state is changed to finishexecution, be the triggering state of waiting for charging information collecting.
Fig. 3 is the test result of the time delay of the resource accounting information of employing the inventive method.The time delay here is defined as: being finished from operation, this operation institute consumed resources amount is resolved comes out and deposits this time interval of database in.Wherein transverse axis is represented job number, and longitudinal axis representative is finished the resolved time interval of coming out and depositing database in of this operation institute consumed resources amount from operation, and unit is ms.As shown in Figure 3: utilize the instant collection method of network resource accounting information based on event trigger mechanism of the present invention, can be shortened to the average delay time of resource accounting information about 1.2 seconds.
Claims (3)
1. the instant collection method of network resource accounting information based on event trigger mechanism is characterized in that, comprises the steps:
(1) monitoring thread jobSupervisor of operation in the grid charge system, this thread is monitored the state of All Jobs in real time, in case monitor the state of operation when being triggering state finishexecution, the gridding resource service recorder collector assembly that then activates immediately in the grid charge system is collected the charge information of this operation;
(2) at first carry out job management system order tracejob, a text that comprises the employed in the process of implementation stock number information of operation is returned in this order;
(3) when this command execution failure, then the state of operation still is finishexecution, waits for that monitoring thread jobSupervisor carries out the collection of step (1) to the employed stock number of this operation again in next monitor procedure;
(4) when this command execution success, then the text that returns in the step (2) is resolved, mate according to the charge information of specific regular expression, and the information after will resolving is stored required collection;
(5) make mistakes when resolving text in the step (4), then the text that again job management system order tracejob is returned is carried out the parsing task;
(6) state of at last step (4) being resolved the back operation is changed to completion, and the charging information collecting of operation so far finishes;
The triggering state finishexecution of described operation obtains according to following steps:
1) job management system at first is submitted to operation in the Global Queue, and its state is changed to submitted, waits for scheduling;
2) state is a be scheduled algorithm process and produced scheduling decision of the operation of submitted, then its state is changed to scheduled, waits for and assigning;
3) state is that the operation of scheduled is dispatched to local operating system, then its state is changed to dispatched, waits for that local operating system starts to carry out;
4) state is that the operation of dispatched is started successfully by local operating system, then its state is changed to running;
5) state is that the operation of running is finished by local operating system, then its state is changed to finishexecution, is the triggering state of waiting for charging information collecting.
2. the instant collection method of network resource accounting information based on event trigger mechanism according to claim 1, it is characterized in that, described step 1), 2) in, state is the operation of submitted or scheduled state, if not success of scheduling or assignment, then corresponding state is changed to waiting_g, waits for scheduling next time.
3. the instant collection method of network resource accounting information based on event trigger mechanism according to claim 1 is characterized in that, state be the operation of dispatched if started successfully by local operating system, then its state is changed to running; If start failure, then its state is changed to waiting_1, requeue and wait for startup.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010101264174A CN101800654B (en) | 2010-03-18 | 2010-03-18 | Instant collection method of network resource accounting information based on event trigger mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010101264174A CN101800654B (en) | 2010-03-18 | 2010-03-18 | Instant collection method of network resource accounting information based on event trigger mechanism |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101800654A CN101800654A (en) | 2010-08-11 |
| CN101800654B true CN101800654B (en) | 2011-11-16 |
Family
ID=42596158
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010101264174A Expired - Fee Related CN101800654B (en) | 2010-03-18 | 2010-03-18 | Instant collection method of network resource accounting information based on event trigger mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101800654B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102496084A (en) * | 2011-11-21 | 2012-06-13 | 南京莱斯信息技术股份有限公司 | Automatic processing method of civil aviation approval message |
| CN109615266B (en) * | 2018-12-26 | 2022-11-04 | 贵州电网有限责任公司 | Text analysis decision method for power grid abnormal information based on data mining |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1744598A (en) * | 2005-09-15 | 2006-03-08 | 上海交通大学 | Grid service allocation system |
| US8578159B2 (en) * | 2006-09-07 | 2013-11-05 | Motorola Solutions, Inc. | Method and apparatus for establishing security association between nodes of an AD HOC wireless network |
-
2010
- 2010-03-18 CN CN2010101264174A patent/CN101800654B/en not_active Expired - Fee Related
Also Published As
| Publication number | Publication date |
|---|---|
| CN101800654A (en) | 2010-08-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ren et al. | Workload characterization on a production hadoop cluster: A case study on taobao | |
| CN100487700C (en) | Data processing method and system of data library | |
| CN101957780B (en) | Resource state information-based grid task scheduling processor and grid task scheduling processing method | |
| Ostermann et al. | Dynamic cloud provisioning for scientific grid workflows | |
| Cheng et al. | Heterogeneity-aware workload placement and migration in distributed sustainable datacenters | |
| CN101692208A (en) | Task scheduling method and task scheduling system for processing real-time traffic information | |
| CN109299180B (en) | ETL operating system of data warehouse | |
| CN106293941A (en) | A kind of distribution method of Database Systems resource | |
| Havet et al. | Genpack: A generational scheduler for cloud data centers | |
| CN104714838A (en) | Task scheduling method and device | |
| CN103716397A (en) | Marching method for service-oriented simulation clock | |
| CN101788908A (en) | Batch data processing method supporting breakpoint reget | |
| Beermann et al. | Implementation of ATLAS Distributed Computing monitoring dashboards using InfluxDB and Grafana | |
| Liu et al. | Temporal QoS management in scientific cloud workflow systems | |
| CN101800654B (en) | Instant collection method of network resource accounting information based on event trigger mechanism | |
| Kang et al. | A SLA driven VM auto-scaling method in hybrid cloud environment | |
| CN103634132A (en) | High-efficiency acquisition system for IT operation and maintenance platform | |
| Du et al. | Dynamic scheduling with process migration | |
| Wang et al. | Scalable run-time correlation engine for monitoring in a cloud computing environment | |
| Li et al. | Fault-aware runtime strategies for high-performance computing | |
| CN112948088A (en) | Cloud workflow intelligent management and scheduling system in cloud computing platform | |
| CN112052284A (en) | Main data management method and system under big data | |
| Grishina et al. | DC energy data measurement and analysis for productivity and waste energy assessment | |
| JP7452668B2 (en) | Task management device, task management method, and task management program | |
| CN114840343A (en) | Task scheduling method and system based on distributed system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20111116 Termination date: 20140318 |