CN100465918C - Automatic configuration of transaction-based performance models - Google Patents
Automatic configuration of transaction-based performance models Download PDFInfo
- Publication number
- CN100465918C CN100465918C CNB2005100885843A CN200510088584A CN100465918C CN 100465918 C CN100465918 C CN 100465918C CN B2005100885843 A CNB2005100885843 A CN B2005100885843A CN 200510088584 A CN200510088584 A CN 200510088584A CN 100465918 C CN100465918 C CN 100465918C
- Authority
- CN
- China
- Prior art keywords
- equipment
- model
- error
- utilization rate
- emulation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Landscapes
- Debugging And Monitoring (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The described systems, methods, and data structures are directed at automatic configuration of transaction-based performance models. Models of an infrastructure are created and automatically configured using data provided by existing management tools that are designed to monitor the infrastructure. These automatically configured models may be used to simulate the performance of the infrastructure in the current configuration or other potential configurations. The models may also be automatically validated calibrated by comparing results from simulations and measured data.
Description
The mutual reference of related application
The application requires in the 60/598th of submission on August 2nd, 2004, No. 569 U.S. Provisional Application No., the latter is entitled as " SYSTEM AND METHOD FOR PROCESSING PERFORMANCEMODELS TO REFLECT ACTUAL COMPUTER SYSTEM DEPLOYMENTSCENARIOS ", and its content is incorporated into this by reference.
The application relates to U.S. Patent application the 09/632nd, No. 521, is entitled as " A PERFORMANCETECHNOLOGY INFRASTRUCTURE FOR MODELING THE PERFORMANCE OFCOMPUTER SYSTEMS ", and its content is incorporated into this by reference.
The application relates to U.S. Patent application the 10/053rd, No. 733, is entitled as " LATE BINDING OFRESOURCE ALLOCATION IN A PERFORMANCE SIMULATIONINFRASTRUCTURE ", and its content is incorporated into this by reference.
The application relates to U.S. Patent application the 10/053rd, No. 731, is entitled as " EVALUATING HARDWAREMODELS HAVING RESOURCE CONTENTION ", and its content is incorporated into this by reference.
The application relates to U.S. Patent application the 10/304th, No. 601, is entitled as " ACTION BASED SERVICESIN A PERFORMANCE SIMULATION INFRASTRUCTURE ", and its content is incorporated into this by reference.
Technical field
The present invention relates to robotization affirmation and collimation technique based on the performance model of issued transaction.
Background technology
The computer system architecture has become one of critical asset of many enterprises.This is especially true for the enterprise that depends on network-based services greatly.In order to ensure steadily and reliably operating, invested ample resources and caught and maintenance calculations machine system infrastructure.Generally speaking, each subsystem of computer system architecture is all monitored by the personal module of this subsystem, such as a performance counter.Can analyze by this subsystem expert's keeper by the data that personal module generates, move reposefully to guarantee this subsystem.
The ability that it catches up with user's request need be improved and expand in the enterprise of a success usually.Ideally, the computer system architecture of this enterprise must be fit to the corporate environment of this variation consistently.In fact, it has spent a large amount of work and special knowledge so that can analyze and assess the performance of existing architecture.For example, if an enterprise anticipates the increase of particular type issued transaction, then need the performance plan to determine how to expand the performance of existing architecture usually to manage this increase.
A kind of mode of execution performance plan is to seek advice to the analyst.Although the working load data can be used for each subsystem, can predict yet the analyst needs the real situation of each system and extensive work for the performance need that improves existing architecture and add or reconfigure which assembly.Owing to, employ an analyst to realize that the performance plan generally is expensive proposal to sizable requirement of the expense and the strength of spending.
The mode of another kind of execution performance plan is to use an available analysis tool to predict the requirement that working load is improved.Yet available many conventional instrument is the program of only inferring from historical data at present, and they are not very accurate or flexible.Equally, can need to make subjective decision selects to provide the assembly of estimating requirement.
Those skilled in the art does not learn the user-friendly instrument that can accurately realize the performance plan as yet.
Summary of the invention
According to a kind of computer implemented method that is used to the architecture modeling that comprises a plurality of equipment of the present invention, described architecture is by a management application monitors, and described computer implemented method comprises: the model of formation base framework; Use and management the information that is provided is provided is disposed described model; Carry out emulation with described model; From emulation, identify performance data; At least partly relatively calculate an error based on what the performance data that records that provided was provided for simulation performance data and management; And if described error is in a threshold value then confirm described model, otherwise comes calibrating patterns with equipment capacity factor as calculated.
According to a kind of computer implemented method that is used to calibrate the model of an architecture of the present invention, described computer implemented method comprises: carry out emulation with basic construction model; Identify the device model that comprises in the described basic construction model, an equipment is associated in described device model and the architecture; The first utilization rate level is determined in the application of carrying out for equipment from emulation; From tolerance, determine the second utilization rate level for described application; By being compared, the first utilization rate level and the second utilization rate level determine a working load correlated error; And at least partly calibrate described device model based on described working load correlated error.
According to a kind of computer implemented method of calibrating the model of an architecture automatically of being used for of the present invention, described computer implemented method comprises: carry out first emulation with first working load; Second emulation with second working load that is different from first working load; Be that an equipment is determined the first emulation utilization rate level from first emulation, described equipment is represented as the device model in the basic construction model; From second emulation, determine the second emulation utilization rate level for described equipment; Determine the first instream factor level for described equipment from tolerance, the described first instream factor level and first working load are associated; Determine the second instream factor level for described equipment from tolerance, the described second instream factor level and second working load are associated; First error that calculating and described first emulation are associated, described first error is represented the poor of the first emulation utilization rate level and the first instream factor level; Second error that calculating and described second emulation are associated, described second error is represented the poor of the second emulation utilization rate level and the second instream factor level; And the working load correlated error of calculating the difference of expression first sum of errors second error; And calibrate described device model with described working load correlated error.
A system in accordance with the present invention comprises: the device that carries out emulation with the model of architecture; Be used to identify the device of the device model of expression in the basic construction model; The device that is used for the emulator utilization rate level of definite described equipment; The device that is used for the physical device utilization rate level of definite described equipment; And at least partly based on the device that relatively calculates an equipment error of described emulator utilization rate level and described physical device utilization rate level; Be used for when described equipment error is in a threshold value, confirming the device of model; And be used for the device that when described equipment error is not in threshold value standby load-factor is as calculated come calibrating patterns.
A system in accordance with the present invention comprises: the device that carries out emulation based on the model of architecture at least partly; Device from the performance counter automatic data collection that is associated with an architecture; By being compared with the data of collecting, simulation result confirms the device of described model automatically; The device of the error between the data that are used to calculate described simulation result and collect; And the device of at least partly calibrating described model automatically based on described error.
Description of drawings
Check following detailed description in conjunction with the accompanying drawings, can understand these and other feature and advantage of the present invention better, in the accompanying drawing
Fig. 1 illustrates and is used for configuration automatically based on the example system of the performance model of issued transaction.
Fig. 2 illustrates the exemplary components of automation modeling module shown in Figure 1.
Fig. 3 illustrates the instantiation procedure of the performance that is used for emulation one architecture.
Fig. 4 illustrates the instantiation procedure that is used for disposing automatically the model of an architecture.
Fig. 5 illustrates and uses automatic configured models to come the instantiation procedure of emulation one architecture.
Fig. 6 illustrates the example calculation machine equipment that is used to realize described system and method.
Fig. 7 illustrates and uses a model of confirming to come the instantiation procedure of emulation architecture performance.
Fig. 8 illustrates the instantiation procedure that is used to confirm basic construction model.
The instantiation procedure that the data of using an application specific counter to be provided are calibrated a device model is provided Fig. 9.
The data that Figure 10 illustrates use to be provided by the repetition emulation of different operating load class are calibrated the instantiation procedure of a device model.
Embodiment
System described here, method and data structure relate to the robotization configuration based on the performance model of issued transaction.Use is designed to monitor that the data that existing management tool provided of architecture are created and the model of configure base framework automatically.These automatic configured models can be used to the performance of emulation architecture in current configuration or other possible configuration.
Robotization performance model configuration-system described below can be effectively and execution performance modeling exactly.This system makes the user carry out each alanysis in mode quick and that the usefulness cost is calculated.For example, described system can be used for carrying out a performance evaluation for current architecture, and current architecture comprises hardware and software component.System can import data from each configuration database and represent the up-to-date of infotech (IT) architecture or deployment in the past.This model configuration can be used as the baseline of analyzing system performance.Analysis type can comprise capacity plan, bottleneck analysis or the like.Capacity plan comprises the request for utilization in future of prognoses system and guarantees that system has the process that enough capacity satisfy these requirements.Bottleneck analysis comprises that analyzing existing system approaches the process of max cap. ground work to determine which assembly in the system.If the raising total system capacity, these are the assemblies that at first will be replaced so.
Described system also can be used for carrying out a what-if.By using baseline model, the user can come the performance of fundamentals of forecasting framework according to one or more variations that configuration is made.The example of what-if scene comprises the change or the like of raising, hardware and/or the software arrangements parameter of working load.
Described system also can be used for the capacity report of robotization.For example, the user can report with the capacity plan that produces robotization for system definition one particular time interval.When this time interval past tense, system imports the data of last reporting period, and disposes described model automatically.Then, system uses the model that is configured to carry out emulation, and is the capacity generation report in future of system.If power system capacity is not enough to carry out reporting period next time, then system can produce a warning.
Described system can be used for operational fault diagnosis.For example, an operations management application can be notified the IT keeper: exceeded performance threshold.The keeper can use described system to represent the current configuration of system.Then, the keeper can carry out an emulation whether identify performance alert be the cause of capacity problem.Particularly, the keeper can determine that performance alert is that intrinsic capacity limit by system is caused or caused by other factors, such as other application that is moved in system by other user.
Fig. 1 illustrates and is used for configuration automatically based on the example system of the performance model of issued transaction.In one implementation, example system can comprise automation model configuration module 100 and emulation module 130, and they are described with module separately in Fig. 1 for purposes of illustration.In reality realized, automation model configuration module 100 and emulation module 130 can be combined into single component.Example system is configured to modeling architecture 110, and modeling event and issued transaction, is used in the performance of emulation architecture 110 in each configuration.
Data-carrier store 123 is configured to preserve the data that administration module 120 is provided.Data can be organized as any data structure, such as one or more operating databases, data warehouse or the like.Data-carrier store 123 can comprise physics or the relevant data of logical topology with architecture 110.Data-carrier store 123 also can comprise and working load, issued transaction workflow or the relevant data of action cost.This data can be presented as the form of the track that the event tracking technology is generated, described event tracking technology such as
(ETW) event tracking (Event Tracing) or the SQL of Microsoft follow the tracks of.
Automation model configuration module 100 is configured to obtain relevant for the information of architecture 110 and is used for emulation for each building component of architecture 110 and allocation models 103 automatically.Model 103 serves as the input of emulation module 130.
Automation model configuration module 100 can be mutual with architecture 100, and carry out network discovery to search the data that are used for component model.Yet automation model configuration module 100 generally is configured to obtain data from operating database and data warehouse, and described operating database or data warehouse are preserved by the collected information of the Management Unit of architecture 100.For example, automation model configuration module 100 can be from data-carrier store 123 retrieve data, and data-carrier store 123 comprises the data that administration module 120 is provided.
Automation model configuration module 100 can provide arbitrary class model to be used to be input to emulation module 130.In one embodiment, automation model is configured to architecture 110 generations and physical topology, logical topology, working load, issued transaction workflow and the relevant model of action cost.
The data that are used for the physical topology of modeling architecture 110 can comprise and to be comprised by the hardware list of emulation how the performance of each assembly and assembly interconnect.Usually select the exact level and the degree of the easy obtained performance data of energy to be complementary.For example, Microsoft
Operating system can be expressed performance data by the usability counter.These counters generally are exemplified as CPU, network interface unit and disk drive.Automation model configuration module 100 can be by being system representation independent CPU, network interface unit and disk drive in physical topology is described, thus this system of modeling.Each component type has the hardware model of a coupling, and the hardware model of this coupling is used to calculate the incident institute's time spent on this assembly.Therefore, the cpu package type is represented that by the CPU hardware module CPU hardware module has been calculated CPU action institute's time spent, such as calculating.
Automation model configuration module 100 can use extend markup language (XML) form of a classification that hardware information is encoded, and server table is shown the container (container) of the equipment that server comprised physically.Assembly can be described with a template, and this template can be encoded to the performance of this assembly.For example, " PIII Xeon700MHz " template is encoded to performance and ability with intel pentium three (Intel PIII) the Xeon CPU of the clock speed of 700MHz operation.After assembly had been named with this hierarchical approaches and described, physical topology was described the network linking that can also comprise inter-module.Physical topology is described and can be expressed as the right tabulation of assembly name, tags with the attribute of corresponding network.When having a more than network interface unit (NIC) in server, the specific NIC that is using also can be designated.Be a code sample relevant below with the physical topology modeling.
<active_device?name＝＂WebSrv?1＂count＝＂1＂>
<！--Compaq?DL-580-->
<active_device?name＝＂cpu＂count＝＂4＂>
<rct?name＝＂cpu＂/>
<use_template?name＝＂Cpu:PIII?Xeon?700
MHz＂/>
</active_device>
</active_device>
The data modeling of architecture 110 logical topologys can comprise component software (or service) tabulation of the application that is modeled, and how assembly is mapped to the description of hardware described in the physical topology.The part that the component software tabulation can be used as application model provides.For example, the application model of an e-commerce website may comprise the application component of representing a web server, such as Microsoft
The internet information service, and comprise another nextport hardware component NextPort of representing database server, such as Microsoft
Sql server.The description of each application component can comprise that application component is in order to move required hardware action.
The logic of application component to the hardware can be expressed with the description that moves the server list of application component (describing with physical topology) separately and how to carry out load balance on server to the mapping of physics.Notice that this is unnecessary to be man-to-man mapping.Single application component can expand on the multiple servers, and individual server can have the several application assembly in the place.Below be the code sample relevant with the logical topology modeling.
<service?name＝＂IIS＂policy＝＂roundrobin＂>
<serverlist>
<server?name＝＂WebSrv1＂/>
<server?name＝＂WebSrv2＂/>
<server?name＝＂WebSrv3＂/>
</serve?rlist>
<actionscheduling>
<schedule?action＝＂Compute＂
policy＝＂freerandom＂>
<target?device＝＂cpu＂/>
</schedule>
</actionscheduling>
</service>
The data that are used for modeling architecture 110 working loads can comprise the tabulation that name/value is right, have defined influence by the digital parameters of the performance of analogue system.For example, above-mentioned e-commerce website may comprise a plurality of concurrent users parameter, they carry out used frequency of different issued transaction or the like.Below be the code sample relevant with the working load modeling.
<pardef>
<parameter?varname＝＂Alerts?TPS＂descr＝＂Alerts
transactions?per?second＂type＝＂float＂
value＝＂203.＂/>
<parameter?varname＝＂Log?TPS＂descr＝＂Logging
transactions?per?second＂type＝＂float＂
value＝＂85.5＂/>
</pardef>
In one implementation, automation model configuration module 100 is configured to: the available datas in the data-carrier store 123 that is provided with administration module 120 are come the model of automatic configure base framework 110.For example, automation model configuration module 100 automatically configures physical topology, application component on from the logical topology to the physical hardware logical mappings and from the parameter value of working load.Generally speaking, automation model configuration module 100 can be model creation to describe the template of hardware or software with generic term at first.Then, every instantiation that the described model of automation model configuration module 110 configuration is modeled with reflection, such as hardware model how to connect, how software model disposes or use or the like.
Can be used on arbitrary IT infrastructure with reference to the described system of Fig. 1 above.For example, the data center (datacenter) that the typical IT of enterprise environment has a plurality of geographical convergent-divergents has organized the hundreds of station server in the network of complexity.The user is difficult to manually catch the configuration of this environment usually.Generally speaking, require the only little chap collection of its environment of modeling of user.Even in this case, modeling process all is labor-intensive.It is available for user base widely that described system makes based on the performance modeling of the emulation of incident.System can dispose performance model from the existing information that business management software obtains automatically by using.
By the configuration of robotization and simplified model, described system makes the user can execution performance plan under multiple environment.For example, allocation models is representing current deployment fast by making the user, and system can create weekly the user or the capacity report of every day, even in fast-changing environment is arranged.An architecture was managed on the property taken the photograph ground before frequent capacity report made IT expert's energy, such as expecting before performance issue takes place and correcting them.
Said system also make the user can be easily to more most modeling of a tissue, so that analyze the performance factor of relative broad range.For example, mail server is disposed can influence a plurality of data centers.If the relevant configuration data can be used, then the model of the existing architecture of mail server can dispose automatically, model can be used to predict the stand-by period of end to end issued transaction, for example determines from the stand-by period of Asia office to U.S. general headquarters send Email.Another example benefit of this analysis is to calculate because the operating position that the mail telephone traffic of Asia/U.S.'s wide-area network link causes.
Use the performance evaluation of described system also can be used for fault diagnosis is carried out in the operation at data center.For example, the operational administrative software as MOM can send a warning relevant with long response time on the mail server.IT expert can use this system to come automatic allocation list to show the performance of model, emulation expection of system's current state and problem identificatioin since capacity problem or owing to some other former thereby cause.
Fig. 2 illustrates the exemplary components of automation modeling module 100 shown in Figure 1.As shown in Figure 2, automation modeling module 100 can comprise physical topology MBM 201, logical topology MBM 202 and event analysis module 203.Module 201-203 only illustrates for the purpose of illustration.In reality realized, module 201-203 generally was integrated in the assembly.
The data of being searched generally comprise the information of the module that is used for the formation base framework, such as server list and the nextport hardware component NextPort that they comprised and the physical topology (for example interconnection between server) of network.Physical topology module 201 can also be configured to the data-switching of being searched is become a kind of form that is used for creating the available model of emulation.For example, the data of being searched can be converted into the XML form.Physical topology module 201 also can be configured to the external information of filtering.For example, even can comprising the memory size of architecture assembly, the data of being searched generally are not the direct modeling memory size of emulation.Physical topology module 201 can also be configured to " semantic extension " of carrying out institute's retrieve data.For example, physical topology module 201 can convert to the title of a disc driver (simply string representation) have disk size, the suitable template of access time, rotating speed equivalence.Physical topology module 201 can be configured to use from difference the data of conversion all kinds form.
Working load MBM 203 is configured to from user job load derived parameter value.Generally speaking, these values are by deriving the data of searching from administration module 210.The data of being searched can comprise the relevant current or historical information of working load that experiences with the one or more application that is monitored.Typical performance counter can comprise concurrent user's number, number of requested different transaction type or the like.Can carry out a switch process, convert the form that can realistic model, use to, and carry out the mathematics conversion where necessary from the basic format of institute's retrieve data.For example, the OM database may be recorded in the individual number of the dissimilar issued transaction of request in one hour time, and model can this same information representation be in one hour the issued transaction sum, add the number percent of the issued transaction that each is dissimilar.
Fig. 3 illustrates the instantiation procedure 300 that is used for emulation architecture performance.At square frame 301, identified the topological sum performance data that is associated with an architecture.The data that identified can be used by one or more management of architecture and provide.Data can be used by management and directly provide, and perhaps provide by operating database or data warehouse.
At square frame 303, the data that processing is identified are to obtain the input of basic construction model.For example, topological data can be converted into the form that can be used by a MBM or an emulation module, such as the XML form.Performance data can be converted into the form of being convenient to represent working load.
At square frame 305, basic construction model disposes automatically with the modeling input.The instantiation procedure that is used for automatic configure base framework model will be discussed in Fig. 4.In brief, this model uses the available data of using from management to dispose, such as with physical topology, logical topology, working load, issued transaction workflow, action cost or the like relevant data.
At square frame 307, carry out one or more emulation based on model.Emulation comes modeling event and action to carry out by physics or the logic module model with architecture.Emulation is carried out on the current configuration of architecture or possible configuration.Will in Fig. 5, discuss and use automatic configured models to come the instantiation procedure of emulation one architecture.At square frame 309, the result of output emulation.
Fig. 4 illustrates the instantiation procedure 400 of the model that is used for automatic configure base framework.Process 400 can be realized by the automation model configuration module 100 shown in Fig. 1 and 2.At square frame 401, nextport hardware component NextPort is used the physical topology that is provided with the management of architecture and is disposed.The physical topology data can comprise the hardware configuration of equipment of architecture and the assembly of these equipment.The physical topology data also can comprise the information how equipment connects.
At square frame 403, software model can be used the logical topology data that provided from the management of architecture and determine.The logical topology data can comprise the component software on the equipment with foundation structure and the relevant information of configuration of component software.At square frame 405, software model is mapped as hardware model.
At square frame 407, from using, the management of architecture determines working load data, issued transaction workflow data and action cost.Particularly, the data time and the working load that can define the incident carried out by hardware and software component and action and be associated with these incidents and action.At square frame 409, data are integrated in the model.For example, can configuration software and the performance of hardware model reflection model when carrying out defined incident and action.
Fig. 5 illustrates and uses automatic configured models to come the instantiation procedure 500 of emulation one infrastructure.Process 500 can be realized by emulation module shown in Figure 1 130.At square frame 501, receive the instruction of carrying out emulation.Instruction can comprise and how carry out the relevant information of emulation.For example, instruction can be carried out and will carry out emulation with the existing configuration of architecture or once the configuration of revising.Instruction can be specified the working load of emulation, such as the different operating load of work at present load that uses architecture or the one or more assemblies of architecture.
At square frame 503, determine the model of existing architecture.Generally speaking, model provides by MBM, and is automatically configured the current state with the reflection architecture.At decision frame 505, determine whether to change the configuration of basic construction model.Before reality realize to change, can carry out configuration change the emulation of architecture come the estimated performance influence.If there is not configuration variation, process 500 just moves to square frame 513.
Fig. 6 illustrates the example calculation machine equipment 600 that is used to realize described system and method.In its most basic configuration, computing equipment 600 generally comprises at least one CPU (central processing unit) (CPU) 605 and internal memory 610.
According to the actual disposition and the type of computing equipment, internal memory 610 can be (such as the RAM) of volatibility, non-volatile (such as ROM, flash memory or the like) or both a certain combinations.In addition, computing equipment 600 also can have additional feature/function.For example, computing equipment 600 can comprise a plurality of CPU.Described method can be carried out in arbitrary mode by the arbitrary processing unit in the computing equipment 600.For example, described process can be by a plurality of CPU executed in parallel.
Computing equipment 600 also can comprise additional storer (removable and/or not removable), includes but not limited to: magnetic or optical disc or tape.This annex memory is used storer 615 explanations in Fig. 6.Computer storage media comprises volatibility and non-volatile, the removable and immovable media of realizing with arbitrary method that is used for canned data or technology, and described information is such as computer-readable instruction, data structure, program module or other data.Internal memory 610 and storer 615 all are the examples of computer storage media.Computer storage media includes but not limited to: RAM, ROM, EEPROM, flash memory or other memory techniques, CD-ROM, Digital video disc (DVD) or other optical memory, magnetic tape, tape, magnetic disk memory or other magnetic storage apparatus or can be used to preserve expectation information and can be by any other media of computing equipment 600 accesses.Any such computer storage media can be the part of computing equipment 600.
Computing equipment 600 also can comprise the communication facilities 640 that equipment can be communicated by letter with miscellaneous equipment.Communication facilities 640 is examples of telecommunication media.Telecommunication media generally embodies computer-readable instruction, data structure, program module or other data in modulated data signal, described modulated data signal is such as carrier wave or other transmission mechanism, and comprises that any information transmits media.Term " modulated data signal " means the signal that its one or more features are provided with or change in the mode to signal internal information coding.For example but unrestricted, telecommunication media comprises as cable network and wired media straight line is connected and the wireless medium as acoustics, RF, infrared or other wireless medium.Term used herein " computer readable medium " comprises computer storage media and telecommunication media.Described method can be encoded with arbitrary computer readable medium of arbitrary form, such as data, computer executable instructions or the like.
Computing equipment 600 also can have input equipment 635, such as keyboard, mouse, pen, voice-input device, touch input device or the like.Also can comprise the output device 630 as display, loudspeaker, printer etc.All these equipment all are well known in the art, and need not go through.
As mentioned above, described system, method and data structure can be used the data used from available management configure base framework model automatically.By having strengthened these systems, method and data structure in conjunction with a robotization affirmation and alignment features.Model can be identified and be calibrated to user-selected order of accuarcy.
After having disposed the model of architecture automatically, can carry out the performance prediction of guaranteeing this model is accurately in the degree that the user carries out.If do not reach the degree of accuracy of appointment, then can carry out calibration and revise the not configurable aspect of model, so that reach the accuracy of appointment.The configurable aspect of model, such as the expression of hardware, topology, working load or the like, generally be not calibrated change.Calibration can change the parameter that is associated with model, such as action cost, background load or other parameter as a model template part.
The action cost is the numerical value of the resource requirement of a particular transaction treatment step on the expression one specific hardware resource.The action cost can use for employed hardware device type specific means measure.Generally speaking, the action cost is independent of the particular instance of equipment.For example, the action cost of CPU can be measured with the megacycle that calculates, and the action cost of disk can be measured with required disk the number of transmissions and institute's data quantity transmitted.Different CPU and disk may adopt the simulation time of varying number to come the action of processing requirements same action cost.The action cost generally obtains between the development stage of basic construction model, by this should be used for obtaining in the indoor modeling of performance test.
Ideally, whole action costs of a particular device type (for example CPU) can be described with single numerical value (for example megacycle), can be on whole examples of this device type convergent-divergent exactly.In the practice, convergent-divergent may be remarkable.For example, moving identical action with the twice clock speed on CPU may not can cause finishing this action with half the time.It is normally unpractiaca to remedy the whole factors that influence this non-linear convergent-divergent.Even the very complicated model that can accurately remedy whole possible factors is provided, also because multiple former thereby still can not use this model.For example, calculating required time of net result and/or internal memory may be than required much higher of naive model, causes high surprising long simulation time.Equally, for simple data aggregation and model configuration, required input variable number may be excessive.May not expect to spend a lot of time and efforts to come Test Application and hardware.
In order to alleviate the difficult trade-off problem between model accuracy and the complexity, can obtain both benefits with calibration, for example for the input of wide region, can use one simply, model fast with the minimum accuracy of appointment.Can realize confirming to determine whether the modeling accuracy is abundant.Can realize calibrating and regulate the action cost, so that reflect employed specific input set preferably.
The background load is another variable that runs into usually in practice, but does not generally realize in conventional model.The background load be meant by be not a working load model part should be used for using hardware resource.For example, the virus examination program can make each disk that reads need extra CPU expense, so that whether the content in the scanning disk has virus signature.Local Area Network is another example, because LAN seldom acts on single application.More commonly, LAN shares at many intercomputers of a plurality of application of operation, and each computing machine all has the influence of himself to network.Sometimes, the user may learn this background load, and can comprise the part of this load as the initial model configuration, for example by specifying the fixedly utilization rate number percent of LAN.As if yet more commonly, the user does not know these additional effect, only inaccurate up to performance model.
In addition, some background load effects may not be constant, but depend on working load.The virus examination program is an example.Usually, disk operating is independent of CPU and modeling.One " CPU cost " field may be provided in the disk model.The effect of virus examination program look be to the increase of whole issued transaction of comprising disk access action the CPU cost.
In order to confirm the accuracy of a performance model, can catch the performance that is modeled application.Performance data can be caught with statistical counter, the aspect of performance that statistical counter tolerance is used and used the hardware device of carrying out thereon.For example, can use Microsoft
" performance counter " announced.Other example comprises that by application program be hardware tolerance (for example employed CPU time amount of CPU) sum counter that metric performance is created, and described performance is such as the average transaction processing speed.
Generally developed model and come the part of usability counter tolerance as the model configuration information.Level of abstraction that can preference pattern comes the availability of matching performance information.The output of model also can be represented with these performance counters.For example, how many CPU time output has been used and has been used the average transaction processing speed that is continued during can being included in a series of emulation issued transaction on specific CPU.
As mentioned above, during the robotization configuration, can import the information relevant from the OM database with the application that is modeled.The example of this database is the database that is kept by Microsoft's Action Manager (MOM), comprises the history value of the performance counter that is modeled application.These counters can be caught input service load (for example handled issued transaction number), can catch observed result (for example CPU time that is consumed) again.
Affirmation can comprise: adopt automatic configured models; Model input be set to the historical perspective that obtains from the OM database to performance counter value (issued transaction number for example hourly); Move a performance simulation; And institute's prediction result and historical perspective to performance counter value (for example CPU time that is consumed) compare.For a performance counter value that predicts, the accuracy of performance model not only can be with relative mode (being number percent), but also can represent with absolute mode (as the megacycle number).Required accuracy can be represented with any of this dual mode.In addition, performance counter can divide in groups.Required accuracy can be applied to this group generally.For example, user's portion's disk bandwidth prediction of demanding perfection possibly is accurate in 20%, and perhaps the whole CPU megacycles predictions on the front end web server are accurate in 5%.
Performance counter can be organized into two classes based on the scope of counter.Some counters are applied to an application-specific.For example, application can apply the cpu usage that is caused by using for mail server.These counters can be defined as the application specific counter.Operating system (OS) also is responsible for the overall performance of surveillance and is applied counter, such as total cpu usage.The counter of these system scopes can comprise whole application that use is carried out in system.When having error in model, these counters can be used for determining the source of error.Error can be characterized by working load correlated error and the irrelevant error of working load.
The working load correlated error comprises such error: its size changes according to the function of application workloads.For example, the working load correlated error may stem from modeling assumption, startup effect (for example cold buffer memory), the application saturated (as locking) of mistake, the issued transaction classification of losing or the like.The issued transaction classification right and wrong of losing usually see, because in general, and the whole issued transaction of supporting of the only the most frequently used issued transaction of modeling, rather than modeling.The effect of working load correlated error can be by comparing application specific counter and modeling result to determine.For example, if the expection CPU usage that mail server is used is 10%, and the actual CPU usage of using is 15%, and then this difference of 5% is exactly the working load correlated error.
The irrelevant error of working load comprises that its size is independent of the error of working load.The irrelevant error of working load generally stems from other working load that does not comprise and the expense of coming from OS or model.For example, individual server equipment may not only move a mail server and uses, but also moves a file server and use.The mail server application model may not can remedy by file server uses caused equipment operating position.The effect of the irrelevant error of working load can be calculated by the counter of system scope and application specific counter are compared.For example, total and CPU usage is 35% if the CPU usage that mail server is used is 25%, then this difference of 10% is exactly owing to working load that constant load or background load the cause error that has nothing to do.
The part of the default value of required accuracy limitations as basic model can be provided.For example, if it has been found in practice that disk model is accurate especially, then can be default required accuracy equipment 5%, because this extraneous value may be the result of the Fundamentals hidden more, such as the background load.On the contrary, if known CPU model is more inaccurate, then default required accuracy can be made as 20% to avoid drawing inaccurate conclusion from the result.
Can show the accuracy grouping with simplification information and reduce user load.For example, confirm that user interface can illustrate the independent expression of front end web server with an accuracy scope (for example-6% to+7%).Rather than all accuracys of front end web servers are shown in the data center.Coloud coding can further strengthen the availability at interface.For example, the performance counter that accuracy drops in user's specified limit can show that the counter of approaching restriction can be with orange demonstration with green, and the counter that exceeds restriction is with red display.
If the user pleases oneself with respect to forecasting accuracy to the observed accuracy of over-all properties counter, then affirmation process is finished, and the user may use this model to carry out what-if, and is more confident for net result.Otherwise, can after confirming, carry out one or more calibration cycles.
Calibration comprises that the action cost of regulating the key property model or background load are to improve the accuracy of model validation.If inaccurate fundamental cause depends on working load (being the working load correlated error), then regulate and move the effect that successfully may produce expectation.If fundamental cause is independent of working load, for example Another Application is being used the number percent of LAN bandwidth, and then regulating the action cost can all cause inaccurate result for all working load class except a rank of selecting for use for affirmation.
The load of adjusting background can improve the accuracy of model validation by the notion that comprises the load of working load background context.The background load can be a constant, or an and scalar multiplying each other of work at present load.The background load can be used according to each equipment, rather than uses according to each action.Yet, underestimate the situation of application performance for Capturing Models, can expand the background load to comprise a negative load (, making it be higher than its right capacity) promptly based on model conditioning equipment capacity.Negative load can be used to the equipment that remedies than the model result situation of convergent-divergent better.
The notion of background load is used in the resource capacity of employed basic hardware model in the emulation.The background load can be constant (is working load irrelevant error), or working load is relevant, and serves as positive or negative factor.Regulate the used correct number of background load and depend on basic model.If model is linear, be sufficient then with correcting multiplied together.Yet, may need unique calculating to determine suitable correction factor than complicated model.According to default accuracy value, these calculating are provided as the calibration function in the hardware model.This calibration function can call for each device type with observed inaccuracy.Calibration function can return suitable factor or constant amount, changes resources costs with this, is zero so that inaccuracy is made as.
After observing the inaccuracy error, execution analysis partly is because the constant effect with which that determine inaccuracy, and which partly is because the working load correlation effect.This is determined and can make by the result of relatively twice emulation.Determine also can make by the result of the result of application specific counter and system scope performance counter is compared.
The inaccuracy assessment that emulation is carried out comprises with two different working load values carries out twice emulation, and determines that the inaccuracy of this twice emulation is to keep identical or variation.Can use any working load of second emulation to change, such as half or twice of last working load.Working load is doubled to cause nonlinear performance effect, such as approaching saturated independent component.For example, the performance of total system may become index, even behavior is normally linear.Hide, half that use working load in second emulation may provide result preferably in many cases.Yet half working load in second emulation may not expect when initialization load is very low, because model is near the fineness (level of granularity) of performance counter, and the performance effect may be lost in noise.Therefore, use the calibration of this solution to comprise:
A) move emulation second period with different working load intensity (as half of working load);
B) each hardware device that is modeled of calibrating for needs:
I) be the performance counter of the observed performance counter of first and second emulation and prediction to use a constant background load or a variable background load with definite equipment;
II) call suitable hardware model calibration function, provide constant or variable background load error and
Obtain corresponding constant or variable background load factor;
III) to basic equipment application load factor.
The inaccuracy assessment that emulation is carried out can be expressed as follows:
Wherein I represents load, and e represents total error, e
cThe expression constant error, e
vThe variable error that expression causes owing to load, u
pThe capacity utilization of expression prediction, u
mThe capacity utilization that expression records.
In above formula, u
m, u
pWith I be known.The single system that has caused having 2 formula of 2 unknown numbers with two kinds of load running emulation.Therefore, e
vAnd e
cDetermine easily.
The inaccuracy assessment that the usability counter carries out generally requires application specific counter and the paired availability of system scope performance counter, has characterized the utilization rate level of same equipment.Calibration can followingly be carried out:
A) determine that error is owing to the background load causes (for example Yu Ce utilization rate counter deducts the system scope counter).The result is the constant background load that is applied to equipment.
B) determine the error (for example Yu Ce utilization rate counter deducts the application specific counter) that working load is relevant.The result is that the function according to load comes the application background load.
C) to the load factor of basic equipment application combination.
After finishing calibration steps, can confirm once more.
Fig. 7 illustrates and uses one to confirm that model comes the instantiation procedure 700 of emulation architecture performance.Process 700 is similar to process shown in Figure 3 300, but comprises the additional step behind the square frame 307.
At decision frame 703, determine whether to carry out the affirmation of automatic allocation models.If not, process 700 just proceeds to square frame 309.If will carry out affirmation, then process 700 moves to square frame 707, confirms this model there.Will be used to confirm the instantiation procedure of model in conjunction with Fig. 8 discussion.This process then moves to square frame 707, exports simulation result there.
Fig. 8 illustrates the instantiation procedure 800 of the model that is used to confirm architecture.At square frame 803, sign is from the result of emulation.At square frame 805, determine the working load data from measuring.The working load data that record offer architecture by administration module.At square frame 807, simulation result is compared with the working load data that record.From comparison, calculate error.At decision frame 809, determine that error is whether in acceptable level.If process 800 just moves to square frame 815, confirm model there.
At square frame 813, calibrate this model with load factor.For example, model can be configured to and remedy during the emulation as the irrelevant error of the working load of constant background load, and comes convergent-divergent working load correlated error based on workload level.At square frame 815, after calibration, confirm model.Should be appreciated that the step in the square frame 809,811 and 813 can repeat, till error is in acceptable level.
The instantiation procedure 900 that the data of using the application specific counter to be provided are calibrated a device model is provided Fig. 9.At square frame 903, sign one utilization rate value from emulation.At square frame 907, the data that counter provided of using system scope are determined total error.For example, determine total error by the utilization rate value that from the utilization rate value of emulator, deducts the system scope counter and provided.Total error can be represented a background load, and this background load comprises a working load correlated components (application load that for example is not modeled) and a working load independent component load that OS generated of equipment (for example by).This background load has produced error, because do not remedied by model during loading on emulation.
At square frame 909, the data of using the application specific counter to be provided are determined the working load correlated error.The application specific counter is determined the utilization rate of application.The working load correlated error can be determined from the emulation that is associated with application and the difference of instream factor value.Remaining total error is the constant error that is independent of working load.At square frame 911, from constant error and working load correlated error, calculate the load factor that is used to calibrate.
The data that Figure 10 illustrates use to be provided with the repetition emulation of different operating load level are calibrated the instantiation procedure 1000 of a device model.At square frame 1005, sign is from the utilization rate value that records of two workload level.At square frame 1007, determine the emulation utilization rate value of two workload level.At square frame 1009, calculate the total error of two workload level.For example, total error can be calculated by deduct simulation result from the data that record.Total error is represented the background load that do not remedied by model.
At square frame 1015, come the computational workload correlated error by the total error that compares two workload level.For example, if total error is different under two workload level, then this difference represents that error depends on working load.All the other errors are independent of working load.At square frame 1017, from the irrelevant sum of errors working load correlated error of working load, determine a load factor.
For affirmation and the calibration that realizes automatic allocation models, the automation modeling module 100 shown in Fig. 1 and 2 can be configured to realize process 800,900 and 1000 discussed above.
Though illustrated and described the preferred embodiments of the present invention, be appreciated that here and can make various variations, and do not deviate from the spirit and scope of the present invention.
Claims (27)
1. computer implemented method that is used to the architecture modeling that comprises a plurality of equipment, described architecture is by a management application monitors, and described computer implemented method comprises:
The model of formation base framework;
Use and management the information that is provided is provided is disposed described model;
Carry out emulation with described model;
From emulation, identify performance data;
At least partly relatively calculate an error based on what the performance data that records that provided was provided for simulation performance data and management; And
If described error in a threshold value then confirm described model, otherwise is come calibrating patterns with equipment capacity factor as calculated.
2. computer implemented method as claimed in claim 1 is characterized in that, the step of calibrating patterns comprises:
For the equipment in the architecture is determined an emulation utilization rate value;
From determining the utilization rate value that records of described equipment with described equipment system associated counter;
By emulation utilization rate value and the utilization rate value that records are compared to determine a total equipment error; And
Calibrate described model with described total equipment error.
3. computer implemented method as claimed in claim 2 is characterized in that, calibrate described model by regulate with described equipment associated action cost or background load at least one carry out.
4. computer implemented method as claimed in claim 1 is characterized in that, the step of calibrating patterns comprises:
Determine the application utilization rate value that records from an application specific counter, described application specific counter and the application of carrying out on the equipment in architecture are associated;
For a Simulation Application utilization rate value is determined in described application;
At least partly relatively come to determine a working load correlated error based on the application utilization rate value of application utilization rate value that records and emulation; And
At least partly calibrate described model based on described working load correlated error.
5. computer implemented method as claimed in claim 4 is characterized in that also comprising:
For described equipment is determined an emulation utilization rate value;
From determining the utilization rate value that records of equipment with described equipment system associated counter;
At least partly relatively come to determine a total equipment error based on emulation utilization rate value and the utilization rate value that records;
From described total equipment error and described working load correlated error, determine a constant error; And
At least partly calibrate described model based on described constant error.
6. computer implemented method as claimed in claim 1 is characterized in that, the step of calibrating patterns comprises:
From emulation, determine the first emulation utilization rate value for an equipment in the architecture;
From with described equipment system associated counter determine the utilization rate value that records of equipment; And
By being compared with the utilization rate value that records, the first emulation utilization rate value determines first equipment error.
7. computer implemented method as claimed in claim 6 is characterized in that also comprising:
Described model is carried out second emulation;
From second emulation, determine the second emulation utilization rate value of described equipment;
By being compared with the utilization rate value that records, the second emulation utilization rate value determines second equipment error.
8. computer implemented method as claimed in claim 7 is characterized in that also comprising:
By described first equipment error and described second equipment error are compared to determine the irrelevant sum of errors working load correlated error of the working load that is associated with described equipment; And
At least partly calibrate described model based on the irrelevant described working load correlated error of sum of errors of described working load.
9. computer implemented method as claimed in claim 8 is characterized in that, the irrelevant error of described working load is at least partly determined based on the difference of first equipment error and second equipment error.
10. computer implemented method as claimed in claim 8 is characterized in that, calibrate described model by regulate with described equipment associated action cost or background load at least one carry out.
11. be configured to a kind of device that enforcement of rights requires 1 described computer implemented method.
12. a computer implemented method that is used to calibrate the model of an architecture, described computer implemented method comprises:
Carry out emulation with basic construction model;
Identify the device model that comprises in the described basic construction model, an equipment is associated in described device model and the architecture;
The first utilization rate level is determined in the application of carrying out for equipment from emulation;
From tolerance, determine the second utilization rate level for described application;
By being compared, the first utilization rate level and the second utilization rate level determine a working load correlated error; And
At least partly calibrate described device model based on described working load correlated error.
13. computer implemented method as claimed in claim 12 is characterized in that, at least partly calibrates described device model based on described working load correlated error and comprises: use with the factor of working load convergent-divergent and correct described device model.
14. computer implemented method as claimed in claim 12 is characterized in that, the described second utilization rate level is determined from the application specific counter that is associated with described application.
15. computer implemented method as claimed in claim 12 is characterized in that also comprising:
From emulation, determine the 3rd utilization rate level for described equipment;
From tolerance, determine the 4th utilization rate level for described equipment;
By being compared, described the 3rd utilization rate level and described the 4th utilization rate level determine an equipment error;
From described equipment error and described working load correlated error, determine a constant error; And
At least partly calibrate described device model based on described constant error.
16. computer implemented method as claimed in claim 15 is characterized in that, calibrates described device model to small part based on described constant error and comprises: correct described device model with the constant factor that does not change with working load.
17. computer implemented method as claimed in claim 15 is characterized in that also comprising:
Confirm each device model in the architecture; And
For each equipment that is not identified identifies, determines and calibration steps.
18. be configured to a kind of device that enforcement of rights requires 12 described computer implemented methods.
19. one kind is used for the computer implemented method of calibrating the model of an architecture automatically, described computer implemented method comprises:
Execution has first emulation of first working load;
Second emulation with second working load that is different from first working load;
Be that an equipment is determined the first emulation utilization rate level from first emulation, described equipment is represented as the device model in the basic construction model;
From second emulation, determine the second emulation utilization rate level for described equipment;
Determine the first instream factor level for described equipment from tolerance, the described first instream factor level and first working load are associated;
Determine the second instream factor level for described equipment from tolerance, the described second instream factor level and second working load are associated;
First error that calculating and described first emulation are associated, described first error is represented the poor of the first emulation utilization rate level and the first instream factor level;
Second error that calculating and described second emulation are associated, described second error is represented the poor of the second emulation utilization rate level and the second instream factor level; And
Calculate the working load correlated error of the difference of expression first sum of errors second error; And
Calibrate described device model with described working load correlated error.
20. computer implemented method as claimed in claim 19 is characterized in that also comprising:
Confirm each device model in the architecture; And
For each equipment that is not identified is determined, calculating and calibration steps.
21. be configured to a kind of device that enforcement of rights requires the computer implemented method described in 19.
22. a system comprises:
Carry out the device of emulation with the model of architecture;
Be used to identify the device of the device model of expression in the basic construction model;
The device that is used for the emulator utilization rate level of definite described equipment;
The device that is used for the physical device utilization rate level of definite described equipment; And
At least part is based on the device that relatively calculates an equipment error of described emulator utilization rate level and described physical device utilization rate level;
Be used for when described equipment error is in a threshold value, confirming the device of model; And
Be used for when described equipment error is not in threshold value standby load-factor as calculated and come the device of calibrating patterns.
23. the system as claimed in claim 22 is characterized in that, the described device that is used for calibrating patterns also comprises:
The device that is used for the Simulation Application utilization rate level of definite described equipment;
The device that is used for the practical application utilization rate level of definite described equipment;
At least part is based on the device that relatively comes to determine a working load pertinency factor of described Simulation Application utilization rate level and described practical application utilization rate level; And
Use a working load pertinency factor to calibrate the device of described equipment.
24. system as claimed in claim 23 is characterized in that, the described device that is used for calibrating patterns also comprises:
At least part is calculated the device of the irrelevant factor of a working load based on described equipment error and described working load pertinency factor; And
Use the irrelevant factor of a working load to calibrate the device of described equipment.
25. system as claimed in claim 23 is characterized in that, the described device that is used for calibrating patterns also comprises:
The device that carries out another emulation with a different working load;
Device for described another another equipment error of simulation calculation;
At least part is based on the device that relatively comes to determine a working load pertinency factor of two equipment errors; And
Use a working load pertinency factor to calibrate the device of described equipment.
26. system as claimed in claim 25 is characterized in that, the described device that is used for calibrating patterns also comprises:
At least partly based on the device that relatively calculates the irrelevant factor of a working load between two equipment errors; And
Use the irrelevant factor of a working load to calibrate the device of described equipment.
27. a system comprises:
At least partly the device that carries out emulation based on the model of architecture;
Device from the performance counter automatic data collection that is associated with an architecture;
By being compared with the data of collecting, simulation result confirms the device of described model automatically;
The device of the error between the data that are used to calculate described simulation result and collect; And
At least partly calibrate the device of described model automatically based on described error.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US59856804P | 2004-08-02 | 2004-08-02 | |
| US60/598,568 | 2004-08-02 | ||
| US11/003,998 | 2004-12-02 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1734427A CN1734427A (en) | 2006-02-15 |
| CN100465918C true CN100465918C (en) | 2009-03-04 |
Family
ID=36076868
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2005100885843A Expired - Fee Related CN100465918C (en) | 2004-08-02 | 2005-08-02 | Automatic configuration of transaction-based performance models |
| CN 200510088588 Pending CN1734428A (en) | 2004-08-02 | 2005-08-02 | Automatic validation and calibration of transaction-based performance models |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 200510088588 Pending CN1734428A (en) | 2004-08-02 | 2005-08-02 | Automatic validation and calibration of transaction-based performance models |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN100465918C (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8572548B2 (en) * | 2008-10-08 | 2013-10-29 | Accenture Global Services Gmbh | Integrated design application |
| US8332820B2 (en) * | 2008-10-30 | 2012-12-11 | Accenture Global Services Limited | Automated load model |
| US8296411B2 (en) | 2010-03-01 | 2012-10-23 | International Business Machines Corporation | Programmatically determining an execution mode for a request dispatch utilizing historic metrics |
| CN102346460B (en) * | 2011-05-27 | 2013-11-13 | 运软网络科技(上海)有限公司 | Transaction-based service control system and method |
| CN106326047B (en) * | 2015-07-02 | 2022-04-05 | 超威半导体(上海)有限公司 | Method for predicting GPU performance and corresponding computer system |
| CN106202386A (en) * | 2016-07-08 | 2016-12-07 | 唐博 | The method automatically analyzing database table relation |
| CN107315889B (en) * | 2017-07-12 | 2020-05-12 | 北京华如科技股份有限公司 | Performance test method of simulation engine and storage medium |
| US10521289B2 (en) * | 2017-10-31 | 2019-12-31 | Hewlett Packard Enterprise Development Lp | Identification of performance affecting flaws in a computing system |
| DE102018202093A1 (en) * | 2018-02-12 | 2019-08-14 | Robert Bosch Gmbh | Method and device for calculating data models in safety-critical systems |
| CN111143127B (en) * | 2019-12-23 | 2023-09-26 | 杭州迪普科技股份有限公司 | Method, device, storage medium and equipment for supervising network equipment |
| CN112783650B (en) * | 2021-01-20 | 2024-01-16 | 之江实验室 | Multi-model parallel reasoning method based on AI chip |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1389829A (en) * | 2001-06-05 | 2003-01-08 | 因芬尼昂技术股份公司 | Fault searching method and fault searching apparatus |
| WO2003039070A2 (en) * | 2001-11-01 | 2003-05-08 | British Telecommunications Public Limited Company | Method and apparatus for analysing network robustness |
-
2005
- 2005-08-02 CN CNB2005100885843A patent/CN100465918C/en not_active Expired - Fee Related
- 2005-08-02 CN CN 200510088588 patent/CN1734428A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1389829A (en) * | 2001-06-05 | 2003-01-08 | 因芬尼昂技术股份公司 | Fault searching method and fault searching apparatus |
| WO2003039070A2 (en) * | 2001-11-01 | 2003-05-08 | British Telecommunications Public Limited Company | Method and apparatus for analysing network robustness |
Non-Patent Citations (2)
| Title |
|---|
| PREDICTING FEEI COMPUTER NETWORKPERFORMANCE USING A CALIBRATED SOFTWARESIMULATION MODEL. DEMPSEY S ET AL.PERFORMANCE,COMPUTING,AND COMMUNICATIONS CONFERENCE. 1997 |
| PREDICTING FEEI COMPUTER NETWORKPERFORMANCE USING A CALIBRATED SOFTWARESIMULATION MODEL. DEMPSEY S ET AL.PERFORMANCE,COMPUTING,AND COMMUNICATIONS CONFERENCE. 1997 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN1734428A (en) | 2006-02-15 |
| CN1734427A (en) | 2006-02-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN100465918C (en) | Automatic configuration of transaction-based performance models | |
| KR20060061759A (en) | Automatic validation and calibration of transaction-based performance models | |
| US7881920B2 (en) | Systemic enterprise management method and apparatus | |
| Măruşter et al. | Redesigning business processes: a methodology based on simulation and process mining techniques | |
| Li et al. | Architectural technical debt identification based on architecture decisions and change scenarios | |
| US6311144B1 (en) | Method and apparatus for designing and analyzing information systems using multi-layer mathematical models | |
| US7996204B2 (en) | Simulation using resource models | |
| US8903815B2 (en) | Enterprise architecture system and method | |
| US20170109657A1 (en) | Machine Learning-Based Model for Identifying Executions of a Business Process | |
| US7974827B2 (en) | Resource model training | |
| US20080262824A1 (en) | Creation of resource models | |
| US20170109676A1 (en) | Generation of Candidate Sequences Using Links Between Nonconsecutively Performed Steps of a Business Process | |
| US11620420B2 (en) | Computing system simulation and testing environment | |
| EP1631002A2 (en) | Automatic configuration of network performance models | |
| US20170109667A1 (en) | Automaton-Based Identification of Executions of a Business Process | |
| WO2008134143A1 (en) | Resource model training | |
| US20170109636A1 (en) | Crowd-Based Model for Identifying Executions of a Business Process | |
| US20170109639A1 (en) | General Model for Linking Between Nonconsecutively Performed Steps in Business Processes | |
| CN110162445A (en) | The host health assessment method and device of Intrusion Detection based on host log and performance indicator | |
| US20170109638A1 (en) | Ensemble-Based Identification of Executions of a Business Process | |
| Hammad et al. | Provenance as a service: A data-centric approach for real-time monitoring | |
| US20170109640A1 (en) | Generation of Candidate Sequences Using Crowd-Based Seeds of Commonly-Performed Steps of a Business Process | |
| Montes et al. | Finding order in chaos: a behavior model of the whole grid | |
| US20170109670A1 (en) | Crowd-Based Patterns for Identifying Executions of Business Processes | |
| Calzarossa et al. | Construction and use of multiclass workload models |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20090304 Termination date: 20140802 |
|
| EXPY | Termination of patent right or utility model |