CN102945198B - A kind of method characterizing high-performance calculation application characteristic - Google Patents
A kind of method characterizing high-performance calculation application characteristic Download PDFInfo
- Publication number
- CN102945198B CN102945198B CN201210398976.XA CN201210398976A CN102945198B CN 102945198 B CN102945198 B CN 102945198B CN 201210398976 A CN201210398976 A CN 201210398976A CN 102945198 B CN102945198 B CN 102945198B
- Authority
- CN
- China
- Prior art keywords
- application
- data
- node
- resource
- performance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 21
- 238000004364 calculation method Methods 0.000 title claims description 10
- 230000008569 process Effects 0.000 claims abstract description 8
- 238000012544 monitoring process Methods 0.000 claims description 17
- 238000012512 characterization method Methods 0.000 claims description 16
- 238000004458 analytical method Methods 0.000 claims description 10
- FFBHFFJDDLITSX-UHFFFAOYSA-N benzyl N-[2-hydroxy-4-(3-oxomorpholin-4-yl)phenyl]carbamate Chemical compound OC1=C(NC(=O)OCC2=CC=CC=C2)C=CC(=C1)N1CCOCC1=O FFBHFFJDDLITSX-UHFFFAOYSA-N 0.000 claims description 6
- 230000006855 networking Effects 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 4
- 238000010276 construction Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 claims description 3
- 230000008520 organization Effects 0.000 claims description 3
- 230000006872 improvement Effects 0.000 abstract description 2
- 230000014509 gene expression Effects 0.000 abstract 1
- 238000005516 engineering process Methods 0.000 description 5
- 238000011161 development Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 241001675026 Larix potaninii Species 0.000 description 1
- 241001116459 Sequoia Species 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- JJWKPURADFRFRB-UHFFFAOYSA-N carbonyl sulfide Chemical compound O=C=S JJWKPURADFRFRB-UHFFFAOYSA-N 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000005433 particle physics related processes and functions Effects 0.000 description 1
- 239000003208 petroleum Substances 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The invention provides a kind of method characterizing high-performance computing sector every profession and trade application software operation characteristic.Application, at the load pressure inputting, store, process, transmit and export these five links, is divided into the large classification of computation-intensive, internal memory restricted type, I/O intensity and network-intensive etc. four by the method comprehensive application program that examines successively.By these four aspect quantificational expressions, fully show that application takies CPU, memory size, internal memory are handled up, the resource requirement of the aspect such as I/O and network data exchange, maximizedly reflected application software operation characteristic.The present invention is simple, practical and reliable, efficient, can reflect the demand of a application software to high-performance hardware resource very intuitively.This application can be run in suitable high performance platform, and then play the performance of application software to greatest extent.According to this feature, the improvement that can shoot the arrow at the target and break through the performance bottleneck of this application software, improves the extendability of application.
Description
Technical field
The present invention relates to the content of high-performance computing sector in application software performance characterization, be specifically related to a kind of utilization and monitor and extract suitable high-performance parameter, the most reasonably reflect that large-scale application software is to the method for computational resource requirements.
Background technology
Along with the continuous progress of human society, the development of science and technology, people are not only more and more extensive to natural understanding, and also more and more urgent to the demand of outfield exploration.This just makes the mankind support the growth of the amount sharpness of the information data of holding, and with time simultaneously, the information data of these magnanimity all needs treatment and analysis timely.Such as, a large-scale astronomical radio telescope array one just can produce the cosmic microwave data of more than 100GB second, and these data all need to be analyzed in time; For another example, in particle physics research field, the data that LHC once clashes also are measured in units of TB; In addition, also more and more higher requirement is proposed to computing power as human genome project, petroleum prospecting, weather forecast etc. field.Under this overall background, numerical evaluation becomes the third the extremely important Science Explorations means except experiment, theoretical analysis already.Just based on such reality, each science and technology power of the world today has been impelled all to greatly develop supercomputer what do one's utmost.As, in the world TOP500 issued in June, 2012, the IBM " Chinese larch (Sequoia) " ranked the first just has reached the peak velocity of 20PFlops, and No. trillion time meanwhile new supercomputer is also among research and planning.Generally speaking, every ten years substantially, the speed of supercomputer just promoted three magnitudes (1000 times), and the ability of therefore building supercomputer has become an embodiment effectively of a national science and technology level and overall national strength.
Although the supercomputer speed of development is surprising, also gratifying, the software engineering regrettably matched with it is hesitated to move forward, this serious performance constraining supercomputer application power.The application software of the present overwhelming majority based on ultimate principle and mathematical algorithm, or 50 ~ sixties of last century proposing and to grow up, these algorithms and large scale computer at that time mate completely and adapt to, with serial or also behavior master between process on a small quantity.But through the development of more than 50 years, present supercomputing machine architecture there occurs earth-shaking change, easily hundreds of thousands and even CPU core up to a million is had, and also having quite a few supercomputer to be the architectural framework of mixing isomery (CPU+GPU/MIC etc.) used, this just makes early stage physical model and mathematical algorithm is unable to do what one wishes, cannot be competent at.The main cause of present most application software inefficiency that Here it is, poor expandability.
Crack these present difficult problems, on the one hand, we should research and development is new energetically the physical model matched with supercomputing machine architecture now and mathematical algorithm, this is the Last Resort breaking through existing bottleneck, but this is an extremely difficult problem after all, cannot see effect at short notice and realize large-scale application; On the other hand, we should set about studying now inherit application software that get off, magnanimity, their operation characteristic of rational sign, find out their performance bottleneck, on existing platform, play the performance of these application to greatest extent, strong foundation can also be provided for the improvement of application performance and breakthrough in addition.Therefore, how rationally, the feature of the sign application of science is exactly subject matter to be solved by this invention.
Summary of the invention
The object of this invention is to provide a kind of method characterizing high-performance calculation application characteristic.
The object of the invention is to realize in the following manner, the technical problem to be solved in the present invention is a kind of method characterizing application program operation characteristic in high-performance calculation fast and efficiently of design, thus rapid, accurate position-location application is to the demand of computational resource, plays the performance of application program to greatest extent.
For the feature of existing high-performance computer architectural framework and computing application, two key steps are substantially divided into by the characterization of application operation characteristic, namely, 1) for application program runs the monitoring that takies computational resource and data are extracted, 2) for the analysis of institute's image data and aftertreatment, for the former, according to the construction characteristic of high-performance calculation platform, from running background watch-dog, realize the real-time monitoring that application programs computational resource takies situation, and extract data, it not only will for the hardware platform of different framework, more require that watch-dog is very little to taking of resource, the normal operation of monitored program can not be had influence on, and for the latter, according to the feature setting reasonably reference amount of hardware platform, choose from monitored mass data and suitable to analyze with reference to amount, with position-location application level is needed to computational resource, require the analyzing and processing ability to mass data of unified standard, concrete analysis, organization flow is as follows:
1) software and hardware platform is determined: refer to the application software according to characterizing, select suitable hardware platform, and dispose the software environments such as good corresponding system, math library, watch-dog, here the performance of hardware platform should balance as much as possible, and leaves certain resource excess as much as possible;
2) running monitor: refer in flow process 1) on the hardware platform disposed, run resource monitor respectively at master and slave node; Here resource monitor should meet the function of monitoring application resource occupancy in real time from all computing nodes, include but not limited to CPU usage, EMS memory occupation amount, in real time IO bandwidth, network throughput, it applies the complete all resources containing monitored hardware platform;
3) send monitor data from node to main controlled node: main controlled node and from node for monitoring, it is a relative concept, main controlled node can be also from node simultaneously, it mainly completes the reception to monitor data, from node be then monitoring own resource take situation and be responsible for main controlled node send monitor data, if main controlled node does not normally receive data just need Returning process 1), can use with the state redefining soft, hard platform;
4) on monitored node, run the application program that will characterize;
5) monitor in real time: should ensure that monitored data are authentic and valid, as data distortion then answers Returning process 2);
6) analytical standard is determined: according to the hardware characteristics of run application hardware platform, determine reference point, if application runs under gigabit networking, then should get the bandwidth upper limit 125MB/s of gigabit networking as reference point, if use the Infiniband network being, then application gets the higher limit of used HCA card as a reference: the defining method of other index parameters is same therewith;
7) characteristic feature is generated: according to monitor data, calculating mean value or choose maximal value, and compared with reference standard, gained ratio is this characterization result, different application each have different characterization results, for application network characterisation, the network traffics mean value of computing application run duration or maximal value, and compared with standard reference point, be the characterization result of this application.
The invention has the beneficial effects as follows: the present invention takes full advantage of the feature of high-performance calculation, the degree of depth is explored and has been excavated the desirability of application program to computational resource, and does not affect the normal operation of analyzed application program with atomic weak resource occupation amount.The method can characterize the feature of application fast, has played the calculated performance of existing magnanimity application greatly.
Accompanying drawing explanation
Fig. 1 is calculation and analysis methods process flow diagram.
Embodiment
With reference to Figure of description, method of the present invention is described in detail below.
The present invention is directed to the feature of existing high-performance computer architectural framework and computing application, two key steps are substantially divided into by the characterization of application operation characteristic, namely, one runs for application program the monitoring that takies computational resource and data are extracted, and two for the analysis of institute's image data and aftertreatment.For the former, mainly according to the construction characteristic of high-performance calculation platform, from running background watch-dog, realize the real-time monitoring that application programs computational resource takies situation, and extract data, it not only for the hardware platform of different framework, will more require that watch-dog is very little to taking of resource, can not have influence on the normal operation of monitored program; And for the latter, mainly according to the feature setting reasonably reference amount of hardware platform, from monitored mass data, choose suitable amount analyze, with position-location application level is needed to computational resource, the analyzing and processing ability to mass data of its major requirement unified standard.Specifically, substantially following several step can be divided into:
1. dispose watch-dog according to computing platform;
2. start watch-dog from main controlled node, collect the characteristic parameter of computing node successively;
3. run the application program of specifying;
4. terminate the monitoring of application, exit monitoring environment;
5. according to hardware platform setting reference standard;
6. analyzing and processing monitor data.
In order to make object of the present invention, technical scheme and advantage more clear, we are to use the characteristic present process of the application program of CPU on two nodes, and by reference to the accompanying drawings, committed step in the present invention is described in detail, for the more computing nodes of use, or use the characteristic manner of the application program of the isomery systems such as GPU/MIC identical with it.
Embodiment
As shown in Figure 1, the schematic diagram of analysis process involved in the present invention is given.Its basic analysis, organization flow are as follows:
1. determine software and hardware platform.Mainly refer to the application software according to characterizing, select suitable hardware platform, and dispose the software environments such as good corresponding system, math library, watch-dog.Here the performance of hardware platform should balance as much as possible, and leaves certain resource excess as much as possible;
2. running monitor.Mainly to refer in flow process 1 on the hardware platform disposed, run resource monitor respectively at master and slave node.Here resource monitor should meet the function of monitoring application resource occupancy in real time from all computing nodes, include but not limited to CPU usage, EMS memory occupation amount, in real time IO bandwidth, network throughput etc., it applies the complete all resources containing monitored hardware platform;
3. send monitor data from node to main controlled node.Main controlled node and from node for monitoring, it is a relative concept, main controlled node can be also from node simultaneously, and it mainly completes the reception to monitor data, is then that monitoring own resource takies situation and is responsible for sending monitor data to main controlled node from node.If in this step, main controlled node does not normally receive data just need Returning process 1, can use with the state redefining soft, hard platform;
4. on monitored node, run the application program that will characterize;
5. monitor in real time.Here mainly should ensure that monitored data are authentic and valid, as data distortion then answers Returning process 2;
6. determine analytical standard.Mainly refer to the hardware characteristics according to run application hardware platform, determine reference point.As, if application runs under gigabit networking, then should get the bandwidth upper limit (125MB/s) of gigabit networking as reference point, if use the Infiniband network being, then application gets the higher limit of used HCA card as a reference.The defining method of other index parameters is same therewith;
7. generate characteristic feature.Mainly refer to according to monitor data, calculating mean value (or choose maximal value etc., determine according to specific occasion) compared with reference standard, gained ratio is this characterization result, different application each have different characterization results.Such as, for the network characterisation of application, the network traffics mean value of computing application run duration (or maximal value etc.), and compared with standard reference point, be the characterization result of this application.
Analytical approach of the present invention can reflect the operation characteristic of application program greatly, thus hold application taking computational resource fast, and the extendability of application etc., thus accurately location at utmost can play the hardware platform of application performance, realize application performance and maximize.
Except the technical characteristic described in instructions, be the known technology of those skilled in the art.
Claims (1)
1. one kind characterizes the method for high-performance calculation application characteristic, it is characterized in that the feature for existing high-performance computer architectural framework and computing application, two key steps are substantially divided into by the characterization of application operation characteristic, namely, 1) for application program runs the monitoring that takies computational resource and data are extracted, 2) for the analysis of institute's image data and aftertreatment, for the former, according to the construction characteristic of high-performance calculation platform, from running background watch-dog, realize the real-time monitoring that application programs computational resource takies situation, and extract data, it not only will for the hardware platform of different framework, more require that watch-dog is very little to taking of resource, the normal operation of monitored program can not be had influence on, and for the latter, according to the feature setting reasonably reference amount of hardware platform, choose from monitored mass data and suitable to analyze with reference to amount, with position-location application level is needed to computational resource, require the analyzing and processing ability to mass data of unified standard, concrete analysis, organization flow is as follows:
1) software and hardware platform is determined: refer to the application software according to characterizing, select suitable hardware platform, and dispose the software environment of good corresponding system, math library, watch-dog, here the performance of hardware platform should balance as much as possible, and leaves certain resource excess as much as possible;
2) running monitor: refer in flow process 1) on the hardware platform disposed, run resource monitor respectively at master and slave node; Here resource monitor should meet the function of monitoring application resource occupancy in real time from all computing nodes, include but not limited to CPU usage, EMS memory occupation amount, in real time IO bandwidth, network throughput, it applies the complete all resources containing monitored hardware platform;
3) send monitor data from node to main controlled node: main controlled node and from node for monitoring, it is a relative concept, main controlled node can be also from node simultaneously, it mainly completes the reception to monitor data, from node be then monitoring own resource take situation and be responsible for main controlled node send monitor data, if main controlled node does not normally receive data just need Returning process 1), can use with the state redefining soft, hard platform;
4) on monitored node, run the application program that will characterize;
5) monitor in real time: should ensure that monitored data are authentic and valid, as data distortion then answers Returning process 2);
6) analytical standard is determined: according to the hardware characteristics of run application hardware platform, determine reference point, if application runs under gigabit networking, then should get the bandwidth upper limit 125MB/s of gigabit networking as reference point, if use the Infiniband network being, then application gets the higher limit of used HCA card as a reference: the defining method of other index parameters is same therewith;
7) characteristic feature is generated: according to monitor data, calculating mean value or choose maximal value, and compared with reference standard, gained ratio is this characterization result, different application each have different characterization results, for application network characterisation, the network traffics mean value of computing application run duration or maximal value, and compared with standard reference point, be the characterization result of this application.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210398976.XA CN102945198B (en) | 2012-10-19 | 2012-10-19 | A kind of method characterizing high-performance calculation application characteristic |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210398976.XA CN102945198B (en) | 2012-10-19 | 2012-10-19 | A kind of method characterizing high-performance calculation application characteristic |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102945198A CN102945198A (en) | 2013-02-27 |
| CN102945198B true CN102945198B (en) | 2016-03-02 |
Family
ID=47728146
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210398976.XA Active CN102945198B (en) | 2012-10-19 | 2012-10-19 | A kind of method characterizing high-performance calculation application characteristic |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102945198B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103246569A (en) * | 2013-05-20 | 2013-08-14 | 浪潮(北京)电子信息产业有限公司 | Method and device for representing high-performance calculation application characteristics |
| CN103501253A (en) * | 2013-10-18 | 2014-01-08 | 浪潮电子信息产业股份有限公司 | Monitoring organization method for high-performance computing application characteristics |
| CN103716210B (en) * | 2014-01-07 | 2017-05-24 | 浪潮(北京)电子信息产业有限公司 | System, device and method for monitoring operation efficiency of calculation application software |
| CN104156296B (en) * | 2014-08-01 | 2017-06-30 | 浪潮(北京)电子信息产业有限公司 | The system and method for intelligent monitoring large-scale data center cluster calculate node |
| CN106201691A (en) * | 2016-07-11 | 2016-12-07 | 浪潮(北京)电子信息产业有限公司 | The dispatching method of a kind of network I/O intensive task and device |
| CN115129541B (en) * | 2022-06-20 | 2024-03-26 | 北京计算机技术及应用研究所 | High-performance computing resource monitoring implementation method based on Feiteng platform |
| CN117407250B (en) * | 2023-12-15 | 2024-03-12 | 上海飞斯信息科技有限公司 | Computer performance control system based on real-time processing of running environment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1026592A2 (en) * | 1999-02-04 | 2000-08-09 | Sun Microsystems, Inc. | Method for analyzing the performance of application programs |
| CN102591921A (en) * | 2010-12-20 | 2012-07-18 | 微软公司 | Scheduling and management in a personal datacenter |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE10214185A1 (en) * | 2002-03-28 | 2003-10-16 | Siemens Ag | PC arrangement for visualization, diagnostic and expert systems for monitoring and control or regulation of high-voltage supply units of electrostatic filters |
| US7996839B2 (en) * | 2003-07-16 | 2011-08-09 | Hewlett-Packard Development Company, L.P. | Heterogeneous processor core systems for improved throughput |
-
2012
- 2012-10-19 CN CN201210398976.XA patent/CN102945198B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1026592A2 (en) * | 1999-02-04 | 2000-08-09 | Sun Microsystems, Inc. | Method for analyzing the performance of application programs |
| CN102591921A (en) * | 2010-12-20 | 2012-07-18 | 微软公司 | Scheduling and management in a personal datacenter |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102945198A (en) | 2013-02-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102945198B (en) | A kind of method characterizing high-performance calculation application characteristic | |
| Tikir et al. | PSINS: An open source event tracer and execution simulator for MPI applications | |
| CN104156296B (en) | The system and method for intelligent monitoring large-scale data center cluster calculate node | |
| CN106547882A (en) | A kind of real-time processing method and system of big data of marketing in intelligent grid | |
| WO2022110446A1 (en) | Simulation method and apparatus for heterogeneous cluster scheduling, computer device, and storage medium | |
| CN104504257B (en) | A kind of online Prony analysis methods calculated based on Dual parallel | |
| CN103501253A (en) | Monitoring organization method for high-performance computing application characteristics | |
| CN103246569A (en) | Method and device for representing high-performance calculation application characteristics | |
| CN102307369A (en) | Device and method for supporting parallel simulation and physical simulation of wireless sensor network | |
| CN104407688A (en) | Virtualized cloud platform energy consumption measurement method and system based on tree regression | |
| CN106408126A (en) | Three-stage optimization method oriented to concurrent acquisition of energy consumption data | |
| Li et al. | A hybrid particle swarm optimization algorithm for load balancing of MDS on heterogeneous computing systems | |
| CN105260253A (en) | Server failure measurement and calculation method and device | |
| CN105700998A (en) | Method and device for monitoring and analyzing performance of parallel programs | |
| Shu et al. | Resource demand prediction of cloud workloads using an attention-based GRU model | |
| CN110322153A (en) | Monitor event processing method and system | |
| CN107679133B (en) | Mining method applicable to massive real-time PMU data | |
| CN103455364B (en) | A kind of multi-core environment concurrent program Cache performance online obtains system and method | |
| Hu et al. | An improved adaptive genetic algorithm in cloud computing | |
| CN104090813B (en) | A kind of method for analyzing and modeling of the virtual machine CPU usage of cloud data center | |
| Xu et al. | Memory-efficient hardware performance counters with approximate-counting algorithms | |
| Li et al. | A GPU-based parallel algorithm for large scale linear programming problem | |
| CN102890642B (en) | Performance analysis method based on heterogeneous reconfigurable computing (HRC) of matching matrix | |
| CN105490871B (en) | A kind of method and system for testing Hadoop cluster stability | |
| Ziganurova et al. | Local Virtual Times Analysis in PCS Model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |