CN108563510B - E-level calculation-oriented architecture perception optimization method - Google Patents
E-level calculation-oriented architecture perception optimization method Download PDFInfo
- Publication number
- CN108563510B CN108563510B CN201810418522.1A CN201810418522A CN108563510B CN 108563510 B CN108563510 B CN 108563510B CN 201810418522 A CN201810418522 A CN 201810418522A CN 108563510 B CN108563510 B CN 108563510B
- Authority
- CN
- China
- Prior art keywords
- program
- independent code
- algorithm
- calculation
- perception
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses an E-level calculation-oriented architecture perception optimization method, which comprises the steps of task flow perception optimization; a step of program code perception optimization; a step of program algorithm perception optimization and a step of vector perception optimization. The E-level computing-oriented architecture perception optimization method provided by the invention adopts various perception optimization algorithms, can reasonably adjust and allocate hardware resources when facing to massive computing resources in E-level computing, and improves the computing efficiency.
Description
Technical Field
The invention particularly relates to an E-level calculation-oriented architecture perception optimization method.
Background
With the development of economic technology, the improvement of living standards of people and the arrival of the information-oriented era, the era of supercomputers for class E computing (millions of times) has come. The E-level calculation has super-strong calculation capability and can bring revolutionary changes to the calculation science and scientific research.
With the development of science, the scale of various numerical models is larger and larger, and the models are closer to the actual situation. However, a more accurate model means a more complex model solution. At present, research expenses of scientific research institutes are relatively tight, and expenses required for purchasing a large number of servers cannot be borne. Therefore, even if a more accurate numerical model is successfully established by the scientific research institution, the numerical model cannot be independently solved by the scientific research institution.
If the scientific research institutions can transplant the codes of the more accurate numerical models to the supercomputing platform facing the E-level calculation, the scientific research institutions do not need to purchase a large number of servers with strong performance, and only need to transplant the calculation codes to the supercomputing platform, remotely check the calculation progress and effect and pay the supercomputing cost. The cost of the overcomputing calculation is undoubtedly much lower than that of a large number of powerful servers.
However, how to effectively transplant the calculation codes of the numerical model to the supercomputing platform facing the E-level calculation so as to exert the calculation capability of the supercomputing platform for the E-level calculation to the maximum extent has no relevant research at present, so that the development of the field is also restricted to a certain extent.
Disclosure of Invention
The invention aims to provide an E-level calculation-oriented architecture perception optimization method which is oriented to E-level calculation and can be used for perceiving and optimizing a program according to a task architecture.
The invention provides an E-level calculation-oriented architecture perception optimization method, which comprises the following steps:
a task flow knowledge optimization step, which is used for perceiving and optimizing all tasks to be executed, adjusting and obtaining the processing sequence of each task, thereby accelerating the processing efficiency of the task flow;
the program code perception optimization step is used for distributing the program codes to be executed to the most appropriate system for distributed calculation and result summarization, so that the processing efficiency of the program codes is improved;
a step of perception optimization of the program algorithm, which is used for analyzing the program algorithm and distributing corresponding computing resources, thereby improving the processing efficiency of the program algorithm;
and a vector perception optimization step, which is used for selecting the largest compressed storage format for the vectorization calculation in the calculation process, so that the processing efficiency of the vectorization calculation is improved.
The task flow perception optimization specifically comprises the following steps:
(1) acquiring all tasks to be executed, and confirming that no interdependence relation exists among all tasks;
(2) identifying the subtasks of all the tasks acquired in the step (1), and uniformly distributing computing resources to each subtask within a time period;
(3) and finding the subtask with the minimum calculated amount in each subtask by adopting a shortest operation optimization algorithm, and preferentially processing the subtask with the minimum calculated amount.
The program code perception optimization specifically comprises the following steps:
1) testing the test codes to find out the optimal matching relation between each typical independent code segment and system hardware;
2) identifying the program code and finding an independent code segment in the program code;
3) matching the independent code segments obtained in the step 2) with the typical independent code segments to obtain the corresponding relation between each independent code segment and the typical independent code segment;
4) according to the corresponding relation between the independent code segments and the typical independent code segments in the step 3) and the optimal matching relation between each typical independent code segment and the system hardware in the step 1), the independent code segments are distributed to the corresponding system hardware, and therefore the processing efficiency of the program codes is improved.
The program algorithm perception optimization specifically comprises the following steps:
A. before the program algorithm is operated, carrying out average distribution of computing resources on different program algorithms;
B. when the program algorithm runs, if the situation of calculation waiting occurs, the resource requirement of the program algorithm is reanalyzed;
C. and D, according to the analysis result obtained in the step B, reallocating the computing resources by adopting a dynamic resource balance (DRF) algorithm.
The step C of reallocating the computing resources specifically includes reallocating the computing resources by using the following principles:
r1. users cannot obtain more resources than other users;
r2. users cannot get more resources by lying their resource needs;
r3, all available resources are distributed without replacing the existing resource distribution;
r4. users will not prefer the resource allocation of other users.
The E-level computing-oriented architecture perception optimization method provided by the invention adopts various perception optimization algorithms, can reasonably adjust and allocate hardware resources when facing to massive computing resources in E-level computing, and improves the computing efficiency.
Drawings
FIG. 1 is a schematic diagram of the process of the present invention.
Detailed Description
FIG. 1 is a schematic flow chart of the method of the present invention: the invention provides an E-level calculation-oriented architecture perception optimization method, which comprises the following steps:
a task flow knowledge optimization step, which is used for perceiving and optimizing all tasks to be executed, adjusting and obtaining the processing sequence of each task, thereby accelerating the processing efficiency of the task flow; the method specifically comprises the following steps:
(1) acquiring all tasks to be executed, and confirming that no interdependence relation exists among all tasks;
(2) identifying the subtasks of all the tasks acquired in the step (1), and uniformly distributing computing resources to each subtask within a time period;
(3) finding the subtask with the minimum calculated amount in each subtask by adopting a shortest operation optimization algorithm, and preferentially processing the subtask with the minimum calculated amount;
the program code perception optimization step is used for distributing the program codes to be executed to the most appropriate system for distributed calculation and result summarization, so that the processing efficiency of the program codes is improved; the method specifically comprises the following steps:
1) testing the test codes to find out the optimal matching relation between each typical independent code segment and system hardware;
2) identifying the program code and finding an independent code segment in the program code;
3) matching the independent code segments obtained in the step 2) with the typical independent code segments to obtain the corresponding relation between each independent code segment and the typical independent code segment;
4) distributing the independent code segments to corresponding system hardware according to the corresponding relation between the independent code segments and the typical independent code segments in the step 3) and the optimal matching relation between each typical independent code segment and the system hardware in the step 1), thereby improving the processing efficiency of the program codes;
a step of perception optimization of the program algorithm, which is used for analyzing the program algorithm and distributing corresponding computing resources, thereby improving the processing efficiency of the program algorithm; the method specifically comprises the following steps:
A. before the program algorithm is operated, carrying out average distribution of computing resources on different program algorithms;
B. when the program algorithm runs, if the situation of calculation waiting occurs, the resource requirement of the program algorithm is reanalyzed;
C. b, according to the analysis result obtained in the step B, a dynamic resource balance (DRF) algorithm is adopted to redistribute the computing resources;
the vector perception optimization step is used for selecting the largest compressed storage format for vectorization calculation in the calculation process, so that the processing efficiency of the vectorization calculation is improved; the following principles are adopted for reallocation:
r1. users cannot obtain more resources than other users;
r2. users cannot get more resources by lying their resource needs;
r3, all available resources are distributed without replacing the existing resource distribution;
r4. users will not prefer the resource allocation of other users.
Claims (1)
1. An E-level computing-oriented architecture perception optimization method comprises the following steps:
a task flow knowledge optimization step, which is used for perceiving and optimizing all tasks to be executed, adjusting and obtaining the processing sequence of each task, thereby accelerating the processing efficiency of the task flow; the method specifically comprises the following steps:
(1) acquiring all tasks to be executed, and confirming that no interdependence relation exists among all tasks;
(2) identifying the subtasks of all the tasks acquired in the step (1), and uniformly distributing computing resources to each subtask within a time period;
(3) finding the subtask with the minimum calculated amount in each subtask by adopting a shortest operation optimization algorithm, and preferentially processing the subtask with the minimum calculated amount;
the program code perception optimization step is used for distributing the program codes to be executed to the most appropriate system for distributed calculation and result summarization, so that the processing efficiency of the program codes is improved; the method specifically comprises the following steps:
1) testing the test codes to find out the optimal matching relation between each typical independent code segment and system hardware;
2) identifying the program code and finding an independent code segment in the program code;
3) matching the independent code segments obtained in the step 2) with the typical independent code segments to obtain the corresponding relation between each independent code segment and the typical independent code segment;
4) distributing the independent code segments to corresponding system hardware according to the corresponding relation between the independent code segments and the typical independent code segments in the step 3) and the optimal matching relation between each typical independent code segment and the system hardware in the step 1), thereby improving the processing efficiency of the program codes;
a step of perception optimization of the program algorithm, which is used for analyzing the program algorithm and distributing corresponding computing resources, thereby improving the processing efficiency of the program algorithm; the method specifically comprises the following steps:
A. before the program algorithm is operated, carrying out average distribution of computing resources on different program algorithms;
B. when the program algorithm runs, if the situation of calculation waiting occurs, the resource requirement of the program algorithm is reanalyzed;
C. b, according to the analysis result obtained in the step B, a dynamic resource balance (DRF) algorithm is adopted to redistribute the computing resources; the following principles are adopted for reallocation:
r1. a user cannot obtain more resources than other users;
r2. users cannot get more resources by lying their resource needs;
r3. allocate all available resources without replacing the existing resource allocation;
r4. users do not prefer the resource allocation of other users;
and a vector perception optimization step, which is used for selecting the largest compressed storage format for the vectorization calculation in the calculation process, so that the processing efficiency of the vectorization calculation is improved.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810418522.1A CN108563510B (en) | 2018-05-04 | 2018-05-04 | E-level calculation-oriented architecture perception optimization method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810418522.1A CN108563510B (en) | 2018-05-04 | 2018-05-04 | E-level calculation-oriented architecture perception optimization method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108563510A CN108563510A (en) | 2018-09-21 |
| CN108563510B true CN108563510B (en) | 2021-07-13 |
Family
ID=63537648
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810418522.1A Active CN108563510B (en) | 2018-05-04 | 2018-05-04 | E-level calculation-oriented architecture perception optimization method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108563510B (en) |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7774191B2 (en) * | 2003-04-09 | 2010-08-10 | Gary Charles Berkowitz | Virtual supercomputer |
| CN101453398A (en) * | 2007-12-06 | 2009-06-10 | 怀特威盛软件公司 | Novel distributed grid super computer system and method |
| JP5699554B2 (en) * | 2010-11-11 | 2015-04-15 | 富士通株式会社 | Vector processing circuit, instruction issue control method, and processor system |
| WO2014065873A1 (en) * | 2012-10-22 | 2014-05-01 | Jeff Willey | Control messaging in multislot link layer flit |
| CN103885839B (en) * | 2014-04-06 | 2017-02-22 | 孙凌宇 | Cloud computing task scheduling method based on multilevel division method and empowerment directed hypergraphs |
| CN107147517A (en) * | 2017-03-24 | 2017-09-08 | 上海交通大学 | A kind of adaptive polo placement resource allocation methods for virtual network function |
| CN107977270A (en) * | 2017-11-22 | 2018-05-01 | 用友金融信息技术股份有限公司 | Peers distribution method, peers distribution system and computer installation |
-
2018
- 2018-05-04 CN CN201810418522.1A patent/CN108563510B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN108563510A (en) | 2018-09-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109993299B (en) | Data training method and device, storage medium and electronic device | |
| JP6844067B2 (en) | Supplying software applications to edge devices in IoT environments | |
| JP2018026114A (en) | Application profiling job management system, program, and method | |
| US11645123B1 (en) | Dynamic distribution of a workload processing pipeline on a computing infrastructure | |
| GB2605922A (en) | Elastic execution of machine learning workloads using application based profiling | |
| KR20190081306A (en) | Method for allocating resource for bigdata analyzing software and apparatus for allocating virtual resource using the method | |
| CN110249312A (en) | Data integration operation changing | |
| Tang et al. | Data-aware resource scheduling for multicloud workflows: A fine-grained simulation approach | |
| CN112965803A (en) | AI model generation method and electronic equipment | |
| CN108633310A (en) | Scheduling is executed based on classification | |
| Silva et al. | SmartRank: a smart scheduling tool for mobile cloud computing | |
| US20210326761A1 (en) | Method and System for Uniform Execution of Feature Extraction | |
| KR102386330B1 (en) | Apparutus and method for analyzing user process based on standard process | |
| Ribas et al. | Pbfvmc: A new pseudo-boolean formulation to virtual-machine consolidation | |
| CN108563510B (en) | E-level calculation-oriented architecture perception optimization method | |
| Sukhija et al. | Evaluating the flexibility of dynamic loop scheduling on heterogeneous systems in the presence of fluctuating load using SimGrid | |
| US20200249740A1 (en) | Methods and devices for power management based on synthetic machine learning benchmarks | |
| Zhang et al. | Inferbench: Understanding deep learning inference serving with an automatic benchmarking system | |
| CN104424525B (en) | Auxiliary is identified project the method and apparatus of scope | |
| US20220050761A1 (en) | Low overhead performance data collection | |
| Cañizares et al. | Eminent: Embarrassingly parallel mutation testing | |
| CN102567024A (en) | Script executing system and method | |
| CN113918211B (en) | Method, device and equipment for executing industrial equipment object data model | |
| KR101273374B1 (en) | System and Method for executing simulation workflow based on plug-in | |
| TWI777695B (en) | Method for automatic scheduling tasks, electronic device, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |