CN102622260A - Optimization method and optimization system of on-line iteration compiling - Google Patents
Optimization method and optimization system of on-line iteration compiling Download PDFInfo
- Publication number
- CN102622260A CN102622260A CN2012100484833A CN201210048483A CN102622260A CN 102622260 A CN102622260 A CN 102622260A CN 2012100484833 A CN2012100484833 A CN 2012100484833A CN 201210048483 A CN201210048483 A CN 201210048483A CN 102622260 A CN102622260 A CN 102622260A
- Authority
- CN
- China
- Prior art keywords
- module
- iteration
- optimization
- performance
- control module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Devices For Executing Special Programs (AREA)
- Stored Programmes (AREA)
Abstract
The invention discloses an optimization method and an optimization system of on-line iteration compiling. The optimization method comprises the following steps that 100 a control module sends a source code or a compiler of the corresponding program in a mode of further optimized intermediate representation to a working module, and needed time of remaining data recording after processing is forecasted in a process that the program parallely processes the data recording stored in the working module; 200 the working module conducts iteration compiling optimization operation on the program and enables data of optimization results to be fed to the control module; and 300 operation time of the program and time of the iteration compiling optimization operation are stored in a performance data base which is used for storing pay expenses and income of all iteration compiling optimization operation to supply to the control module for use.
Description
Technical field
The present invention relates to computer compile technology, relate in particular to a kind of optimization method and optimization system of On-line Iterative compiling.
Background technology
Iteration technique of compiling [1,2,3,4,5,6,7,8,9] is a kind of simple but efficient optimization method.It improves the performance of given computer program in given computer software and hardware environment through the best combination mode of search Compiler Optimization.Yet, be applicable to different computer programs or or even the best optimal combination of the difference of same computer program input possibly be different.Therefore, existing iteration technique of compiling is subject to following problem and is not widely used:
One, find best optimal combination mode to need a large amount of compile and test operations.
Two, the iteration compiling selects the process of optimum combination to depend on the data set of computer program.
Three, the search procedure of iteration compiling has bigger expense, needs to obtain enough big performance benefits and offsets.
Summary of the invention
For addressing the above problem, the object of the invention is problems such as the process of the above-mentioned existing iteration technique of compiling of solution is complicated, and data set relies on, and expense is big for disclosing a kind of method and system of On-line Iterative compiling.
The present invention discloses a kind of optimization method of On-line Iterative compiling, comprises the steps:
Step 100; Control module sends to operational module with the form of the intermediate representation that the source code or the compiler of corresponding program are further optimized; And handle in the process of the data recording that is stored in operational module the intact remaining data record of prediction processing required time at said program parallelization;
Step 200, operational module carries out the operation of iteration compile optimization to said program, and the Optimization result data are fed back to control module;
Step 300 is stored in working time of said program and iteration compile optimization operation institute's time spent in the performance database, and the expense and the income of said performance data library storage total data iteration optimization operation are used for control module;
Said step 100 also comprises:
Step 400 is summarized and is safeguarded the expense and the performance benefits of being brought of the operation of iteration optimization in the said data each run;
Step 500, control module is selected for the part decision-making of the optimum combination that whole operational modules send, and selects the optimal combination of global optimum, recommends whole operational modules as global decisions.
Described On-line Iterative compile optimization method, said step 100 also comprises:
Step 210 when said program is program function, is stored in this program function parallel processing in the process of data recording of operational module, the intact remaining data record of prediction processing required time;
Step 220, the time overhead of control module decision iteration compile optimization operation, and the notice operational module carries out corresponding iteration compile optimization operation.
Described On-line Iterative compile optimization method, said step 100 also comprises:
Step 310 is attempted the different required compilers of compile optimization combination and is deployed on all operational modules;
Step 320; Said program function run-time optimizing system is expanded, make the form of its intermediate representation that can further be optimized the source code of said program function or other by dynamic or static compiler send to all operational modules from control module;
Step 330, the execution of the quantity of decision Optimizing operation and Control and Optimization operation.
Described On-line Iterative compile optimization method, said step 200 also comprises:
Step 410; Said operational module one end; After receiving the time prediction task that control module is distributed, the data recording that is distributed on the operational module is carried out random sampling, form the assessment data record; And in said assessment data record, the program implementation time is carried out partial estimation, pass partial estimation back control module then;
Step 420, said control module are used the linear session forecast model after obtaining the partial estimation that operational module passes back, the total run time of said program function is estimated;
Step 430 is a unit with the optimization time, decides the operation that need carry out how many iteration compilings, and iteration compile optimization task is distributed to the concurrent execution of operational module.
Described On-line Iterative compile optimization method, said step 400 also comprises:
Step 510, with said program function not the expense of homogeneous iteration optimization in service operation sum up and be maintained in the performance savings with the performance benefits of being brought;
Step 520 is controlled the opportunity and the frequency of iteration compiling according to the data message in the performance savings, guarantees that to reach expense only accounts for the target of the predefine small scale of income all the time; The mode of said control is the expense when a part of learning performance benefits through performance database and iteration compilation operations when suitable, then carries out the iteration compilation operations one time, and said part occurrence is set 0% to 100% by the user;
Step 530 from by using the resulting performance savings of high performance optimal combination, distributes a part to be used for the iteration compilation operations, and said part occurrence can be set 0% to 100% by the user;
Step 540 is utilized assessment data to write down performance benefits is assessed; Each operational module is assessed the speed-up ratio of new optimal combination with respect to acquiescence optimal combination on its assessment data record, it is sent it back control module; The performance that control module is utilized the information of speed-up ratio information and pending data recording number total amount to estimate and used this new optimal combination to bring is put aside;
Step 550, based on the performance savings that is predicted, the number of the optimal combination that the control module decision can be assessed on operational module, and the notice operational module is accomplished corresponding operation work.
Described On-line Iterative compile optimization method, said step 200 also comprises:
Step 610, said operational module one end utilizes the form of the intermediate representation that compiler further optimized by dynamic or static compiler the source code of function or other with the combination of different compile optimization to compile and obtains corresponding executable code;
Step 620 is utilized the assessment data record of sampling gained, and the executable code that the compile optimization combination of designated test is obtained carries out performance evaluating to be tested with correctness;
Step 630 after each operational module carries out the iteration compilation operations, can send to control module with the speed-up ratio mark of the optimal combination of being evaluated and tested.
Described On-line Iterative compile optimization method, said step 500 also comprises:
Step 710 after receiving that the higher production of performance that control module is recommended is with optimal combination, can replace with said program function the version of optimization on the operational module.
Described On-line Iterative compile optimization method, said step 100 also comprises:
When step 810, said program are the formed job queue of working procedure, the operation of monitoring task formation, said job queue forms with the task of the working procedure submitted to;
Step 820; Operation is assigned in the processing module cluster and carries out with Performance Evaluation with iteration compiling through job queue; The local file system of each processing module exists with absolute version, or the form that is present in the processing module cluster with a distributed file system part exists;
Step 830 deposits performance database in monitoring the information gathering of said job queue with the said program implementation time.
Described On-line Iterative compile optimization method, said step 810 also comprises:
Step 910, through receiving step 820 assign compiling and Performance Evaluation, test iteration compilation operations with return Optimization result;
Step 920, call compiler with said program from the formal transformation of source code or intermediate code process for the object code that can on processing module, directly carry out; And in said process, need open through methods such as command line parameter or control documents or environmental variances, close or adjust the Optimizing operation that compiler is carried out in transcode process.
Described On-line Iterative compile optimization method, said step 100 also comprises:
Step 1010, the executable object code that compiling is obtained moves on processing module, and measures working time and operation result is verified, when operation, need the sample input be provided for object code;
Step 1020, after the combination of program feature better optimize was found and tested, processing module was used it and is replaced existing production and use optimal combination, makes it can act on the follow-up request to this program, the acquisition more performance;
Step 1030; The part of processing module usability savings is carried out the iteration compilation operations; And through switching the frequency of controlling the generation of iteration compilation operations at conservative mode and radical pattern, said part occurrence is set 0% to 100% by the user.
Described On-line Iterative compile optimization method, said step 830 also comprises:
Step 1110, the required data set of Performance Evaluation test is sampled from a data set pond and is obtained; The inside, data set pond is storing operation to being listed as the employed data set of nearest carrying out of task;
Step 1120 is with collecting the working time that compiles and the Performance Evaluation test is spent and depositing in the performance database, to be used for estimating the expense of iteration compilation operations and the speed-up ratio of the optimal combination of estimating to evaluate and test.
The optimization method of described On-line Iterative compiling, said step 1110 also comprises:
Step 1210 is safeguarded the performance of an incompatible assessment Different Optimization combination of the test set that is made up of the data set of D picked at random and the test of correctness is carried out in the test set that said D is adjustable numeral;
Step 1220, said test set credit union periodically is updated to the data set of the up-to-date use of user, with the continuous variation in time of reply data set.
The present invention discloses a kind of optimization system of On-line Iterative compiling, comprising:
Control module; The form of the intermediate representation that is used for the source code of corresponding program or compiler are further optimized sends to operational module; And handle in the process of the data recording that is stored in operational module the intact remaining data record of prediction processing required time at said program parallelization;
Operational module is used for said program is carried out the operation of iteration compile optimization, and the Optimization result data is fed back to control module;
The performance data library module is used for working time and iteration compile optimization operation institute's time spent of said program are stored in performance database, and the expense and the income of said performance data library storage total data iteration optimization operation are used for control module;
Said control module also comprises:
Performance savings module is used for summarizing and safeguarding the expense of said data each run iteration optimization operation and the performance benefits of being brought;
The global decisions module is used for control module and selects for the part decision-making of the optimum combination of whole operational modules transmissions, selects the optimal combination of global optimum, recommends whole operational modules as global decisions.
Described On-line Iterative compile optimization system, said control module also comprises:
When said data are program function, this program function parallel processing is stored in the process of data recording of operational module the intact remaining data record of prediction processing required time; The time overhead of control module decision iteration compile optimization operation, and the notice operational module carries out corresponding iteration compile optimization operation.
Described On-line Iterative compile optimization system, said control module also comprises:
The compiler deployment module is used to attempt the different required compilers of compile optimization combination and is deployed in all operational modules;
The expansion sending module; Be used for said program function run-time optimizing system is expanded, make the form of its intermediate representation that can further be optimized the source code of said program function or other by dynamic or static compiler send to all operational modules from control module;
Iteration compiling total control module is used to determine the quantity of Optimizing operation and the execution of Control and Optimization operation.
Described On-line Iterative compile optimization system, said operational module also comprises:
Sampling module; Be used for said operational module one end; After receiving the time prediction task that control module is distributed, the data recording that is distributed on the operational module is carried out random sampling, form the assessment data record; And in said assessment data record, the program implementation time is carried out partial estimation, pass partial estimation back control module then;
The linear session prediction module is used for said control module after obtaining the partial estimation that operational module passes back, uses the linear session prediction, and the total run time of said program function is estimated;
Iteration compiling budget module, being used for the optimization time is unit, decides the operation that need carry out how many iteration compilings, and iteration compile optimization task is distributed to the concurrent execution of operational module.
Described On-line Iterative compile optimization system, said performance savings module also comprises:
Optimize to sum up module, be used for said program function not the expense of homogeneous iteration optimization operation in service sum up and be maintained in the performance savings with the performance benefits of being brought;
The optimal control module is used for according to the data message of performance savings the opportunity and the frequency of iteration compiling being controlled, and guarantees that to reach expense only accounts for the target of the predefine small scale of income all the time; The mode of said control is the expense when a part of learning performance benefits through performance database and iteration compilation operations when suitable, then carries out the iteration compilation operations one time, and said part occurrence is set 0% to 100% by the user;
The savings distribution module is used for distributing a part to be used for the iteration compilation operations from by using the resulting performance savings of high performance optimal combination, and the occurrence of said part can be set 0% to 100% by the user;
The savings estimation module is used to utilize assessment data to write down performance benefits is assessed; Each operational module is assessed the speed-up ratio of new optimal combination with respect to acquiescence optimal combination on its assessment data record, it is sent it back control module; The performance that control module is utilized the information of speed-up ratio information and pending data recording number total amount to estimate and used this new optimal combination to bring is put aside;
Optimize the quantity module, be used for based on the performance savings that is predicted, the number of the optimal combination that the control module decision can be assessed on operational module, and the notice operational module is accomplished corresponding operation work.
Described On-line Iterative compile optimization system, said operational module also comprises:
The actuating code generation module is used to utilize compiler with different compile optimization combinations, and the form of the intermediate representation that the source code of function or other are further optimized by dynamic or static compiler compiles and obtains corresponding executable code;
Optimal combination evaluation and test module is utilized the assessment data record of sampling gained, and the executable code that the compile optimization combination of designated test is obtained carries out performance evaluating to be tested with correctness;
The mark module after being used for each operational module and carrying out the iteration compilation operations, can send to control module with the speed-up ratio mark of the optimal combination of being evaluated and tested.
Described On-line Iterative compile optimization system, said global decisions module also comprises:
Optimization result schistosity module is used on the operational module after receiving that the higher production of performance that control module is recommended is with optimal combination, can said program function being replaced with the version of optimization.
Described On-line Iterative compile optimization system, said control module also comprises:
Processing module, when being used for said data and being the formed job queue of working procedure, the operation of monitoring task formation, said job queue, the task of the working procedure of submitting to the user forms;
The operation dispatch module; Be used for that operation is assigned to processing module cluster and carries out with Performance Evaluation with iteration compiling through job queue; The local file system of each processing module exists with absolute version, or the form that is present in the processing module cluster with a distributed file system part exists;
Monitoring modular is used for depositing performance database in monitoring the information gathering of said job queue with the said program implementation time.
Described On-line Iterative compile optimization system, said processing module also comprises:
Iteration compilation operations module, be used for through receive that the operation dispatch module assigns compiling and Performance Evaluation, test iteration compilation operations with return Optimization result;
Collector, be used to call compiler with said program from the formal transformation of source code or intermediate code process for the object code that can on processing module, directly carry out; And in said process, need open through systems such as command line parameter or control documents or environmental variances, close or adjust the Optimizing operation that compiler is carried out in transcode process.
Described On-line Iterative compile optimization system, said control module also comprises:
Optimize the evaluation and test module, be used for the executable object code that compiling obtains is moved on processing module, and measure working time and operation result is verified, when operation, need the sample input be provided for object code;
The optimized choice module is used for after the combination of program feature better optimize is found and tested, and processing module is used it and replaced existing production and use optimal combination, makes it can act on the follow-up request to this program, the acquisition more performance;
Mode selection module; A part that is used for processing module usability savings is carried out the iteration compilation operations; And through switching the frequency of controlling the generation of iteration compilation operations at conservative mode and radical pattern, said part occurrence is set 0% to 100% by the user.
Described On-line Iterative compile optimization system, said monitoring modular also comprises:
Data set pond module is used for the required data set of Performance Evaluation test and obtains from a data set pond sampling; The inside, data set pond is storing operation to being listed as the employed data set of nearest carrying out of task;
The performance inventory module is used for compiling and Performance Evaluation are tested collection working time of being spent and deposited performance database in, to be used for estimating the expense of iteration compilation operations and the speed-up ratio of the optimal combination of estimating to evaluate and test.
The optimization system of described On-line Iterative compiling, said data set pond module also comprises:
The test set compound module is used to safeguard the performance of an incompatible assessment Different Optimization combination of test set that is made up of the data set of D picked at random and the test of correctness is carried out in the test set that said D is adjustable numeral; Said test set credit union periodically is updated to the data set of the up-to-date use of user, with the continuous variation in time of reply data set.
The present invention acts on unit or the multi-computer system, and its beneficial effect is:
1, automatically the iteration compiling is searched for step application such as needed compiling of optimal combination and performance test in the computer program that on unit or multimachine, moves or the part of program; Reach iteration compile optimization process to the transparent effect of the user of computer program;
2, the program of being optimized is monitored under the ruuning situation on the computer system, according to the income that is obtained after the iteration compile optimization method, the execution opportunity and the frequency of the operation of control iteration compile optimization; Reach the expense of iteration compiling is controlled, make expense less than income or less than the effect of user's setting value;
3, in the production environment that computer program really is used, program is carried out iteration optimization; Reach the real operation of computer program hardware and the data set of use on it carried out adaptive optimization;
4, whole iteration optimization process is transparent to the final user of computer program; Reach automatically and carry out, not the effect of interference user.
The present invention can be full automatic; In the environment that really is performed of computer program; To a specific software and hardware and a true group data set that uses, application programs is carried out the On-line Iterative compile optimization, reaches to improve application program execution efficient; And the Control and Optimization expense makes it less than optimizing income automatically, reduces the effect of total system power consumption.
Description of drawings
Fig. 1. the On-line Iterative compiling system is applied to the embodiment of MapReduce;
Fig. 2. the On-line Iterative compiling system is applied to the embodiment of high-throughput services device program;
Fig. 3. the evolution exemplary plot of performance savings;
Fig. 4. radical pattern and conservative mode switch figure.
Embodiment
Provide embodiment of the present invention below, the present invention has been made detailed description in conjunction with accompanying drawing.
The present invention discloses a kind of optimization method of On-line Iterative compiling, comprises the steps:
Step 100; Control module sends to operational module with the form of the intermediate representation that the source code or the compiler of corresponding program are further optimized; And handle in the process of the data recording that is stored in operational module the intact remaining data record of prediction processing required time at said program parallelization;
Step 200, operational module carries out the operation of iteration compile optimization to said program, and the Optimization result data are fed back to control module;
Step 300 is stored in working time of said program and iteration compile optimization operation institute's time spent in the performance database, and the expense and the income of said performance data library storage total data iteration optimization operation are used for control module;
Said step 100 also comprises:
Step 400 is summarized and is safeguarded the expense and the performance benefits of being brought of the operation of iteration optimization in the said data each run;
Step 500, control module is selected for the part decision-making of the optimum combination that whole operational modules send, and selects the optimal combination of global optimum, recommends whole operational modules as global decisions.
Described On-line Iterative compile optimization method, said step 100 also comprises:
Step 210 when said program is program function, is stored in this program function parallel processing in the process of data recording of operational module, the intact remaining data record of prediction processing required time;
Step 220, the time overhead of control module decision iteration compile optimization operation, and the notice operational module carries out corresponding iteration compile optimization operation.
Described On-line Iterative compile optimization method, said step 100 also comprises:
Step 310 is attempted the different required compilers of compile optimization combination and is deployed on all operational modules;
Step 320; Said program function run-time optimizing system is expanded, make the form of its intermediate representation that can further be optimized the source code of said program function or other by dynamic or static compiler send to all operational modules from control module;
Step 330, the execution of the quantity of decision Optimizing operation and Control and Optimization operation.
Described On-line Iterative compile optimization method, said step 200 also comprises:
Step 410; Said operational module one end; After receiving the time prediction task that control module is distributed, the data recording that is distributed on the operational module is carried out random sampling, form the assessment data record; And in said assessment data record, the program implementation time is carried out partial estimation, pass partial estimation back control module then;
Step 420, said control module are used the linear session forecast model after obtaining the partial estimation that operational module passes back, the total run time of said program function is estimated;
Step 430 is a unit with the optimization time, decides the operation that need carry out how many iteration compilings, and iteration compile optimization task is distributed to the concurrent execution of operational module.
Described On-line Iterative compile optimization method, said step 400 also comprises:
Step 510, with said program function not the expense of homogeneous iteration optimization in service operation sum up and be maintained in the performance savings with the performance benefits of being brought;
Step 520 is controlled the opportunity and the frequency of iteration compiling according to the data message in the performance savings, guarantees that to reach expense only accounts for the target of the predefine small scale of income all the time; The mode of said control is the expense when a part of learning performance benefits through performance database and iteration compilation operations when suitable, then carries out the iteration compilation operations one time, and said part occurrence is set 0% to 100% by the user;
Step 530 from by using the resulting performance savings of high performance optimal combination, distributes a part to be used for the iteration compilation operations, and said part occurrence can be set 0% to 100% by the user;
Step 540 is utilized assessment data to write down performance benefits is assessed; Each operational module is assessed the speed-up ratio of new optimal combination with respect to acquiescence optimal combination on its assessment data record, it is sent it back control module; The performance that control module is utilized the information of speed-up ratio information and pending data recording number total amount to estimate and used this new optimal combination to bring is put aside;
Step 550, based on the performance savings that is predicted, the number of the optimal combination that the control module decision can be assessed on operational module, and the notice operational module is accomplished corresponding operation work.
Described On-line Iterative compile optimization method, said step 200 also comprises:
Step 610, said operational module one end utilizes the form of the intermediate representation that compiler further optimized by dynamic or static compiler the source code of function or other with the combination of different compile optimization to compile and obtains corresponding executable code;
Step 620 is utilized the assessment data record of sampling gained, and the executable code that the compile optimization combination of designated test is obtained carries out performance evaluating to be tested with correctness;
Step 630 after each operational module carries out the iteration compilation operations, can send to control module with the speed-up ratio mark of the optimal combination of being evaluated and tested.
Described On-line Iterative compile optimization method, said step 500 also comprises:
Step 710 after receiving that the higher production of performance that control module is recommended is with optimal combination, can replace with said program function the version of optimization on the operational module.
Described On-line Iterative compile optimization method, said step 100 also comprises:
When step 810, said program are the formed job queue of working procedure, the operation of monitoring task formation, said job queue forms with the task of the working procedure submitted to;
Step 820; Operation is assigned in the processing module cluster and carries out with Performance Evaluation with iteration compiling through job queue; The local file system of each processing module exists with absolute version, or the form that is present in the processing module cluster with a distributed file system part exists;
Step 830 deposits performance database in monitoring the information gathering of said job queue with the said program implementation time.
Described On-line Iterative compile optimization method, said step 810 also comprises:
Step 910, through receiving step 820 assign compiling and Performance Evaluation, test iteration compilation operations with return Optimization result;
Step 920, call compiler with said program from the formal transformation of source code or intermediate code process for the object code that can on processing module, directly carry out; And in said process, need open through methods such as command line parameter or control documents or environmental variances, close or adjust the Optimizing operation that compiler is carried out in transcode process.
Described On-line Iterative compile optimization method, said step 100 also comprises:
Step 1010, the executable object code that compiling is obtained moves on processing module, and measures working time and operation result is verified, when operation, need the sample input be provided for object code;
Step 1020, after the combination of program feature better optimize was found and tested, processing module was used it and is replaced existing production and use optimal combination, makes it can act on the follow-up request to this program, the acquisition more performance;
Step 1030; The part of processing module usability savings is carried out the iteration compilation operations; And through switching the frequency of controlling the generation of iteration compilation operations at conservative mode and radical pattern, said part occurrence is set 0% to 100% by the user.
Described On-line Iterative compile optimization method, said step 830 also comprises:
Step 1110, the required data set of Performance Evaluation test is sampled from a data set pond and is obtained; The inside, data set pond is storing operation to being listed as the employed data set of nearest carrying out of task;
Step 1120 is with collecting the working time that compiles and the Performance Evaluation test is spent and depositing in the performance database, to be used for estimating the expense of iteration compilation operations and the speed-up ratio of the optimal combination of estimating to evaluate and test.
The optimization method of described On-line Iterative compiling, said step 1110 also comprises:
Step 1210 is safeguarded the performance of an incompatible assessment Different Optimization combination of the test set that is made up of the data set of D picked at random and the test of correctness is carried out in the test set that said D is adjustable numeral;
Step 1220, said test set credit union periodically is updated to the data set of the up-to-date use of user, with the continuous variation in time of reply data set.
The present invention discloses a kind of optimization system of On-line Iterative compiling, comprising:
Control module; The form of the intermediate representation that is used for the source code of corresponding program or compiler are further optimized sends to operational module; And handle in the process of the data recording that is stored in operational module the intact remaining data record of prediction processing required time at said program parallelization;
Operational module is used for said program is carried out the operation of iteration compile optimization, and the Optimization result data is fed back to control module;
The performance data library module is used for working time and iteration compile optimization operation institute's time spent of said program are stored in performance database, and the expense and the income of said performance data library storage total data iteration optimization operation are used for control module;
Said control module also comprises:
Performance savings module is used for summarizing and safeguarding the expense of said data each run iteration optimization operation and the performance benefits of being brought;
The global decisions module is used for control module and selects for the part decision-making of the optimum combination of whole operational modules transmissions, selects the optimal combination of global optimum, recommends whole operational modules as global decisions.
Described On-line Iterative compile optimization system, said control module also comprises:
When said data are program function, this program function parallel processing is stored in the process of data recording of operational module the intact remaining data record of prediction processing required time; The time overhead of control module decision iteration compile optimization operation, and the notice operational module carries out corresponding iteration compile optimization operation.
Described On-line Iterative compile optimization system, said control module also comprises:
The compiler deployment module is used to attempt the different required compilers of compile optimization combination and is deployed in all operational modules;
The expansion sending module; Be used for said program function run-time optimizing system is expanded, make the form of its intermediate representation that can further be optimized the source code of said program function or other by dynamic or static compiler send to all operational modules from control module;
Iteration compiling total control module is used to determine the quantity of Optimizing operation and the execution of Control and Optimization operation.
Described On-line Iterative compile optimization system, said operational module also comprises:
Sampling module; Be used for said operational module one end; After receiving the time prediction task that control module is distributed, the data recording that is distributed on the operational module is carried out random sampling, form the assessment data record; And in said assessment data record, the program implementation time is carried out partial estimation, pass partial estimation back control module then;
The linear session prediction module is used for said control module after obtaining the partial estimation that operational module passes back, uses the linear session prediction, and the total run time of said program function is estimated;
Iteration compiling budget module, being used for the optimization time is unit, decides the operation that need carry out how many iteration compilings, and iteration compile optimization task is distributed to the concurrent execution of operational module.
Described On-line Iterative compile optimization system, said performance savings module also comprises:
Optimize to sum up module, be used for said program function not the expense of homogeneous iteration optimization operation in service sum up and be maintained in the performance savings with the performance benefits of being brought;
The optimal control module is used for according to the data message of performance savings the opportunity and the frequency of iteration compiling being controlled, and guarantees that to reach expense only accounts for the target of the predefine small scale of income all the time; The mode of said control is the expense when a part of learning performance benefits through performance database and iteration compilation operations when suitable, then carries out the iteration compilation operations one time, and said part occurrence is set 0% to 100% by the user;
The savings distribution module is used for distributing a part to be used for the iteration compilation operations from by using the resulting performance savings of high performance optimal combination, and the occurrence of said part can be set 0% to 100% by the user;
The savings estimation module is used to utilize assessment data to write down performance benefits is assessed; Each operational module is assessed the speed-up ratio of new optimal combination with respect to acquiescence optimal combination on its assessment data record, it is sent it back control module; The performance that control module is utilized the information of speed-up ratio information and pending data recording number total amount to estimate and used this new optimal combination to bring is put aside;
Optimize the quantity module, be used for based on the performance savings that is predicted, the number of the optimal combination that the control module decision can be assessed on operational module, and the notice operational module is accomplished corresponding operation work.
Described On-line Iterative compile optimization system, said operational module also comprises:
The actuating code generation module is used to utilize compiler with different compile optimization combinations, and the form of the intermediate representation that the source code of function or other are further optimized by dynamic or static compiler compiles and obtains corresponding executable code;
Optimal combination evaluation and test module is utilized the assessment data record of sampling gained, and the executable code that the compile optimization combination of designated test is obtained carries out performance evaluating to be tested with correctness;
The mark module after being used for each operational module and carrying out the iteration compilation operations, can send to control module with the speed-up ratio mark of the optimal combination of being evaluated and tested.
Described On-line Iterative compile optimization system, said global decisions module also comprises:
Optimization result schistosity module is used on the operational module after receiving that the higher production of performance that control module is recommended is with optimal combination, can said program function being replaced with the version of optimization.
Described On-line Iterative compile optimization system, said control module also comprises:
Processing module, when being used for said data and being the formed job queue of working procedure, the operation of monitoring task formation, said job queue, the task of the working procedure of submitting to the user forms;
The operation dispatch module; Be used for that operation is assigned to processing module cluster and carries out with Performance Evaluation with iteration compiling through job queue; The local file system of each processing module exists with absolute version, or the form that is present in the processing module cluster with a distributed file system part exists;
Monitoring modular is used for depositing performance database in monitoring the information gathering of said job queue with the said program implementation time.
Described On-line Iterative compile optimization system, said processing module also comprises:
Iteration compilation operations module, be used for through receive that the operation dispatch module assigns compiling and Performance Evaluation, test iteration compilation operations with return Optimization result;
Collector, be used to call compiler with said program from the formal transformation of source code or intermediate code process for the object code that can on processing module, directly carry out; And in said process, need open through systems such as command line parameter or control documents or environmental variances, close or adjust the Optimizing operation that compiler is carried out in transcode process.
Described On-line Iterative compile optimization system, said control module also comprises:
Optimize the evaluation and test module, be used for the executable object code that compiling obtains is moved on processing module, and measure working time and operation result is verified, when operation, need the sample input be provided for object code;
The optimized choice module is used for after the combination of program feature better optimize is found and tested, and processing module is used it and replaced existing production and use optimal combination, makes it can act on the follow-up request to this program, the acquisition more performance;
Mode selection module; A part that is used for processing module usability savings is carried out the iteration compilation operations; And through switching the frequency of controlling the generation of iteration compilation operations at conservative mode and radical pattern, said part occurrence is set 0% to 100% by the user.
Described On-line Iterative compile optimization system, said monitoring modular also comprises:
Data set pond module is used for the required data set of Performance Evaluation test and obtains from a data set pond sampling; The inside, data set pond is storing operation to being listed as the employed data set of nearest carrying out of task;
The performance inventory module is used for compiling and Performance Evaluation are tested collection working time of being spent and deposited performance database in, to be used for estimating the expense of iteration compilation operations and the speed-up ratio of the optimal combination of estimating to evaluate and test.
The optimization system of described On-line Iterative compiling, said data set pond module also comprises:
The test set compound module is used to safeguard the performance of an incompatible assessment Different Optimization combination of test set that is made up of the data set of D picked at random and the test of correctness is carried out in the test set that said D is adjustable numeral; Said test set credit union periodically is updated to the data set of the up-to-date use of user, with the continuous variation in time of reply data set.
Fig. 1 is an On-line Iterative compile optimization of the present invention system; Be designated hereinafter simply as IODC 8 (Iterative Optimization for the Data Center; Be applied to the iteration compiling system of data center), be applied to an embodiment of data center's MapReduce (mapping stipulations) application program commonly used.In environment when IODC 8 is integrated in the MapReduce operation.
The environment in MapReduce when operation does, Map that the user write and Reduce function is distributed to removes parallel processing lot of data record in the computer cluster.These data recording are stored and are managed by a distributed file system usually.The main logic of MapReduce runtime environment is arranged in control module 17, some operational module 18 concurrent processing data in the control module 17 control clusters.The MapReduce application program that the user is write as that said main logic refers to is distributed to the step of the needed control of parallel processing mass data in the computer cluster.
In order to support the iteration compile optimization, need expand MapReduce run-time optimizing system, said expansion refers to the function that system is expanded and adds.And the literary style of SQL need not change.Make it be distributed to all operational modules 18 to the source code of Map and two User-Defined Functions of Reduce from control module 17.Make up 24 required compilers and be deployed on all operational modules 18 through attempting different compile optimizations.At control module 17 these ends, IODC 8 uses a linear session forecast model 20, and the total run time of MapReduce program is estimated.Said linear session forecast model is exactly the expression formula of a linearity, as: time=a* data volume+b.Expression formula with a linearity is converted into the time to data volume exactly.Before use, need to confirm the value of a and b.Method is the value of surveying two groups of times and data volume, and substitution expression formula then forms a linear equation in two unknowns group, finds the solution promptly to get.
In view of the above, IODC 8 decisions need be carried out the operation (is unit with optimization time 23) of how many iteration compilings and iteration compile optimization task is distributed to the 18 concurrent execution of one group of operational module.Control module 17 is chosen the best optimal combination 24 of the overall situation after combining the part decision-making 22 about optimum combination that all working module 18 sends, and recommends all working module 18 as global decisions 21.
Information all is collected and is stored in the performance database 9 during the needed operation of all IODC 8.IODC8 with same MapReduce program not the expense of homogeneous iteration optimization in service operation sum up and be maintained in the performance savings 11 with the performance benefits of being brought.IODC 8 comes the opportunity and the frequency of iteration compiling to control according to the information in the performance savings 11, guarantees that to reach expense only accounts for the target of a predefine small scale of income all the time.
At operational module 18 these ends; IODC 8 is after receiving the iteration compiler task that control module 17 is distributed; Carry out random sampling to being distributed in local data recording; Form assessment data record 201, and the program implementation time is carried out partial estimation above that, performance evaluating and correctness test are carried out in the compile optimization combination 24 of designated test.After every operational module 18 carries out the iteration compilation operations, can the speed-up ratio mark 25 of the optimal combination of being evaluated and tested 24 be sent to control module 17.IODC 8 on the operational module 18 after receiving that the higher production of performance that control module 17 is recommended is with optimal combination 26, the version that can the Map and the Reduce function of this locality be replaced with this Combinatorial Optimization of usefulness.
For the point of any time in the implementation repeatedly of MapReduce program obtains net proceeds all the time, 8 of IODC can distribute sub-fraction to be used for the iteration compilation operations from by using the high performance optimal combination 24 resulting performance savings 11.For this reason, it must be put aside 11 to performance and accurately evaluates and tests.IODC 8 utilizes assessment data record 201 to come performance benefits is assessed.Each operational module 18 is assessed the speed-up ratio of new optimal combination with respect to acquiescence optimal combination on its assessment data record 201, it is sent it back control module 17.The performance that control module 17 is utilized the information of speed-up ratio information and pending data recording number total amount to estimate and used this new optimal combination to bring puts aside 11.This is to accomplish through linear extrapolation, in Fig. 1, is called as linear session forecast model 20.Based on the performance savings 11 that is predicted, the number of the optimal combination 24 that control module 17 decisions can be assessed on operational module 18, and notice operational module 18 is accomplished corresponding work.
Be exactly the cardinal rule in this process is avoided excessive expense total the sub-fraction (P%) of the performance that only spends savings 11 is done the operation of iteration compiling.
Fig. 3 example the change procedure of performance savings 11.When beginning most to have no performance savings 11, the sub-fraction of total execution time of program that IODC 8 costs estimate goes to carry out the iteration compilation operations.In case the better optimize combination is found 28; (P represents the numeral that can transfer to the P% of the performance savings 11 that the better optimize combination 28 that use is found is brought; As 5%, 10%...) be used to do more iteration compilation operations, purpose is to find the higher option of performance to make up.This just uses, and IODC 8 can possess most performance savings 11.Under the worst situation, there is not the better optimize combination to be found 29, IODC 8 also only can cause with respect to the almost negligible loss of total run time.
IODC 8 will periodically spend the performance savings 11 of a part and do the iteration compilation operations.The iteration compilation operations is accomplished by operational module 18.Operational module 18 can one of evaluation and test arrive a plurality of optimal combinations in the iteration compilation operations.
The evaluation and test operation has dual task: 1, find than the existing best higher new optimal combination of composite behaviour; 2, evaluation and test used optimal combination 24 in the past again.
Purpose is that under the continuous situation about changing of the data set of program, the system performance of keeping that can be more stable is in higher state.
IODC 8 not only can be applied to separately within the once execution of MapReduce program, can also be applied between same application repeatedly carries out.For this reason; 8 of IODC need constantly note the high-performance optimal combination of finding; And in the follow-up performance of preferentially assessing them in the caused iteration compiling of repeatedly carrying out of program; Just can reach the optimization information of utilizing of striding execution, and constantly growth property can put aside 11 with the more optimal combinations of assessment to find the higher combination of performance.
Fig. 2 is the embodiment that On-line Iterative compiling system IODC 8 of the present invention is applied to the server-side application of high-throughput.In the present embodiment, IODC 8 and a job queue system 5 integrate.
This job queue 5 is received as the task requests of user's working procedure 4, and concurrent task is distributed on the cluster 1 that is made up of many computer nodes 2 and carries out.In this scene, every computer node 2 has 3, one local file systems 100 of one or more central processing units.This document system can be independently, or the part of the distributed file system of having done to exist between many computer nodes 2.Program 4 be present in the local file system 100 with the form of the intermediate representation that the form of executable file and source code or other can further be optimized by dynamic or static compiler.(compiler is a newly-increased module to the required compiler of iteration compiling, is used for program function is optimized.) also be present in the local file system 100 of every computer node 2.IODC 8 is positioned at wherein on the computing machine.IODC 8 treats that with the execution time 15 of program 4 information gathering deposits performance database 9 in to row 5 through monitoring task.Based on the data in the performance database 9, corresponding program 4 and timing optimization are optimized in IODC 8 decisions.
IODC 8 dispatches the steps necessary of iteration compilation operations such as compiling 7 and Performance Evaluation test 6 through operation allocator 16.This steps necessary does, compiling 7 be meant call compiler with program from the formal transformation of source code or intermediate code process for the object code that can directly carry out on computers.In this process, need open, close or adjust the Optimizing operation that compiler is carried out in transcode process through methods such as command line parameter or control documents or environmental variances.Performance Evaluation test 6 is meant that the executable object code that compiling is obtained moves on computers, and measures working time and operation result is verified.When operation, need the sample input be provided for object code.Performance Evaluation is tested 6 required data sets and from a data set pond 12, is sampled and obtain.12 the insides, data set pond are storing the employed data set of task that operation was carried out the row system recently.IODC 8 will compile and collect and deposit in the performance database 97 working times 15 of being spent with Performance Evaluation test 6, to be used for estimating the expense of iteration compilation operations and the speed-up ratio of the optimal combination of estimating to evaluate and test 24.After a program feature better optimize combination was found and tested, IODC 8 used it and replaces existing production with optimal combination 10, makes it can act on the follow-up request to this program, obtains more performance.8 of IODC can usability savings 11 sub-fraction carry out the iteration compilation operations, and control the frequency that the iteration compilation operations takes place through switching at conservative mode 13 and radical pattern 14.
IODC 8 safeguard one by D (D is an adjustable numeral, as 3,5, the 10...) performance of the incompatible assessment Different Optimization combination of test set that constitutes of the data set of individual picked at random and they are carried out the test of correctness.This test set credit union periodically is updated to the data set of the up-to-date use of user, with the continuous variation in time of reply data set.D is set as a less value 3, to avoid too high optimization expense.We use t-check, and said t-check is a kind of check commonly used in the statistics, also claim student t check (Student ' s t test), are mainly used in sample content less (for example n＜30), the normal distribution data that population standard deviation σ is unknown.The t-check is the method for inspection that is used for two mean value difference degrees of small sample (sample size is less than 30).It is to infer the probability that difference takes place with the t-distribution theory, thereby whether the difference of judging two averages is remarkable.Whether test two performance gap between the optimal combination remarkable.Have only when new optimal combination to be issued to 1% performance boost, just can be used to replace current production with optimal combination 10 at 95% confidence index.
In this scene, IODC 8 can be by the number of times of user's request, so can't estimate the performance savings that an optimal combination can bring in advance because can't foresee a program.Therefore, need after the each user's request of response, reappraise about the decision-making of iteration compiling.At any time point; IODC 8 can be used for carrying out the iteration compilation operations budget what not only by the accumulation decision of producing the performance savings 11 in one period that is replaced with optimal combination since the last time, also make up found frequency and determine by better optimize at no distant date.Higher when the found frequency ratio of better optimize combination, we should drop into more budget and carry out the iteration compilation operations with the expectation higher performance boost of acquisition faster.Under opposite situation, what we should more guard carries out the iteration compilation operations to keep more income.
Like Fig. 4, IODC 8 automaticallyes switch between conservative mode 13 and radical pattern 14.Unique difference between these two patterns is that the ratio P of spendable performance savings 11 is different.P=5% in conservative mode 13, P=50% in radical pattern 14.Ratio P is not meant the ratio of the performance savings 11 that starts anew all, but can put aside from the cumulative bad that last mode switch begins.Therefore, no matter how pattern is switched, and total performance savings 11 can guarantee to increase all the time.
Those skilled in the art can also carry out various modifications to above content under the condition that does not break away from the definite the spirit and scope of the present invention of claims.Therefore scope of the present invention is not limited in above explanation, but confirm by the scope of claims.
Claims (24)
1. the optimization method of an On-line Iterative compiling is characterized in that, comprises the steps:
Step 100; Control module sends to operational module with the form of the intermediate representation that the source code or the compiler of corresponding program are further optimized; And handle in the process of the data recording that is stored in operational module the intact remaining data record of prediction processing required time at said program parallelization;
Step 200, operational module carries out the operation of iteration compile optimization to said program, and the Optimization result data are fed back to control module;
Step 300 is stored in working time of said program and iteration compile optimization operation institute's time spent in the performance database, and the expense and the income of said performance data library storage total data iteration optimization operation are used for control module;
Said step 100 also comprises:
Step 400 is summarized and is safeguarded the expense and the performance benefits of being brought of the operation of iteration optimization in the said data each run;
Step 500, control module is selected for the part decision-making of the optimum combination that whole operational modules send, and selects the optimal combination of global optimum, recommends whole operational modules as global decisions.
2. On-line Iterative compile optimization method as claimed in claim 1 is characterized in that said step 100 also comprises:
Step 210 when said program is program function, is stored in this program function parallel processing in the process of data recording of operational module, the intact remaining data record of prediction processing required time;
Step 220, the time overhead of control module decision iteration compile optimization operation, and the notice operational module carries out corresponding iteration compile optimization operation.
3. On-line Iterative compile optimization method as claimed in claim 2 is characterized in that said step 100 also comprises:
Step 310 is attempted the different required compilers of compile optimization combination and is deployed on all operational modules;
Step 320; Said program function run-time optimizing system is expanded, make the form of its intermediate representation that can further be optimized the source code of said program function or other by dynamic or static compiler send to all operational modules from control module;
Step 330, the execution of the quantity of decision Optimizing operation and Control and Optimization operation.
4. On-line Iterative compile optimization method as claimed in claim 2 is characterized in that said step 200 also comprises:
Step 410; Said operational module one end; After receiving the time prediction task that control module is distributed, the data recording that is distributed on the operational module is carried out random sampling, form the assessment data record; And in said assessment data record, the program implementation time is carried out partial estimation, pass partial estimation back control module then;
Step 420, said control module are used the linear session forecast model after obtaining the partial estimation that operational module passes back, the total run time of said program function is estimated;
Step 430 is a unit with the optimization time, decides the operation that need carry out how many iteration compilings, and iteration compile optimization task is distributed to the concurrent execution of operational module.
5. On-line Iterative compile optimization method as claimed in claim 2 is characterized in that said step 400 also comprises:
Step 510, with said program function not the expense of homogeneous iteration optimization in service operation sum up and be maintained in the performance savings with the performance benefits of being brought;
Step 520 is controlled the opportunity and the frequency of iteration compiling according to the data message in the performance savings, guarantees that to reach expense only accounts for the target of the predefine small scale of income all the time; The mode of said control is the expense when a part of learning performance benefits through performance database and iteration compilation operations when suitable, then carries out the iteration compilation operations one time, and said part occurrence is set 0% to 100% by the user;
Step 530 from by using the resulting performance savings of high performance optimal combination, distributes a part to be used for the iteration compilation operations, and said part occurrence can be set 0% to 100% by the user;
Step 540 is utilized assessment data to write down performance benefits is assessed; Each operational module is assessed the speed-up ratio of new optimal combination with respect to acquiescence optimal combination on its assessment data record, it is sent it back control module; The performance that control module is utilized the information of speed-up ratio information and pending data recording number total amount to estimate and used this new optimal combination to bring is put aside;
Step 550, based on the performance savings that is predicted, the number of the optimal combination that the control module decision can be assessed on operational module, and the notice operational module is accomplished corresponding operation work.
6. On-line Iterative compile optimization method as claimed in claim 2 is characterized in that said step 200 also comprises:
Step 610, said operational module one end utilizes the form of the intermediate representation that compiler further optimized by dynamic or static compiler the source code of function or other with the combination of different compile optimization to compile and obtains corresponding executable code;
Step 620 is utilized the assessment data record of sampling gained, and the executable code that the compile optimization combination of designated test is obtained carries out performance evaluating to be tested with correctness;
Step 630 after each operational module carries out the iteration compilation operations, can send to control module with the speed-up ratio mark of the optimal combination of being evaluated and tested.
7. On-line Iterative compile optimization method as claimed in claim 2 is characterized in that said step 500 also comprises:
Step 710 after receiving that the higher production of performance that control module is recommended is with optimal combination, can replace with said program function the version of optimization on the operational module.
8. On-line Iterative compile optimization method as claimed in claim 1 is characterized in that said step 100 also comprises:
When step 810, said program are the formed job queue of working procedure, the operation of monitoring task formation, said job queue forms with the task of the working procedure submitted to;
Step 820; Operation is assigned in the processing module cluster and carries out with Performance Evaluation with iteration compiling through job queue; The local file system of each processing module exists with absolute version, or the form that is present in the processing module cluster with a distributed file system part exists;
Step 830 deposits performance database in monitoring the information gathering of said job queue with the said program implementation time.
9. On-line Iterative compile optimization method as claimed in claim 8 is characterized in that said step 810 also comprises:
Step 910, through receiving step 820 assign compiling and Performance Evaluation, test iteration compilation operations with return Optimization result;
Step 920, call compiler with said program from the formal transformation of source code or intermediate code process for the object code that can on processing module, directly carry out; And in said process, need open through methods such as command line parameter or control documents or environmental variances, close or adjust the Optimizing operation that compiler is carried out in transcode process.
10. On-line Iterative compile optimization method as claimed in claim 8 is characterized in that said step 100 also comprises:
Step 1010, the executable object code that compiling is obtained moves on processing module, and measures working time and operation result is verified, when operation, need the sample input be provided for object code;
Step 1020, after the combination of program feature better optimize was found and tested, processing module was used it and is replaced existing production and use optimal combination, makes it can act on the follow-up request to this program, the acquisition more performance;
Step 1030; The part of processing module usability savings is carried out the iteration compilation operations; And through switching the frequency of controlling the generation of iteration compilation operations at conservative mode and radical pattern, said part occurrence is set 0% to 100% by the user.
11. On-line Iterative compile optimization method as claimed in claim 8 is characterized in that said step 830 also comprises:
Step 1110, the required data set of Performance Evaluation test is sampled from a data set pond and is obtained; The inside, data set pond is storing operation to being listed as the employed data set of nearest carrying out of task;
Step 1120 is with collecting the working time that compiles and the Performance Evaluation test is spent and depositing in the performance database, to be used for estimating the expense of iteration compilation operations and the speed-up ratio of the optimal combination of estimating to evaluate and test.
12. the optimization method of On-line Iterative compiling as claimed in claim 11 is characterized in that said step 1110 also comprises:
Step 1210 is safeguarded the performance of an incompatible assessment Different Optimization combination of the test set that is made up of the data set of D picked at random and the test of correctness is carried out in the test set that said D is adjustable numeral;
Step 1220, said test set credit union periodically is updated to the data set of the up-to-date use of user, with the continuous variation in time of reply data set.
13. the optimization system of an On-line Iterative compiling is characterized in that, comprising:
Control module; The form of the intermediate representation that is used for the source code of corresponding program or compiler are further optimized sends to operational module; And handle in the process of the data recording that is stored in operational module the intact remaining data record of prediction processing required time at said program parallelization;
Operational module is used for said program is carried out the operation of iteration compile optimization, and the Optimization result data is fed back to control module;
The performance data library module is used for working time and iteration compile optimization operation institute's time spent of said program are stored in performance database, and the expense and the income of said performance data library storage total data iteration optimization operation are used for control module;
Said control module also comprises:
Performance savings module is used for summarizing and safeguarding the expense of said data each run iteration optimization operation and the performance benefits of being brought;
The global decisions module is used for control module and selects for the part decision-making of the optimum combination of whole operational modules transmissions, selects the optimal combination of global optimum, recommends whole operational modules as global decisions.
14. On-line Iterative compile optimization as claimed in claim 13 system is characterized in that said control module also comprises:
When said data are program function, this program function parallel processing is stored in the process of data recording of operational module the intact remaining data record of prediction processing required time; The time overhead of control module decision iteration compile optimization operation, and the notice operational module carries out corresponding iteration compile optimization operation.
15. On-line Iterative compile optimization as claimed in claim 14 system is characterized in that said control module also comprises:
The compiler deployment module is used to attempt the different required compilers of compile optimization combination and is deployed in all operational modules;
The expansion sending module; Be used for said program function run-time optimizing system is expanded, make the form of its intermediate representation that can further be optimized the source code of said program function or other by dynamic or static compiler send to all operational modules from control module;
Iteration compiling total control module is used to determine the quantity of Optimizing operation and the execution of Control and Optimization operation.
16. On-line Iterative compile optimization as claimed in claim 14 system is characterized in that said operational module also comprises:
Sampling module; Be used for said operational module one end; After receiving the time prediction task that control module is distributed, the data recording that is distributed on the operational module is carried out random sampling, form the assessment data record; And in said assessment data record, the program implementation time is carried out partial estimation, pass partial estimation back control module then;
The linear session prediction module is used for said control module after obtaining the partial estimation that operational module passes back, uses the linear session prediction, and the total run time of said program function is estimated;
Iteration compiling budget module, being used for the optimization time is unit, decides the operation that need carry out how many iteration compilings, and iteration compile optimization task is distributed to the concurrent execution of operational module.
17. On-line Iterative compile optimization as claimed in claim 14 system is characterized in that, said performance savings module also comprises:
Optimize to sum up module, be used for said program function not the expense of homogeneous iteration optimization operation in service sum up and be maintained in the performance savings with the performance benefits of being brought;
The optimal control module is used for according to the data message of performance savings the opportunity and the frequency of iteration compiling being controlled, and guarantees that to reach expense only accounts for the target of the predefine small scale of income all the time; The mode of said control is the expense when a part of learning performance benefits through performance database and iteration compilation operations when suitable, then carries out the iteration compilation operations one time, and said part occurrence is set 0% to 100% by the user;
The savings distribution module is used for distributing a part to be used for the iteration compilation operations from by using the resulting performance savings of high performance optimal combination, and the occurrence of said part can be set 0% to 100% by the user;
The savings estimation module is used to utilize assessment data to write down performance benefits is assessed; Each operational module is assessed the speed-up ratio of new optimal combination with respect to acquiescence optimal combination on its assessment data record, it is sent it back control module; The performance that control module is utilized the information of speed-up ratio information and pending data recording number total amount to estimate and used this new optimal combination to bring is put aside;
Optimize the quantity module, be used for based on the performance savings that is predicted, the number of the optimal combination that the control module decision can be assessed on operational module, and the notice operational module is accomplished corresponding operation work.
18. On-line Iterative compile optimization as claimed in claim 14 system is characterized in that said operational module also comprises:
The actuating code generation module is used to utilize compiler with different compile optimization combinations, and the form of the intermediate representation that the source code of function or other are further optimized by dynamic or static compiler compiles and obtains corresponding executable code;
Optimal combination evaluation and test module is utilized the assessment data record of sampling gained, and the executable code that the compile optimization combination of designated test is obtained carries out performance evaluating to be tested with correctness;
The mark module after being used for each operational module and carrying out the iteration compilation operations, can send to control module with the speed-up ratio mark of the optimal combination of being evaluated and tested.
19. On-line Iterative compile optimization as claimed in claim 14 system is characterized in that said global decisions module also comprises:
Optimization result schistosity module is used on the operational module after receiving that the higher production of performance that control module is recommended is with optimal combination, can said program function being replaced with the version of optimization.
20. On-line Iterative compile optimization as claimed in claim 13 system is characterized in that said control module also comprises:
Processing module, when being used for said data and being the formed job queue of working procedure, the operation of monitoring task formation, said job queue, the task of the working procedure of submitting to the user forms;
The operation dispatch module; Be used for that operation is assigned to processing module cluster and carries out with Performance Evaluation with iteration compiling through job queue; The local file system of each processing module exists with absolute version, or the form that is present in the processing module cluster with a distributed file system part exists;
Monitoring modular is used for depositing performance database in monitoring the information gathering of said job queue with the said program implementation time.
21. On-line Iterative compile optimization as claimed in claim 20 system is characterized in that said processing module also comprises:
Iteration compilation operations module, be used for through receive that the operation dispatch module assigns compiling and Performance Evaluation, test iteration compilation operations with return Optimization result;
Collector, be used to call compiler with said program from the formal transformation of source code or intermediate code process for the object code that can on processing module, directly carry out; And in said process, need open through systems such as command line parameter or control documents or environmental variances, close or adjust the Optimizing operation that compiler is carried out in transcode process.
22. On-line Iterative compile optimization as claimed in claim 20 system is characterized in that said control module also comprises:
Optimize the evaluation and test module, be used for the executable object code that compiling obtains is moved on processing module, and measure working time and operation result is verified, when operation, need the sample input be provided for object code;
The optimized choice module is used for after the combination of program feature better optimize is found and tested, and processing module is used it and replaced existing production and use optimal combination, makes it can act on the follow-up request to this program, the acquisition more performance;
Mode selection module; A part that is used for processing module usability savings is carried out the iteration compilation operations; And through switching the frequency of controlling the generation of iteration compilation operations at conservative mode and radical pattern, said part occurrence is set 0% to 100% by the user.
23. On-line Iterative compile optimization as claimed in claim 20 system is characterized in that said monitoring modular also comprises:
Data set pond module is used for the required data set of Performance Evaluation test and obtains from a data set pond sampling; The inside, data set pond is storing operation to being listed as the employed data set of nearest carrying out of task;
The performance inventory module is used for compiling and Performance Evaluation are tested collection working time of being spent and deposited performance database in, to be used for estimating the expense of iteration compilation operations and the speed-up ratio of the optimal combination of estimating to evaluate and test.
24. the optimization system of On-line Iterative compiling as claimed in claim 23 is characterized in that said data set pond module also comprises:
The test set compound module is used to safeguard the performance of an incompatible assessment Different Optimization combination of test set that is made up of the data set of D picked at random and the test of correctness is carried out in the test set that said D is adjustable numeral; Said test set credit union periodically is updated to the data set of the up-to-date use of user, with the continuous variation in time of reply data set.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210048483.3A CN102622260B (en) | 2012-02-27 | 2012-02-27 | Optimization method and optimization system of on-line iteration compiling |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210048483.3A CN102622260B (en) | 2012-02-27 | 2012-02-27 | Optimization method and optimization system of on-line iteration compiling |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102622260A true CN102622260A (en) | 2012-08-01 |
| CN102622260B CN102622260B (en) | 2014-12-10 |
Family
ID=46562190
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210048483.3A Expired - Fee Related CN102622260B (en) | 2012-02-27 | 2012-02-27 | Optimization method and optimization system of on-line iteration compiling |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102622260B (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103901810A (en) * | 2012-12-31 | 2014-07-02 | 施耐德电器工业公司 | Optimizing system and method for user application of PLC |
| CN104516770A (en) * | 2014-12-31 | 2015-04-15 | 北京神舟航天软件技术有限公司 | Program calculation cost estimation technology based on high speed simulation |
| CN105374002A (en) * | 2014-08-20 | 2016-03-02 | 中国移动通信集团广东有限公司 | Formula efficiency assessment method and apparatus for network evaluation index |
| CN108415836A (en) * | 2018-02-23 | 2018-08-17 | 清华大学 | Utilize the method and system of application program detection computer system performance variation |
| CN108681487A (en) * | 2018-05-21 | 2018-10-19 | 千寻位置网络有限公司 | The distributed system and tuning method of sensing algorithm arameter optimization |
| CN110321116A (en) * | 2019-06-17 | 2019-10-11 | 大连理工大学 | A kind of effectively optimizing method towards calculating cost restricted problem in compiling optimization |
| CN110875085A (en) * | 2018-09-03 | 2020-03-10 | 中国石油化工股份有限公司 | Method for efficiently optimizing molecular structures in batches |
| CN110941424A (en) * | 2019-11-28 | 2020-03-31 | Oppo广东移动通信有限公司 | Compiling parameter optimization method and device and electronic equipment |
| TWI696948B (en) * | 2018-05-30 | 2020-06-21 | 臺灣發展軟體科技股份有限公司 | Compiler optimization method |
| CN112527300A (en) * | 2019-09-18 | 2021-03-19 | 无锡江南计算技术研究所 | Fine-grained compiling self-optimization method for multiple targets |
| CN112540767A (en) * | 2020-12-25 | 2021-03-23 | 北京百度网讯科技有限公司 | Program code generation method, program code generation device, electronic device and storage medium |
| CN114174983A (en) * | 2019-07-31 | 2022-03-11 | 国际商业机器公司 | Optimization for automatic verification of advanced constructs using test vectors |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101387965A (en) * | 2008-10-24 | 2009-03-18 | 北京科技大学 | Concurrent program compiling method and system |
| US20090158262A1 (en) * | 2007-12-12 | 2009-06-18 | Oracle International Corporation | Efficient compilation and execution of imperative-query languages |
-
2012
- 2012-02-27 CN CN201210048483.3A patent/CN102622260B/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090158262A1 (en) * | 2007-12-12 | 2009-06-18 | Oracle International Corporation | Efficient compilation and execution of imperative-query languages |
| CN101387965A (en) * | 2008-10-24 | 2009-03-18 | 北京科技大学 | Concurrent program compiling method and system |
Non-Patent Citations (2)
| Title |
|---|
| GRIGORI FURSIN ETC.: ""A Practical Method for Quickly Evaluating Program Optimizations"", 《HIGH PERFORMANCE EMBEDDED ARCHITECTURES AND COMPILERS》, 31 December 2005 (2005-12-31), pages 29 - 46 * |
| 汤伟等: "《面向IXP网络处理器的内联优化》", 《计算机科学》, vol. 33, no. 4, 30 April 2006 (2006-04-30) * |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103901810A (en) * | 2012-12-31 | 2014-07-02 | 施耐德电器工业公司 | Optimizing system and method for user application of PLC |
| CN103901810B (en) * | 2012-12-31 | 2017-04-12 | 施耐德电器工业公司 | Optimizing system and method for user application of PLC |
| CN105374002A (en) * | 2014-08-20 | 2016-03-02 | 中国移动通信集团广东有限公司 | Formula efficiency assessment method and apparatus for network evaluation index |
| CN104516770A (en) * | 2014-12-31 | 2015-04-15 | 北京神舟航天软件技术有限公司 | Program calculation cost estimation technology based on high speed simulation |
| CN108415836A (en) * | 2018-02-23 | 2018-08-17 | 清华大学 | Utilize the method and system of application program detection computer system performance variation |
| CN108681487A (en) * | 2018-05-21 | 2018-10-19 | 千寻位置网络有限公司 | The distributed system and tuning method of sensing algorithm arameter optimization |
| CN108681487B (en) * | 2018-05-21 | 2021-08-24 | 千寻位置网络有限公司 | Distributed system and method for adjusting and optimizing sensor algorithm parameters |
| TWI696948B (en) * | 2018-05-30 | 2020-06-21 | 臺灣發展軟體科技股份有限公司 | Compiler optimization method |
| CN110875085B (en) * | 2018-09-03 | 2022-07-29 | 中国石油化工股份有限公司 | Method for efficiently optimizing molecular structure in batches |
| CN110875085A (en) * | 2018-09-03 | 2020-03-10 | 中国石油化工股份有限公司 | Method for efficiently optimizing molecular structures in batches |
| CN110321116B (en) * | 2019-06-17 | 2021-04-20 | 大连理工大学 | Efficient optimization method for calculation cost constraint problem in compilation optimization |
| CN110321116A (en) * | 2019-06-17 | 2019-10-11 | 大连理工大学 | A kind of effectively optimizing method towards calculating cost restricted problem in compiling optimization |
| CN114174983A (en) * | 2019-07-31 | 2022-03-11 | 国际商业机器公司 | Optimization for automatic verification of advanced constructs using test vectors |
| CN114174983B (en) * | 2019-07-31 | 2022-10-11 | 国际商业机器公司 | Method and system for optimized automatic verification of advanced constructs |
| CN112527300A (en) * | 2019-09-18 | 2021-03-19 | 无锡江南计算技术研究所 | Fine-grained compiling self-optimization method for multiple targets |
| CN112527300B (en) * | 2019-09-18 | 2023-06-13 | 无锡江南计算技术研究所 | Multi-target-oriented fine granularity compiling self-optimizing method |
| CN110941424A (en) * | 2019-11-28 | 2020-03-31 | Oppo广东移动通信有限公司 | Compiling parameter optimization method and device and electronic equipment |
| CN112540767A (en) * | 2020-12-25 | 2021-03-23 | 北京百度网讯科技有限公司 | Program code generation method, program code generation device, electronic device and storage medium |
| CN112540767B (en) * | 2020-12-25 | 2023-07-25 | 北京百度网讯科技有限公司 | Program code generation method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102622260B (en) | 2014-12-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102622260B (en) | Optimization method and optimization system of on-line iteration compiling | |
| CN110825518B (en) | Micro-service-oriented nanosecond power resource distribution method and system | |
| Liu et al. | A hybrid ICT-solution for smart meter data analytics | |
| CN102436494B (en) | Device and method for optimizing execution plan and based on practice testing | |
| CN110249310A (en) | The resource management for virtual machine in cloud computing system | |
| CN110352415A (en) | The prediction table pre-connection of figure group monitoring is utilized in large-scale data management system | |
| CN105069134A (en) | Method for automatically collecting Oracle statistical information | |
| CN105074664A (en) | Cost-minimizing task scheduler | |
| CN102831012A (en) | Task scheduling device and task scheduling method in multimode distributive system | |
| Frincu et al. | Accurate and efficient selection of the best consumption prediction method in smart grids | |
| CN103106152A (en) | Data scheduling method based on gradation storage medium | |
| CN102081625A (en) | Data query method and query server | |
| CN106503007B (en) | Database operation method and device | |
| Mu et al. | Evaluating the potential of graphics processors for high performance embedded computing | |
| Kamthe et al. | A stochastic approach to estimating earliest start times of nodes for scheduling DAGs on heterogeneous distributed computing systems | |
| CN109978177A (en) | Model training method, method for processing business, device and relevant device | |
| CN108885579A (en) | For tracking the method and apparatus for carrying out data mining according to core | |
| Wang et al. | Lube: Mitigating bottlenecks in wide area data analytics | |
| Takeda et al. | Sensory uncertainty field for mobile robot navigation | |
| Venkataswamy et al. | Rare: Renewable energy aware resource management in datacenters | |
| Hartmann et al. | CSAR: the cross-sectional autoregression model for short and long-range forecasting | |
| CN100518146C (en) | A method for global buffer management of the cluster storage system | |
| Yi et al. | Not all explorations are equal: Harnessing heterogeneous profiling cost for efficient mlaas training | |
| CN103246539A (en) | GIS (Geographical Information System) module loading method based on use habit of user | |
| Geimer et al. | Recent developments in the scalasca toolset |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20141210 Termination date: 20190227 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |