CN101446898A - Method for supporting large scale parallel distributed computation through functional programming model - Google Patents
Method for supporting large scale parallel distributed computation through functional programming model Download PDFInfo
- Publication number
- CN101446898A CN101446898A CNA2008102354359A CN200810235435A CN101446898A CN 101446898 A CN101446898 A CN 101446898A CN A2008102354359 A CNA2008102354359 A CN A2008102354359A CN 200810235435 A CN200810235435 A CN 200810235435A CN 101446898 A CN101446898 A CN 101446898A
- Authority
- CN
- China
- Prior art keywords
- function
- code
- call
- variable
- functions
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 31
- 230000006870 function Effects 0.000 claims abstract description 68
- 230000000694 effects Effects 0.000 claims abstract description 8
- 239000011800 void material Substances 0.000 claims description 12
- 238000004088 simulation Methods 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 2
- 238000004043 dyeing Methods 0.000 claims description 2
- 230000003139 buffering effect Effects 0.000 abstract 1
- 238000012512 characterization method Methods 0.000 abstract 1
- 230000006698 induction Effects 0.000 abstract 1
- 208000015181 infectious disease Diseases 0.000 abstract 1
- 238000011161 development Methods 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000008676 import Effects 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000001939 inductive effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Landscapes
- Devices For Executing Special Programs (AREA)
Abstract
The invention discloses a method for supporting large scale parallel distributed computation through functional programming model. The method comprises the following steps: firstly, converting realization codes to function formulae based on process; then, eliminating the edge effect of functions; establishing fork and forkout functions; conducting simulated implementation process, in the event of function call, converting the function call to use functions and the parameters thereof as parameters, and introducing pseudo-call function; carrying out call relations of stain acquired functions to function call through adopting the infection characterization of VOLD; producing a series of function implementation sequences when reaching the terminal of the procedure; conducting distributed computation for one time by the implementation sequences and the parameters; and buffering the result and conducting second implementation to the code, in the process, in the event of function call, directly returning to the buffered data. The invention realizes the distribution and scheduling implementation computation of functions through establishing a model which can automatically dividing induction problems based on the general purpose language, and is suitable for applying in a large-scale distributed system.
Description
Technical field:
The present invention relates to the realization technology of function programming model, the model that a kind of automatic conclusion is provided especially and has pinpointed the problems and cut apart, and the algorithm of distribution schedule execution.
Background technology:
Because restrictions such as power consumption, dominant frequency, processor speed promotes and progressively arrives bottleneck now.In order further to promote processing speed, the processor development has to transfer under the multinuclear pattern that more concurrent calculating nuclear is provided from simple raising computing velocity.Along with progressively increasing of multinuclear quantity, the increase of concurrent quantity, the large-scale parallel system that only exists in large-scale scientific research institutions will progressively appear under the consumption platform in the past.
Because the development of computing machine in the past lacks the robotization support at multinuclear and even massively parallel system substantially all based under the single-processor mode.The developer must self-dependent experience, and the shoring of foundation of Distribution calculation is cut apart and realized to problem, to developer's Capability Requirement than higher.
The functional expression programming model has natural concurrency, very is suitable in the large-scale distributed system.Yet because the design philosophy of functional expression programming with very big based on the language difference of process, is difficult to popularize in the developer.
This shows that two kinds of development technique at massively parallel system all have higher requirement to the developer, popular along with the multinuclear hardware system supports also to have proposed more and more urgent requirement to developing software of large-scale distributed calculating.
Summary of the invention:
Technical matters to be solved by this invention provides a kind of method by the large-scale distributed parallel computation of function programming model supports.
A kind of method by the large-scale distributed parallel computation of function programming model supports of the present invention may further comprise the steps:
1) at first, be the functional expression code with the program code conversion of process type, process is: eliminate the existence of variable, for quoting of each variable, the function on the right of this variable assignments formula partly replaces;
2) then, eliminate the edge effect of function, process is: utilize the global field of the code object of python code, obtain the visit information of global variable, will be converted to the parameter of importing into to reading of global variable, writing of global variable converted to rreturn value;
3) by fork and forkout function, the function that offsets except marginal utility carries out implementation twice, and step is: duplicate the frame object, comprise its local variable and storehouse, give python virtual machine Simulation execution the frame that is replicated subsequently.And when running into function call, code is replaced to pseudo-invoke code;
4) structure VOID class makes any operation of any example to this class all can return and understands this class itself, by the machine-processed dependence of knowing function call of this dyeing;
5) utilize pseudo-call function, the function that goes execute script to carry out, the parameter of importing into is any VOID class that all can return this example itself to the operation of such example;
When 6) carrying out the forkout function, execution result is returned by the forkout function, and the frame object that is replicated is abandoned, global context is reset into the state before of carrying out, and normal the execution;
7) forkout returns is that a series of functions that can executed in parallel that produce in the Simulation execution process are carried out sequence, carry out Distributed Calculation one time according to this execution sequence and parameter, the result is cushioned, and code is carried out the second time carry out, in the process of carrying out for the second time, run into function call, directly return the data that cushioned.
The present invention realizes the distribution schedule execution algorithm of function by setting up the model of cutting apart based on frugal FORTRAN Rules Used as a General Applications Language and robotization inductive problem, is fit to be applied to large-scale distributed system.By the present invention, in the discovery serial code of robotization can executed in parallel part.And because minimum thread is a function, thereby fully excavated degree of parallelism, thus can effectively be used in the massively parallel system.
Description of drawings:
Fig. 1 Python code robotization executed in parallel overall framework;
Fig. 2 Code object structure;
Fig. 3 fork and forkout examples of functions;
The pseudo-call function synoptic diagram of Fig. 4;
The characteristics of Fig. 5 VOID class;
Fig. 6 executed in parallel example.
Embodiment:
The present invention at first will convert functional expression to based on the code of process, be global variable owing to hinder the key issue of function executed in parallel.Because the existence of global variable makes function have state, can't satisfy input and output condition one to one.Therefore at first need to eliminate the edge effect of function.Shown in Fig. 1 flow process, at first be written into the py code, and be compiled into the code object.The structure of code object as shown in Figure 2, they are organized into a code tree.Travel through this code tree, the function with the all-access global variable imports global variable into as parameter.With all functions that writes global variable, global variable is returned as one of return results.So through once traveling through, will generate a new code tree, all functions in this code tree all do not have edge effect.
Based on the code of no marginal effect,, realize Simulation execution by structure fork and forkout function as shown in Figure 3.Its process comprises its local variable and storehouse for duplicating the frame object.The frame that is replicated with relief gives the python virtual machine and normally carries out, and when carrying out the forkout function, execution result is returned by the forkout function.And the frame object that is replicated abandoned, return to the state before carrying out again, and normal the execution.
Subsequently,, need carry out Simulation execution one time, analyze the relation of interdependence between the function this code for the equivalent code that utilizes no marginal effect reaches the purpose of concurrent execution.Carry out ergodic process as shown in Figure 1 again to code tree, current when running into function call, code is replaced to pseudo-invoke code.
As shown in Figure 4.Utilize pseudo-call function, the function that goes execute script to carry out, the parameter of importing into is a specifically-built VOID class.As shown in Figure 5, the characteristics of VOID class are that any operation to VOID class example all can be returned this example itself.
The process of once carrying out subsequently in this process, when the parameter of importing into does not comprise the VOID example, illustrates this function less than relying on, and can independently carry out.When the function that imports into has comprised the VOID example, illustrate that then this function will depend on a last function that returns this VOID example, must serial computing in parallel computation the two.
In the process of Simulation execution,, function all become VOID because carrying out the data relied on, thus can there be real calculating, the only dependence of analytic function and execution parameter sequence, thereby execution speed is very fast.
When Simulation execution arrives the terminal point of program, then produced a series of function and carried out sequence, as shown in Figure 6.Because there is not edge effect in the function in this version, therefore the input of determining is bound to produce definite output.Carry out Distributed Calculation one time according to this execution sequence and parameter, the result is cushioned, and code is carried out the second time carry out.In the process of carrying out for the second time, run into function call, directly return the data that cushioned, thereby reach parallel computation, improve the purpose of execution speed.
Claims (1)
1, a kind of method by the large-scale distributed parallel computation of function programming model supports is characterized in that may further comprise the steps:
1) at first, be the functional expression code with the program code conversion of process type, process is: eliminate the existence of variable, for quoting of each variable, the function on the right of this variable assignments formula partly replaces;
2) then, eliminate the edge effect of function, process is: utilize the global field of the code object of python code, obtain the visit information of global variable, will be converted to the parameter of importing into to reading of global variable, writing of global variable converted to rreturn value;
3) by fork and forkout function, the function that offsets except marginal utility carries out implementation twice, step is: duplicate the frame object, comprise its local variable and storehouse, give python virtual machine Simulation execution the frame that is replicated subsequently, and when running into function call, code is replaced to pseudo-invoke code;
4) structure VOID class makes any operation of any example to this class all can return and understands this class itself, by the machine-processed dependence of knowing function call of this dyeing;
5) utilize pseudo-call function, the function that goes execute script to carry out, the parameter of importing into is any VOID class that all can return this example itself to the operation of such example;
When 6) carrying out the forkout function, execution result is returned by the forkout function, and the frame object that is replicated is abandoned, global context is reset into the state before of carrying out, and normal the execution;
7) the forkout function returns is that a series of functions that can executed in parallel that produce in the Simulation execution process are carried out sequence, carry out Distributed Calculation one time according to this execution sequence and parameter, the result is cushioned, and code is carried out the second time carry out, in the process of carrying out for the second time, run into function call, directly return the data that cushioned.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2008102354359A CN101446898B (en) | 2008-12-02 | 2008-12-02 | Method for supporting large scale parallel distributed computation through functional programming model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2008102354359A CN101446898B (en) | 2008-12-02 | 2008-12-02 | Method for supporting large scale parallel distributed computation through functional programming model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101446898A true CN101446898A (en) | 2009-06-03 |
| CN101446898B CN101446898B (en) | 2011-06-15 |
Family
ID=40742589
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2008102354359A Expired - Fee Related CN101446898B (en) | 2008-12-02 | 2008-12-02 | Method for supporting large scale parallel distributed computation through functional programming model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101446898B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103635879A (en) * | 2011-07-08 | 2014-03-12 | 索尼公司 | Information processing device, information processing method, and computer program |
| CN104092755A (en) * | 2014-07-07 | 2014-10-08 | 武汉科技大学 | Capturing method and device for cloud service origin data |
| US9338234B2 (en) | 2014-04-16 | 2016-05-10 | Microsoft Technology Licensing, Llc | Functional programming in distributed computing |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1180192A (en) * | 1996-10-10 | 1998-04-29 | 西北工业大学 | Single program multidata type program automatic parallelization method based on shared data |
| US5987255A (en) * | 1997-03-12 | 1999-11-16 | International Business Machines Corporation | Method of, system for, and article of manufacture for providing a generic adaptor for converting from a sequential iterator to a pre-thread parallel iterator |
-
2008
- 2008-12-02 CN CN2008102354359A patent/CN101446898B/en not_active Expired - Fee Related
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103635879A (en) * | 2011-07-08 | 2014-03-12 | 索尼公司 | Information processing device, information processing method, and computer program |
| CN103635879B (en) * | 2011-07-08 | 2017-09-08 | 索尼公司 | Message processing device, information processing method |
| US9338234B2 (en) | 2014-04-16 | 2016-05-10 | Microsoft Technology Licensing, Llc | Functional programming in distributed computing |
| CN104092755A (en) * | 2014-07-07 | 2014-10-08 | 武汉科技大学 | Capturing method and device for cloud service origin data |
| CN104092755B (en) * | 2014-07-07 | 2017-06-13 | 武汉科技大学 | A kind of method and device for capturing of cloud service origination data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101446898B (en) | 2011-06-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105868019B (en) | A kind of Spark platform property automatic optimization method | |
| CN103019742B (en) | A kind of real time workshop method in many DSP platform | |
| CN104570081A (en) | Pre-stack reverse time migration seismic data processing method and system by integral method | |
| CN101216803A (en) | Test program control stream path set creation method based on base path | |
| US20200167139A1 (en) | Dynamic generation of cpu instructions and use of the cpu instructions in generated code for a softcore processor | |
| CN107341761A (en) | A kind of calculating of deep neural network performs method and system | |
| CN102880509A (en) | Compute unified device architecture (CUDA) based grid digital elevation model (DEM) neighborhood analysis system and method | |
| CN104899369A (en) | Simulator multithreading operation method utilizing PERL script | |
| CN101446898B (en) | Method for supporting large scale parallel distributed computation through functional programming model | |
| CN107562894A (en) | A kind of API of web application systems behavior simulation obtains and methods for using them | |
| CN110851123A (en) | WebGIS power grid visualization framework construction method, system and device based on SpringMVC | |
| CN102929853A (en) | DCS (Data Communication System) project data generating system and method based on Excel sheet association | |
| CN104299170B (en) | Intermittent energy source mass data processing method | |
| CN110334001A (en) | A kind of method and apparatus that batch automatically generates echo test | |
| Govett et al. | The scalable modeling system: Directive-based code parallelization for distributed and shared memory computers | |
| CN114064012B (en) | Dynamic and static combined interface code generation method and system and electronic equipment | |
| CN109388876A (en) | A kind of groundwater solute transfer numerical simulation parallel acceleration method | |
| CN104731968A (en) | Cluster excavation method of large-scale data set of single machine | |
| Sha et al. | Performance modeling of openstack cloud computing platform using performance evaluation process algebra | |
| CN107831672A (en) | Intelligent energy system Optimized Design of Visualization runs integrated apparatus and method | |
| CN102073617B (en) | WPF technology is utilized to complete the system of calculating | |
| CN113626035A (en) | Neural network compiling method facing RISC-V equipment based on TVM | |
| Hasanov et al. | High-level topology-oblivious optimization of MPI broadcast algorithms on extreme-scale platforms | |
| Pöppl et al. | Shallow water waves on a deep technology stack: Accelerating a finite volume tsunami model using reconfigurable hardware in invasive computing | |
| CN101441571A (en) | Gridding system implementing method based on Python language |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20110615 Termination date: 20131202 |