CN108334410A - A kind of distributed application program client light weight method and computer equipment - Google Patents
A kind of distributed application program client light weight method and computer equipment Download PDFInfo
- Publication number
- CN108334410A CN108334410A CN201810077348.9A CN201810077348A CN108334410A CN 108334410 A CN108334410 A CN 108334410A CN 201810077348 A CN201810077348 A CN 201810077348A CN 108334410 A CN108334410 A CN 108334410A
- Authority
- CN
- China
- Prior art keywords
- finger daemon
- distributed
- group terminal
- calculation
- application program
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
- G06F9/5088—Techniques for rebalancing the load in a distributed system involving task migration
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer And Data Communications (AREA)
- Multi Processors (AREA)
Abstract
The present invention provides a kind of distributed application program client light weight method, and the finger daemon at distributed application program end submits calculating task and computing resource application to the finger daemon of Distributed Calculation collection group terminal；The finger daemon of Distributed Calculation collection group terminal carries out computational resource allocation, and carries out calculating task；The finger daemon of Distributed Calculation collection group terminal monitors the calculate node used；The finger daemon of Distributed Calculation collection group terminal receives the result of calculation of each calculate node, carries out aggregation process；The result of calculation summarized is sent to the finger daemon at distributed application program end by the finger daemon of Distributed Calculation collection group terminal；The present invention also provides a kind of computer equipments, simplify client, prevent the wasting of resources.
Description
Technical field
The present invention relates to a kind of distributed application program client light weight method and computer equipments.
Background technology
Distributed Calculation is a kind of computational methods, and it is opposite that centralization, which calculates,.With the development of computing technique, some
Using needing very huge computing capability that could complete, if calculated using centralization, needs to expend and come for quite a long time
It completes.The application decomposition at many small parts, is distributed to multiple stage computers and is handled by Distributed Calculation.It can save in this way
About the overall calculation time, greatly improve computational efficiency.Cluster management mode to Distributed Calculation cluster (multiple stage computers) is
Generating a finger daemon on each computer can as explorer of this computer node in entire cluster
Manage the computing resource of this computer and each to fall on point that the calculating of the distributed application program on this computer provides resource
Match and management service, is also responsible for starting, monitoring and tracking fall on the functions such as the process of distributed application program on this computer.
Each Session of each distributed application program can have a client to keep in distributed application program client
Shield process, this finger daemon are responsible for submitting distributed application program to distributed type assemblies, are responsible for this Session and are counted in distribution
The application of resource in cluster is calculated, the calculate node used in monitoring distributed cluster receives and dispatches calculate node communication message, monitoring
This Session runs the health status of process in distributed type assemblies, monitors the health of this Session distributed type assemblies operation nodes
State, reception summarize operation result, and client-side program is waited for obtain operation result etc. from finger daemon.
This can be carried inside each Session clients according to the present each distributed application program of design
The Client of distributed application program is guarded, and Session clients each in this way will seem very too fat to move, due to being counted with distribution
It calculates cluster and there is a large amount of communication, these communications are all by network connection, and some is even also right across multiple network segments or exchange
The network bandwidth requirement used is higher, while also bigger to the resource requirement of the computer of distributed application program deployment, after
Phase resource capacity expansion is more dispersed, and there are the variety of problems such as a large amount of wasting of resources.
Invention content
The technical problem to be solved in the present invention, be to provide a kind of distributed application program client light weight method and
Computer equipment simplifies client, prevents the wasting of resources.
What one of present invention was realized in：A kind of distributed application program client light weight method, including：
Step 1, distributed application program end finger daemon submit calculating task and computing resource application to distribution
The finger daemon at computing cluster end；
Step 2, the finger daemon progress computational resource allocation of Distributed Calculation collection group terminal, and carry out calculating task；
The calculate node that the finger daemon monitoring of step 3, Distributed Calculation collection group terminal is used；
Step 4, Distributed Calculation collection group terminal finger daemon receive the result of calculation of each calculate node, carry out summarizing place
Reason；
Step 5, Distributed Calculation collection group terminal finger daemon the result of calculation summarized is sent to distributed application program
The finger daemon at end.
Further, the step 3 is specially further：What the finger daemon monitoring of Distributed Calculation collection group terminal was used
Calculate node is applied for that resource is handled again if some node processing fails, if without node resource application location,
Miss data is then returned to the finger daemon at distributed application program end, terminates calculating task.
What the two of the present invention were realized in：A kind of computer equipment, including memory, processor and it is stored in memory
Computer program that is upper and can running on a processor, the processor realize following steps when executing described program：
Step 1, distributed application program end finger daemon submit calculating task and computing resource application to distribution
The finger daemon at computing cluster end；
Step 2, the finger daemon progress computational resource allocation of Distributed Calculation collection group terminal, and carry out calculating task；
The calculate node that the finger daemon monitoring of step 3, Distributed Calculation collection group terminal is used；
Step 4, Distributed Calculation collection group terminal finger daemon receive the result of calculation of each calculate node, carry out summarizing place
Reason；
Step 5, Distributed Calculation collection group terminal finger daemon the result of calculation summarized is sent to distributed application program
The finger daemon at end.
Further, the step 3 is specially further：What the finger daemon monitoring of Distributed Calculation collection group terminal was used
Calculate node is applied for that resource is handled again if some node processing fails, if without node resource application location,
Miss data is then returned to the finger daemon at distributed application program end, terminates calculating task.
The invention has the advantages that：1, the Distributed Calculation cluster Client finger daemons at distributed application program end are big
Partial function toward migrating on Distributed Calculation cluster, the client volume of distributed application program is reduced with this.
2, the Distributed Calculation cluster Client finger daemons Yu Distributed Calculation cluster at distributed application program end is logical
News are simplified to minimum, and a large amount of communication all remains into Distributed Calculation cluster internal (since most function module is moved
Move on to the finger daemon on distributed type assemblies, that also imply that between these functions and function module and distributed type assemblies it
Between communication also all move in big data platform), this be all Intranet communication, so greatly reduce client network need
It asks.
3, all tasks of original Distributed Calculation cluster Client finger daemons are all loaded in the visitor of distributed application program
Family end, then these tasks occupy a large amount of resource, and these resources are all to hold for a long time, and the task that arrive terminates just release
It puts, also there is no all using in real time, existing a large amount of idle wasting phenomenon arrives these task immigrations now these resources
In Distributed Calculation cluster, there is special scheduling of resource module in distributed type assemblies, allotment resource that can be quite reasonable makes
With, while whole interaction flow is also optimized to reduce the use of resource, a large amount of resource is saved with this and is used.
4, the Distributed Calculation cluster Client finger daemons at distributed application program end are simplified to minimum, realize visitor
The lightweight deployment of family end.
5, previous cross-network segment is communicated, the optimization that the flows such as result set full dose transmission carry out greatly improves application
The performance of program.
Description of the drawings
The present invention is further illustrated in conjunction with the embodiments with reference to the accompanying drawings.
Fig. 1 is the method for the present invention execution flow chart.
Specific implementation mode
Distributed application program client light weight method of the present invention, including：
Step 1, distributed application program end finger daemon submit calculating task and computing resource application to distribution
The finger daemon at computing cluster end；
Step 2, the finger daemon progress computational resource allocation of Distributed Calculation collection group terminal, and carry out calculating task；
The calculate node that the finger daemon monitoring of step 3, Distributed Calculation collection group terminal is used, if some node processing is lost
It loses, is then applied for that resource is handled again, if without node resource application location, return to miss data to Distributed Application
The finger daemon of program end terminates calculating task；
Step 4, Distributed Calculation collection group terminal finger daemon receive the result of calculation of each calculate node, carry out summarizing place
Reason；
Step 5, Distributed Calculation collection group terminal finger daemon the result of calculation summarized is sent to distributed application program
The finger daemon at end.
Computer equipment of the present invention, including memory, processor and storage can be run on a memory and on a processor
Computer program, the processor realizes following steps when executing described program：
Step 1, distributed application program end finger daemon submit calculating task and computing resource application to distribution
The finger daemon at computing cluster end；
Step 2, the finger daemon progress computational resource allocation of Distributed Calculation collection group terminal, and carry out calculating task；
The calculate node that the finger daemon monitoring of step 3, Distributed Calculation collection group terminal is used, if some node processing is lost
It loses, is then applied for that resource is handled again, if without node resource application location, return to miss data to Distributed Application
The finger daemon of program end terminates calculating task；
Step 4, Distributed Calculation collection group terminal finger daemon receive the result of calculation of each calculate node, carry out summarizing place
Reason；
Step 5, Distributed Calculation collection group terminal finger daemon the result of calculation summarized is sent to distributed application program
The finger daemon at end.
A kind of specific implementation mode of the present invention：
1, submit distributed application program to distributed type assemblies
This task is still completed by the Distributed Calculation cluster Client finger daemons at distributed application program end, due to
Task is by being generated in Distributed Application formula, so this function can not go to the completion of the process in Distributed Calculation cluster.
2, the application of this Session resources in Distributed Calculation cluster
Computing resource application is sent always by the Distributed Calculation cluster Client finger daemons at distributed application program end
Resource bid require group terminal to collection cluster Client finger daemons, collect group terminal Client finger daemons be responsible for monitor resource exist
It is unified after the completion of all tasks of application etc. one by one of cluster once to give the process at application program end one reply, reduce a large amount of
The task of communication and interaction.
3, the calculate node used in monitoring distributed cluster
This task is distributed the Client finger daemons at formula computing cluster end to complete, the meter used this Session
Operator node is monitored, if it find that some node failure, and be responsible for applying again by the Client finger daemons of collection group terminal
Resource takes over the task of failure node.This task is communicated substantially without with application program end, until no task node provides
Until source can apply, lead to mission failure, just can send message to application program end is informed.
4, calculate node communication message and scheduling information are received and dispatched
This task is completed by the Client finger daemons of collection group terminal completely, receives and dispatches the communication message between calculate node, and appoint
Business scheduling message etc..
5, health status and Task Progress that this Session runs process in distributed type assemblies are monitored
This task is completed by the Client finger daemons of collection group terminal completely, is monitored this Session and is run in distributed type assemblies
The health status of process judges whether to need to open more if it find that the calculation procedure failure of operation, restarts operation process
The work such as a parallel task process.
6, it receives and summarizes operation result
This task is completed by the Client finger daemons of collection group terminal completely, and distribution is transferred to after result sequence number is compressed
The Distributed Calculation cluster Client finger daemons at application program end, the data volume of transmission is reduced with this.
7, client-side program is waited for obtain operation result from finger daemon
This task is still completed by the Distributed Calculation cluster Client finger daemons at distributed application program end.
In the past since most of function is all docked to client, then the pressure communication between distributed type assemblies and client
Bigger, especially in the environment of distributed type assemblies and client cross-network segment, the pressure of this communication is with regard to bigger, present function
It inside adjourns on distributed type assemblies, reduces the traffic between a large amount of distributed type assemblies and client, distributed type assemblies and visitor
Most of communication at family end is also to be completed by the finger daemon of two places, and the finger daemon on distributed type assemblies is responsible for collecting whole
A distributed type assemblies need the message with user client communication to carry out unified communication, are inherently derived in this way in the pressure of network communication
Alleviate；
In result set full dose transfer process, the Session finger daemons first on distributed type assemblies can be responsible for collecting data,
Last processing is done, by compression, paging waits different technologies to transfer data in the Session finger daemons of client.
Will not as before each distributed type assemblies node result of calculation is transmitted directly to client Session finger daemon
In.
Although specific embodiments of the present invention have been described above, those familiar with the art should manage
Solution, we are merely exemplary described specific embodiment, rather than for the restriction to the scope of the present invention, it is familiar with this
The technical staff in field modification and variation equivalent made by the spirit according to the present invention, should all cover the present invention's
In scope of the claimed protection.
Claims (4)
1. a kind of distributed application program client light weight method, it is characterised in that：Including：
Step 1, distributed application program end finger daemon submit calculating task and computing resource application to Distributed Calculation
Collect the finger daemon of group terminal；
Step 2, the finger daemon progress computational resource allocation of Distributed Calculation collection group terminal, and carry out calculating task；
The calculate node that the finger daemon monitoring of step 3, Distributed Calculation collection group terminal is used；
Step 4, Distributed Calculation collection group terminal finger daemon receive the result of calculation of each calculate node, carry out aggregation process；
Step 5, Distributed Calculation collection group terminal finger daemon the result of calculation summarized is sent to distributed application program end
Finger daemon.
2. a kind of distributed application program client light weight method according to claim 1, it is characterised in that：The step
Rapid 3 are specially further：The finger daemon of Distributed Calculation collection group terminal monitors the calculate node used, if some node processing
Failure, then applied for that resource is handled, if without node resource application location, return to miss data to distribution and answer again
With the finger daemon of program end, terminate calculating task.
3. a kind of computer equipment, including memory, processor and storage are on a memory and the meter that can run on a processor
Calculation machine program, which is characterized in that the processor realizes following steps when executing described program：
Step 1, distributed application program end finger daemon submit calculating task and computing resource application to Distributed Calculation
Collect the finger daemon of group terminal；
Step 2, the finger daemon progress computational resource allocation of Distributed Calculation collection group terminal, and carry out calculating task；
The calculate node that the finger daemon monitoring of step 3, Distributed Calculation collection group terminal is used；
Step 4, Distributed Calculation collection group terminal finger daemon receive the result of calculation of each calculate node, carry out aggregation process；
Step 5, Distributed Calculation collection group terminal finger daemon the result of calculation summarized is sent to distributed application program end
Finger daemon.
4. a kind of computer equipment according to claim 3, it is characterised in that：The step 3 is specially further：Distribution
The finger daemon at formula computing cluster end monitors the calculate node used, if some node processing fails, is applied again
Resource is handled, if without node resource application location, returns to miss data to the finger daemon at distributed application program end,
Terminate calculating task.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810077348.9A CN108334410A (en) | 2018-01-26 | 2018-01-26 | A kind of distributed application program client light weight method and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810077348.9A CN108334410A (en) | 2018-01-26 | 2018-01-26 | A kind of distributed application program client light weight method and computer equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108334410A true CN108334410A (en) | 2018-07-27 |
Family
ID=62926626
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810077348.9A Pending CN108334410A (en) | 2018-01-26 | 2018-01-26 | A kind of distributed application program client light weight method and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108334410A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109375873A (en) * | 2018-09-27 | 2019-02-22 | 郑州云海信息技术有限公司 | The initial method of data processing finger daemon in a kind of distributed storage cluster |
| CN111158872A (en) * | 2019-12-16 | 2020-05-15 | 北京明朝万达科技股份有限公司 | Method and device for submitting and guarding spark task |
| CN112230284A (en) * | 2019-07-15 | 2021-01-15 | 中国石油天然气集团有限公司 | Parallel random noise attenuation method, monitoring method and node for three-dimensional pre-stack data |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102436404A (en) * | 2011-08-24 | 2012-05-02 | 苏州阔地网络科技有限公司 | Daemon implementing method |
| CN102854853A (en) * | 2012-08-13 | 2013-01-02 | 北京和利时系统工程有限公司 | Cross-platform lightweight distributed control system |
| US20150106811A1 (en) * | 2011-05-25 | 2015-04-16 | Vmware, Inc. | System and method for managing a virtualized computing environment |
| CN106681824A (en) * | 2015-11-10 | 2017-05-17 | 阿里巴巴集团控股有限公司 | Distributed system scheduling method and device and electronic equipment |
| CN106843997A (en) * | 2016-10-26 | 2017-06-13 | 西安交通大学 | A kind of parallel virtual machine polymerization based on Spark with optimization MBBO algorithms |
| CN106936911A (en) * | 2017-03-13 | 2017-07-07 | 广州五舟科技股份有限公司 | Lightweight Distributed Computing Platform and its computing resource management-control method |
| CN106951357A (en) * | 2017-05-10 | 2017-07-14 | 焦点科技股份有限公司 | A kind of server end application issue monitoring method and device based on finger daemon |
-
2018
- 2018-01-26 CN CN201810077348.9A patent/CN108334410A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150106811A1 (en) * | 2011-05-25 | 2015-04-16 | Vmware, Inc. | System and method for managing a virtualized computing environment |
| CN102436404A (en) * | 2011-08-24 | 2012-05-02 | 苏州阔地网络科技有限公司 | Daemon implementing method |
| CN102854853A (en) * | 2012-08-13 | 2013-01-02 | 北京和利时系统工程有限公司 | Cross-platform lightweight distributed control system |
| CN106681824A (en) * | 2015-11-10 | 2017-05-17 | 阿里巴巴集团控股有限公司 | Distributed system scheduling method and device and electronic equipment |
| CN106843997A (en) * | 2016-10-26 | 2017-06-13 | 西安交通大学 | A kind of parallel virtual machine polymerization based on Spark with optimization MBBO algorithms |
| CN106936911A (en) * | 2017-03-13 | 2017-07-07 | 广州五舟科技股份有限公司 | Lightweight Distributed Computing Platform and its computing resource management-control method |
| CN106951357A (en) * | 2017-05-10 | 2017-07-14 | 焦点科技股份有限公司 | A kind of server end application issue monitoring method and device based on finger daemon |
Non-Patent Citations (3)
| Title |
|---|
| T. C. BRESSOUD等: "Results of a Model for Hadoop YARN MapReduce Tasks", 《2016 IEEE INTERNATIONAL CONFERENCE ON CLUSTER COMPUTING (CLUSTER)》 * |
| ZNI.FENG: "YARN学习总结", 《HTTPS://WWW.CNBLOGS.COM/ZNICY/P/6714039.HTML》 * |
| 梁寿愚等: "基于云计算架构的新一代调度自动化系统", 《南方电网技术》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109375873A (en) * | 2018-09-27 | 2019-02-22 | 郑州云海信息技术有限公司 | The initial method of data processing finger daemon in a kind of distributed storage cluster |
| CN112230284A (en) * | 2019-07-15 | 2021-01-15 | 中国石油天然气集团有限公司 | Parallel random noise attenuation method, monitoring method and node for three-dimensional pre-stack data |
| CN111158872A (en) * | 2019-12-16 | 2020-05-15 | 北京明朝万达科技股份有限公司 | Method and device for submitting and guarding spark task |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Sajjad et al. | Spanedge: Towards unifying stream processing over central and near-the-edge data centers | |
| CN106131213B (en) | Service management method and system | |
| Verma et al. | An efficient data replication and load balancing technique for fog computing environment | |
| JP4970939B2 (en) | Hierarchical management of dynamic resource allocation in multi-node systems | |
| CN108845878A (en) | The big data processing method and processing device calculated based on serverless backup | |
| US20100153482A1 (en) | Cloud-Based Automation of Resources | |
| CN105357296A (en) | Elastic caching system based on Docker cloud platform | |
| CN107291536B (en) | Application task flow scheduling method in cloud computing environment | |
| CN107239675A (en) | Biological information analysis system based on cloud platform | |
| CN104618164A (en) | Management method for rapid cloud computing platform application deployment | |
| CN111459641B (en) | Method and device for task scheduling and task processing across machine room | |
| CN108334410A (en) | A kind of distributed application program client light weight method and computer equipment | |
| CN109614227A (en) | Task resource concocting method, device, electronic equipment and computer-readable medium | |
| US8458702B1 (en) | Method for implementing user space up-calls on java virtual machine before/after garbage collection | |
| CN111124640A (en) | Task allocation method and system, storage medium and electronic device | |
| Subramaniyan et al. | GEMS: Gossip-enabled monitoring service for scalable heterogeneous distributed systems | |
| CN110011984B (en) | REST and RPC-based distributed cluster system and method | |
| Albrecht et al. | Making work queue cluster-friendly for data intensive scientific applications | |
| Mershad et al. | A study of the performance of a cloud datacenter server | |
| Khetan et al. | A novel survey on load balancing in cloud computing | |
| Jana et al. | Enhancement of QoS for fog computing model aspect of robust resource management | |
| CN116192849A (en) | Heterogeneous accelerator card calculation method, device, equipment and medium | |
| CN114745377A (en) | Edge cloud cluster service system and implementation method | |
| CN111831503A (en) | Monitoring method based on monitoring agent and monitoring agent device | |
| Deepa et al. | Load Balancing Algorithms in cloud computing: A comparative study |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |