CN108563508B - YARN resource allocation method and device - Google Patents

YARN resource allocation method and device Download PDF

Info

Publication number
CN108563508B
CN108563508B CN201810392267.8A CN201810392267A CN108563508B CN 108563508 B CN108563508 B CN 108563508B CN 201810392267 A CN201810392267 A CN 201810392267A CN 108563508 B CN108563508 B CN 108563508B
Authority
CN
China
Prior art keywords
client
spark process
spark
yarn
server
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810392267.8A
Other languages
Chinese (zh)
Other versions
CN108563508A (en
Inventor
户蕾蕾
史宁宁
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
New H3C Big Data Technologies Co Ltd
Original Assignee
New H3C Big Data Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by New H3C Big Data Technologies Co Ltd filed Critical New H3C Big Data Technologies Co Ltd
Priority to CN201810392267.8A priority Critical patent/CN108563508B/en
Publication of CN108563508A publication Critical patent/CN108563508A/en
Application granted granted Critical
Publication of CN108563508B publication Critical patent/CN108563508B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/48Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806Task transfer initiation or dispatching
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/14Session management
    • H04L67/141Setup of application sessions

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Telephonic Communication Services (AREA)

Abstract

The disclosure relates to a YARN resource allocation method and a YARN resource allocation device, wherein the method is applied to a server and comprises the following steps: starting a Thrift server; the method comprises the steps that a Thrift server receives a connection request sent by at least one client; for any client: the Thrift server starts a Spark process corresponding to the client according to the connection request, so that the Spark process requests resources from the YARN; after the Spark process successfully requests resources from the YARN, the Thrift server receives a registration request of the Spark process; and after the Spark process is successfully registered, the Thrift server informs the client that the connection is successful. The method and the system realize that each client has the corresponding resource queue in the YARN, and can improve the task execution efficiency of each client.

Description

YARN resource allocation method and device
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a YARN resource allocation method and apparatus.
Background
In the traditional method for distributing the YARN resources, the YARN organizes the distributed resources in a hierarchical queue mode, so that the method is beneficial to the distribution and sharing of the resources among different queues and can improve the utilization rate of cluster resources. When a client requests resources from the YARN through the Thriftserver, the client applies for the resources from the YARN by using a Spark process started when the Thrift server is started. Resources applied by different clients are in one resource queue, and different resource queues cannot be distinguished according to the clients, so that the time for waiting for the resources when the clients execute tasks is long.
Disclosure of Invention
In view of this, the present disclosure provides a YARN resource allocation method and apparatus, so as to solve the problem of YARN resource waste.
According to an aspect of the present disclosure, there is provided a YARN resource allocation method, which is applied in a server, the method including:
starting a cross-language service deployment framework Thrift server;
the method comprises the steps that a Thrift server receives a connection request sent by at least one client;
for any of the clients:
the thread server starts a Spark process corresponding to the client according to the connection request, so that the Spark process requests resources from the YARN;
after the Spark process successfully requests resources from the YARN, the Thrift server receives a registration request of the Spark process;
and after the Spark process is successfully registered, the Thrift server informs the client that the connection is successful.
According to another aspect of the present disclosure, there is provided a YARN resource allocation apparatus, the apparatus comprising:
the starting module is used for starting a cross-language service deployment framework Thrift server;
a connection request receiving module, configured to receive a connection request sent by at least one client;
a Spark process starting module, configured to, for any one of the clients: starting a Spark process corresponding to the client according to the connection request so that the Spark process requests resources from YARN;
a registration request message receiving module, configured to receive a registration request of the Spark process after the Spark process successfully requests resources from the YARN;
and the connection success message sending module is used for notifying the client that the connection is successful after the Spark process is successfully registered.
In the embodiment of the disclosure, the thread server starts a Spark process corresponding to the client for the client according to the connection request sent by the client. Each client has a corresponding resource queue in the YARN, so that the execution efficiency of the client task is improved, and the waste of the resources of the YARN when no client accesses is avoided.
Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.
Fig. 1 illustrates a flowchart of a YARN resource allocation method according to an embodiment of the present disclosure;
fig. 2 shows a flowchart of a YARN resource allocation method according to an embodiment of the present disclosure;
fig. 3 shows a flowchart of a YARN resource allocation method according to an embodiment of the present disclosure;
fig. 4 shows a flowchart of a YARN resource allocation method according to an embodiment of the present disclosure;
fig. 5 shows a flowchart of a YARN resource allocation method according to an embodiment of the present disclosure;
fig. 6 shows a flowchart of step S90 in the YARN resource allocation method according to an embodiment of the present disclosure;
fig. 7 shows a system architecture block diagram of a YARN resource allocation method according to an embodiment of the present disclosure;
fig. 8 shows a block diagram of a YARN resource allocation apparatus according to an embodiment of the present disclosure.
Detailed Description
Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numbers in the drawings identify functionally the same or similar elements. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.
For a clearer explanation of the disclosure of the present application, the following concepts will be described.
The YARN is a cluster resource management system, and the YARN organizes resources in a hierarchical queue mode, so that the resource is distributed and shared among different queues in an organizing mode, and the utilization rate of cluster resources is further improved.
In the framework of the Spark thread Server, the thread Server provides an interface, and a user can access the Spark process through connecting the thread Server through the interface. When the thread Server is started, a Spark process is started, different clients share the resources of the Spark process, and different clients can share data.
Since the Spark thread Server starts the Spark process when starting, the Spark process uses the default resource queue of YARN. When a plurality of clients are connected with a Spark thread Server through a JDBC interface, SQL statements including task information submitted by each client are executed in a resource queue corresponding to a Spark process, and tasks of each client need to be queued.
Fig. 1 is a flowchart illustrating a YARN resource allocation method according to an embodiment of the present disclosure, which is applied to a server, and as shown in fig. 1, the YARN resource allocation method includes:
step S10, starting a cross-language service deployment framework Thrift server;
step S20, the Thrift server receives a connection request sent by at least one client;
for any of the clients:
step S30, the thread server starts a Spark process corresponding to the client according to the connection request, so that the Spark process requests a resource from YARN;
step S40, after the spare process successfully requests resources from YARN, the thread server receives the registration request of the spare process;
step S50, after the Spark process is successfully registered, the thread server notifies the client that the connection is successful.
In one possible implementation, the client may comprise a client process installed on a client hardware carrier. When the thread server receives a connection request sent by a client, the thread server may start a corresponding Spark process for the client sending the connection request.
The thread server can receive connection requests sent by a plurality of clients. The thread server may start the Spark processes corresponding to the clients one to one according to the connection requests sent by the clients.
In a possible implementation manner, the spare process may be disposed in a server where the thread server is located, or may be disposed in a server other than the server where the thread server is located. After the Spark process is started, the Spark process requests the YARN to allocate a resource queue, and when the resource queue in the YARN is successfully allocated, the Spark process is successfully started. YARN may allocate a separate resource queue for each Spark process. After successfully requesting the resource queue from YARN, the Spark process may send a registration request to the swift server.
And when the Spark process is successfully registered, the Thrift server sends a connection success message to the client, and the client is successfully connected with the Thrift server.
In this embodiment, the thread server starts a corresponding Spark process for the client according to the connection request sent by the client. The Spark process requests the YARN to allocate a resource queue corresponding to the Spark process. And after the resource queue is successfully distributed and the Spark process is successfully registered, the client successfully connects the ThriftServer. Each client has a corresponding resource queue in the YARN, which can improve the task execution efficiency of each client. And simultaneously, the waste of resource queues in the YARN when no client accesses is avoided.
In a possible implementation manner, when the thread server is started, a Java database is provided to connect with the JDBC interface. And the thread server receives a connection request sent by a client through the JDBC interface.
The JDBC interface is a Java application program interface for executing SQL statements and can provide uniform access to various relational databases, and the JDBC interface consists of a group of classes and interfaces written in Java language. When the thread server is started, the Spark process is not started, and the resource queue is not requested to the YARN by the Spark process.
When the thread server is started, a JDBC interface can be provided. The client may send a connection request through the JDBC interface. When the client sends the connection request, the thread server can start corresponding Spark processes for different clients according to the connection request, and request the resource queue from the YARN through the Spark processes, so that the use efficiency of the resource queue in the YARN is improved.
Fig. 2 shows a flowchart of a YARN resource allocation method according to an embodiment of the present disclosure, as shown in fig. 2, step S30 in the YARN resource allocation method includes:
and step S31, the Thrift server calls a Spark Context interface according to the connection request, and starts a Spark process corresponding to the client through a start command.
In a possible implementation manner, the Spark Context interface is an entry of the Spark process, and the Spark Context interface is responsible for interaction between the Spark process and the outside. The thread server may call a Spark Context interface, and start a Spark process corresponding to the client sending the connection request by sending a start command.
Fig. 3 is a flowchart illustrating a YARN resource allocation method according to an embodiment of the present disclosure, where the start command includes a client identifier and a communication address of a thread server, so that the spare process acquires and records the client identifier and the communication address of the thread server after starting; as shown in fig. 3, step S40 in the YARN resource allocation method includes:
step S41, the thread server receives a registration request sent by the Spark process according to the communication address of the thread server, where the registration request carries the client identifier.
In a possible implementation manner, the start command of the thread server to start the spare process may include the client identifier and the communication address of the thread server. The communication address of the Thrift server may include an IP address of the thriftsserver. The client identifier in the start command may be used to identify a client corresponding to a spare process started by the thread server. The communication address of the thread server in the start command can make it more convenient for the Spark process to send a registration request to the thread server.
In a possible implementation manner, according to a start command sent by the thread server, the Spark process requests the YARN for a resource queue, and after the request is successful, sends a registration request to the thread server according to a client identifier in the start command and a communication address of the thread server. The registration request sent by the Spark process may include a client identifier.
In this embodiment, the start command sent by the thread server includes the client identifier and the communication address of the thread server. According to the communication address of the Thrift server, the Thrift server can conveniently establish a communication channel between the Spark process and the Thrift server. According to the client identification, the thread server can conveniently confirm the corresponding relation between the Spark process and the client.
Fig. 4 is a flowchart illustrating a YARN resource allocation method according to an embodiment of the present disclosure, and as shown in fig. 4, after the Spark process is successfully registered, the YARN resource allocation method further includes:
step S60, the thread server establishes a mapping relationship between the client and the spare process according to the client identifier.
In a possible implementation manner, the thread server may establish a mapping relationship between the client and the Spark process according to the client identifier in the registration request. A Netty communication framework may be used between the thread server and Spark processes. The Spark process can determine the Thrift server according to the communication address of the Thrift server in the starting command, and establish a communication channel with the Thrift server. Then, the interaction between the Spark process and the Thrift server is based on the established communication channel.
Fig. 5 is a flowchart illustrating a YARN resource allocation method according to an embodiment of the present disclosure, where after the client succeeds in connecting, as shown in fig. 5, the YARN resource allocation method further includes:
and step S70, the thread server receives the task sent by the client.
And step S80, the thread server sends the task to the Spark process corresponding to the client according to the mapping relation between the client and the Spark process.
Step S90, the thread server obtains an execution result of the task from the spare process, and sends the execution result to the client.
In a possible implementation manner, after the client is successfully connected with the thread server, when the client has a task to be executed, the client may send an SQL statement containing task information to the thread server. The Thriftserver may send the SQL statement including the task information to the spare process corresponding to the client according to the mapping relationship between the client and the spare process.
In a possible implementation manner, the Spark process may execute the task by using the resource queue allocated to the client in the YARN according to the received SQL statement containing the task information.
In one possible implementation manner, the thread server may obtain an execution result of the task from the spare process, where the execution result may include success or failure of execution of the task. The thread server can send the obtained execution result of the task to the client.
When a plurality of clients send SQL statements including task information to the thread server, the thread server may execute tasks using the resource queues corresponding to the clients in the YARN through the corresponding Spark processes of the clients according to the Spark processes corresponding to the clients. The method avoids the waiting phenomenon caused by the fact that the same resource queue in the YARN is needed to be used when a plurality of clients perform tasks concurrently in the traditional YARN resource queue allocation method.
In this embodiment, when a task needs to be executed at a client, the thread server sends the task to a spare process corresponding to the client, and sends an execution result of the task to the spare process to the client. Each client can use the corresponding resource queue in the YARN through the thread server, so that the task execution efficiency of the client is improved.
Fig. 6 shows a flowchart of step S90 in the YARN resource allocation method according to an embodiment of the present disclosure, and as shown in fig. 6, step S90 in the YARN resource allocation method includes:
in step S91, the thread server acquires the execution state of the task from the spare process.
Step S92, the thread server determines the execution result of the task according to the execution state, and sends the execution result to the client.
In one possible implementation, the thread server may periodically obtain the execution status of the task from the spare process. Different periods may be set for different tasks. The same period may also be set for all tasks. The execution state of the task may include any one of in-execution, execution success, and execution failure.
When the execution state of the task acquired by the thread server is successful in execution or failed in execution, the successful in execution or the failed in execution can be determined as an execution result of the task, and the execution result is sent to the client.
Application example:
fig. 7 is a system architecture block diagram of a YARN resource allocation method according to an embodiment of the present disclosure, and as shown in the system architecture diagram shown in fig. 7, a client user1, a client user2, and a client user3 access a thread server. The three clients in the figure can be simultaneously accessed to the Thrift server, and can also be respectively accessed to the Thrift server. Thriftserver starts Spark process 1 for client user1 by calling Spark Context1, Spark process 2 for client user2 by calling Spark Context2, and Spark process 3 for client user3 by calling Spark Context 3.
In YARN, Spark process 1 corresponds to resource queue1, Spark process 2 corresponds to resource queue2, and Spark process 3 corresponds to resource queue 3. When a client has a task to execute, the corresponding resource queue can be called in YARN to execute.
Fig. 8 shows a block diagram of a YARN resource allocation apparatus according to an embodiment of the present disclosure, as shown in fig. 8, the YARN resource allocation apparatus includes:
the starting module 10 is used for starting a cross-language service deployment framework Thrift server;
a connection request receiving module 20, configured to receive a connection request sent by at least one client;
a Spark process starting module 30, configured to, for any one of the clients: starting a Spark process corresponding to the client according to the connection request so that the Spark process requests resources from YARN;
a registration request message receiving module 40, configured to receive a registration request of the Spark process after the Spark process successfully requests resources from the YARN;
a connection success message sending module 50, configured to notify the client that the connection is successful after the Spark process is successfully registered.
In a possible implementation manner, the spare process starting module 30 includes:
and the first Spark process starting module is used for calling a Spark Context interface according to the connection request and starting a Spark process corresponding to the client through a starting command.
In a possible implementation manner, the start command includes a client identifier and a communication address of a thread server, so that the spare process obtains and records the client identifier and the communication address of the thread server after starting; the registration request message receiving module 40 includes:
and the first registration request message receiving submodule is used for receiving a registration request sent by the Spark process according to the communication address of the thread server, and the registration request carries the client identifier.
In one possible implementation, the apparatus further includes:
and the mapping relation establishing module is used for establishing the mapping relation between the client and the Spark process according to the client identification after the Spark process is successfully registered.
In one possible implementation, the apparatus further includes:
the task receiving module is used for receiving the tasks sent by the client;
the task sending module is used for sending the task to the Spark process corresponding to the client according to the mapping relation between the client and the Spark process;
and the execution result sending module is used for obtaining the execution result of the task from the Spark process and sending the execution result to the client.
In a possible implementation manner, the execution result sending module includes:
the execution state acquisition submodule is used for acquiring the execution state of the task from the Spark process;
and the first execution result sending submodule is used for determining the execution result of the task according to the execution state and sending the execution result to the client.
Having described embodiments of the present disclosure, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein were chosen in order to best explain the principles of the embodiments, the practical application, or technical improvements to the techniques in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims (10)

1. A resource coordinator YARN resource allocation method, characterized in that the method comprises:
starting a cross-language service deployment framework Thrift server;
the method comprises the steps that a Thrift server receives a connection request sent by at least one client;
for any of the clients:
the thread server starts a Spark process corresponding to the client according to the connection request, so that the Spark process requests resources from the YARN; the client-side is in one-to-one correspondence with the resource queues in the YARN;
after the Spark process successfully requests resources from the YARN, the Thrift server receives a registration request of the Spark process; the registration request carries the client identifier;
after the Spark process is successfully registered, the Thrift server informs the client that the connection is successful;
after the Spark process is successfully registered, the method further includes:
and the thread server establishes the mapping relation between the client and the Spark process according to the client identifier.
2. The method according to claim 1, wherein the thread server starts a Spark process corresponding to the client according to the connection request, and the method comprises:
and the Thrift server calls a Spark Context interface according to the connection request and starts a Spark process corresponding to the client through a starting command.
3. The method according to claim 2, wherein the start command includes a client identifier and a communication address of the Thriftserver, so that the spare process acquires and records the client identifier and the communication address of the Thriftserver after starting:
the thread server receives a registration request of the Spark process, and the registration request comprises the following steps:
and the thread server receives a registration request sent by the Spark process according to the communication address of the thread server.
4. The method of claim 1, wherein after the client connection is successful, the method further comprises: the thread server receives the tasks sent by the client;
the thread server sends the task to a Spark process corresponding to the client according to the mapping relation between the client and the Spark process;
and the thread server acquires the execution result of the task from the Spark process and sends the execution result to the client.
5. The method according to claim 4, wherein the thread server obtains an execution result of the task from the Spark process and sends the execution result to the client, and the method includes:
the thread server acquires the execution state of the task from the Spark process;
and the thread server determines the execution result of the task according to the execution state and sends the execution result to the client.
6. An apparatus for YARN resource allocation, the apparatus comprising: the starting module is used for starting a cross-language service deployment framework Thrift server; a connection request receiving module, configured to receive a connection request sent by at least one client:
a Spark process starting module, configured to, for any one of the clients: starting a Spark process corresponding to the client according to the connection request so that the Spark process requests resources from YARN; the client-side is in one-to-one correspondence with the resource queues in the YARN;
a registration request message receiving module, configured to receive a registration request of the Spark process after the Spark process successfully requests resources from the YARN;
a connection success message sending module, configured to notify the client that the connection is successful after the Spark process is successfully registered;
and the mapping relation establishing module is used for establishing the mapping relation between the client and the Spark process according to the client identification after the Spark process is successfully registered.
7. The apparatus of claim 6, wherein the Spark process initiating module comprises:
and the first Spark process starting module is used for calling a Spark Context interface according to the connection request and starting a Spark process corresponding to the client through a starting command.
8. The apparatus according to claim 7, wherein the start command includes a client identifier and a communication address of Thriftserver, so that the client identifier and the communication address of Thriftserver are obtained and recorded after the Spark process is started; the registration request message receiving module includes:
and the second registration request message receiving submodule is used for receiving a registration request sent by the Spark process according to the communication address of the thread server, and the registration request carries the client identifier.
9. The apparatus of claim 6, further comprising: the task receiving module is used for receiving the tasks sent by the client;
the task sending module is used for sending the task to the Spark process corresponding to the client according to the mapping relation between the client and the Spark process;
and the execution result sending module is used for obtaining the execution result of the task from the Spark process and sending the execution result to the client.
10. The apparatus of claim 9, wherein the execution result sending module comprises: the execution state acquisition submodule is used for acquiring the execution state of the task from the Spark process;
and the first execution result sending submodule is used for determining the execution result of the task according to the execution state and sending the execution result to the client.
CN201810392267.8A 2018-04-27 2018-04-27 YARN resource allocation method and device Active CN108563508B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810392267.8A CN108563508B (en) 2018-04-27 2018-04-27 YARN resource allocation method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810392267.8A CN108563508B (en) 2018-04-27 2018-04-27 YARN resource allocation method and device

Publications (2)

Publication Number Publication Date
CN108563508A CN108563508A (en) 2018-09-21
CN108563508B true CN108563508B (en) 2020-03-17

Family

ID=63537072

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810392267.8A Active CN108563508B (en) 2018-04-27 2018-04-27 YARN resource allocation method and device

Country Status (1)

Country Link
CN (1) CN108563508B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109558446B (en) * 2018-12-13 2020-12-22 杭州数梦工场科技有限公司 Job request method, job request device, electronic equipment and storage medium
CN109684092B (en) * 2018-12-24 2023-03-10 新华三大数据技术有限公司 Resource allocation method and device
CN110362391B (en) * 2019-06-12 2021-08-13 北京达佳互联信息技术有限公司 Resource scheduling method and device, electronic equipment and storage medium
CN113127289B (en) * 2019-12-31 2024-04-12 奇安信科技集团股份有限公司 Resource management method, computer equipment and storage medium based on YARN cluster

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104407921A (en) * 2014-12-25 2015-03-11 浪潮电子信息产业股份有限公司 Time-based method for dynamically scheduling yann task resources
US10789547B2 (en) * 2016-03-14 2020-09-29 Business Objects Software Ltd. Predictive modeling optimization
CN107544844A (en) * 2016-06-27 2018-01-05 中兴通讯股份有限公司 A kind of method and device of lifting Spark Operating ettectiveness
CN106790084A (en) * 2016-12-22 2017-05-31 中国科学院电子学研究所苏州研究院 A kind of heterogeneous resource integrated framework and its integrated approach based on ICE middlewares
CN107797874B (en) * 2017-10-12 2021-04-27 南京中新赛克科技有限责任公司 Resource management and control method based on embedded jetty and spark on grow framework

Also Published As

Publication number Publication date
CN108563508A (en) 2018-09-21

Similar Documents

Publication Publication Date Title
CN108563508B (en) YARN resource allocation method and device
CN110351384B (en) Big data platform resource management method, device, equipment and readable storage medium
US10686728B2 (en) Systems and methods for allocating computing resources in distributed computing
US20160041923A1 (en) Inter-Manycore Communications Method and System
CN107203560B (en) Database, multi-database operation transaction consistency ensuring method and system
CN106817432B (en) Method, system and equipment for elastically stretching virtual resources in cloud computing environment
CN101453404A (en) Method and apparatus for implementing load balance
CN111225007B (en) Database connection method, device and system
CN111431970B (en) Resource allocation method, device, equipment and storage medium based on micro-service architecture
CN111163140A (en) Method, apparatus and computer readable storage medium for resource acquisition and allocation
CN111831504A (en) Monitoring method, monitoring apparatus, computer device, and medium
US11442756B2 (en) Common service resource application method, related device, and system
CN114615308A (en) RPC-based asynchronous multithreading concurrent network communication method and device
CN111381868B (en) Method and device for implementing embedded system driving layer
CN108958933B (en) Configuration parameter updating method, device and equipment of task executor
CN111930482A (en) Task processing method, device and equipment based on node cluster and storage medium
KR101560879B1 (en) System for managing task and executing service of steel process middleware
CN112328403B (en) SparkContext configuration method, sparkContext configuration device and server
US7328291B2 (en) System and method for controlling the service engagement in a data bus system
CN112689008A (en) Network connection pool management method, device, equipment and storage medium
CN114911632A (en) Method and system for controlling inter-process communication
US7912922B2 (en) Globally unique instance identification
CN114064317A (en) Node calling method in distributed system and related device
CN111340613B (en) Job processing method, job processing system and storage medium
JP3079241B2 (en) Communication control device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant