CN116028178A - Execution method and device of job in private cloud environment - Google Patents
Execution method and device of job in private cloud environment Download PDFInfo
- Publication number
- CN116028178A CN116028178A CN202211720428.4A CN202211720428A CN116028178A CN 116028178 A CN116028178 A CN 116028178A CN 202211720428 A CN202211720428 A CN 202211720428A CN 116028178 A CN116028178 A CN 116028178A
- Authority
- CN
- China
- Prior art keywords
- target
- job
- node
- information
- private cloud
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The disclosure provides an execution method of a job in a private cloud environment, and relates to the technical fields of artificial intelligence such as big data, cloud service, cloud computing, deep learning and the like. The execution method of the job comprises the following steps: acquiring a target job through a job query interface; determining a target machine cluster in the candidate machine clusters in the private cloud environment according to the target job; acquiring node information of nodes in the target machine cluster through a node query interface, and determining target nodes in the target machine cluster according to the node information and the target operation; and sending target data corresponding to the target job to the target node through a job creation interface, so that the target node creates the target job and processes the target data. The method and the device can realize execution of the operation under the condition of no invasion of the private cloud environment, enhance the safety of the private cloud and improve the efficiency of the private cloud in operation execution.
Description
Technical Field
The disclosure relates to the technical field of data processing, in particular to the technical field of artificial intelligence such as big data, cloud service, cloud computing, deep learning and the like. Provided are a method and device for executing a job in a private cloud environment, an electronic device and a readable storage medium.
Background
With the rapid development of cloud computing services, more and more enterprises use private clouds to perform cloud computing services to meet their computing needs. However, since the private cloud has a strict limitation in use, how to execute the job on the premise of ensuring the security of the private cloud is a technical problem to be solved at present.
Disclosure of Invention
According to a first aspect of the present disclosure, there is provided a method for executing a job in a private cloud environment, including: acquiring a target job through a job query interface; determining a target machine cluster in the candidate machine clusters in the private cloud environment according to the target job; acquiring node information of nodes in the target machine cluster through a node query interface, and determining target nodes in the target machine cluster according to the node information and the target operation; the method comprises the steps of carrying out a first treatment on the surface of the And sending target data corresponding to the target job to the target node through a job creation interface, so that the target node creates the target job and processes the target data.
According to a second aspect of the present disclosure, there is provided an execution apparatus of a job in a private cloud environment, including: the acquisition unit is used for acquiring the target operation through the operation inquiry interface; the first determining unit is used for determining a target machine cluster in the candidate machine clusters in the private cloud environment according to the target job; the second determining unit is used for obtaining node information of nodes in the target machine cluster through a node query interface and determining target nodes in the target machine cluster according to the node information and the target job; and the execution unit is used for sending the target data corresponding to the target job to the target node through a job creation interface so as to be used for creating the target job by the target node and processing the target data.
According to a third aspect of the present disclosure, there is provided an electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described above.
According to a fourth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method as described above.
According to a fifth aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements a method as described above.
According to a sixth aspect of the present disclosure, an execution system for a job in a private cloud environment is provided, including the above electronic device, a machine cluster in the private cloud environment, and a storage device.
According to the technical scheme, under the condition that invasive operations such as component installation and the like are not required to be performed on the machine cluster in the private cloud environment, the purpose of executing the operation in the private cloud environment is achieved, the safety of the private cloud is enhanced, the target data are directly sent to the determined target node, the machine cluster in the private cloud environment is not required to perform the determination of the target node, and therefore the efficiency of the private cloud in executing the operation is improved.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.
Drawings
The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
FIG. 1 is a schematic diagram according to a first embodiment of the present disclosure;
FIG. 2 is a schematic diagram according to a second embodiment of the present disclosure;
FIG. 3 is a schematic diagram according to a third embodiment of the present disclosure;
FIG. 4 is a schematic diagram according to a fourth embodiment of the present disclosure;
FIG. 5 is a schematic diagram according to a fifth embodiment of the present disclosure;
FIG. 6 is a schematic diagram according to a sixth embodiment of the present disclosure;
FIG. 7 is a schematic diagram according to a seventh embodiment of the present disclosure;
fig. 8 is a block diagram of an electronic device for implementing a method of executing a job in a private cloud environment of an embodiment of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic diagram according to a first embodiment of the present disclosure. As shown in fig. 1, the method for executing a job in the private cloud environment of the present embodiment specifically includes the following steps:
s101, acquiring a target job through a job query interface;
s102, determining a target machine cluster in the candidate machine clusters in the private cloud environment according to the target job;
s103, acquiring node information of nodes in the target machine cluster through a node query interface, and determining target nodes in the target machine cluster according to the node information and the target job;
s104, sending target data corresponding to the target job to the target node through a job creation interface, so that the target node can be used for creating the target job and processing the target data.
According to the method for executing the operation in the private cloud environment, the acquisition of the target operation, the determination of the target node and the transmission of the target data are completed through different interfaces, so that the purpose of executing the operation in the private cloud environment is achieved under the condition that invasive operations such as component installation and the like are not required to be performed on a machine cluster in the private cloud environment, the safety of the private cloud can be enhanced, the target data are directly transmitted to the target node in the target machine cluster, the determination of the target node is not required to be performed by the machine cluster in the private cloud environment, and the efficiency of the private cloud in executing the operation is improved.
The execution main body of the execution method of the job in the private cloud environment of the embodiment may be a server or a terminal device connected with the private cloud through a plurality of interfaces; the server or the terminal equipment can realize the purpose of executing the job in the private cloud under the condition of no intrusion into the private cloud only by relying on the query authority of the nodes in the machine cluster in the private cloud environment.
In the embodiment, when executing S101, a target job is acquired through the job query interface, where the acquired target job is a task to be executed in the machine cluster, for example, a model training job, a voice recognition job, and the like, the job type of the target job is not limited in the embodiment.
The server or the terminal device in this embodiment is connected to a storage device through a job query interface, and the storage device stores a job created by a user and job data corresponding to a different job, for example, a training sample corresponding to a model training job, voice data corresponding to a voice recognition job, and the like.
The number of target jobs obtained in the step S101 may be one, that is, one target job is executed in the private cloud environment; the method can also be used for executing a plurality of target jobs simultaneously in a private cloud environment, and the job types of the plurality of target jobs can be the same or different.
In this embodiment, after the execution of S101, the target job is acquired through the job query interface, the acquired target job may be registered, so that operations such as executing the target job, managing the target job, and synchronizing the status of the target job in the private cloud environment are completed according to the registration information of the target job; wherein different target jobs have different registration information.
In the embodiment, when the S101 registration is performed on the target job, a static registration mode may be adopted, or a dynamic registration mode may be adopted; wherein, the static registration is local registration, and the registration is carried out through a configuration file located locally; dynamic registration is registered through a registration service provided by an RPC (Remote Procedure Call Protocol ) server.
After the registration of the target job is completed in S101, the present embodiment may consider the target job to be effective, and further implement operations such as executing the target job, managing the target job, synchronizing the status of the target job, and the like according to the registration information of the target job.
In this embodiment, after S101 is executed to obtain the target job, the target plug-in corresponding to the target job may also be determined; the job plug-in the embodiment corresponds to different job types, and the target plug-in can be determined according to the job type of the target job, so that operations such as executing the target job, managing the target job, synchronizing the state of the target job and the like through the target plug-in are realized, and convenience in executing, managing or synchronizing the target job is improved.
The plug-ins corresponding to different job types in this embodiment may support an in-tree mode (i.e., running together with source codes corresponding to the method provided in this embodiment) or an out-of-tree mode (i.e., running separately and independently from source codes corresponding to the method provided in this embodiment).
After executing S101 to obtain a target job, executing S102 to determine a target machine cluster in a candidate machine cluster in a private cloud environment according to the target job; the private cloud environment in this embodiment may include multiple candidate machine clusters, and different candidate machine clusters may have the same cluster type, or may have different cluster types, for example, kubernetes cluster, yarn cluster, and the like.
In the embodiment, when executing S102 to determine the target machine cluster in the candidate machine clusters in the private cloud environment according to the target job, the optional implementation manner may be: acquiring job information of a target job, wherein the job information can comprise information such as a job type, a job appointed cluster, a resource request amount of the job, job appointed node identification information, a job appointed node label and the like; and taking the machine cluster matched with the acquired operation information in the candidate machine clusters as a target machine cluster.
That is, the present embodiment may determine the target machine cluster according to the job information of the target job, so that the determined target machine cluster corresponds to the job type or job-specified cluster of the target job, which improves the accuracy of the determined target machine cluster.
In the embodiment, when executing S102 to use the machine cluster matched with the acquired job information in the candidate machine cluster as the target machine cluster, the following manner may be adopted: acquiring a resource request amount of a target job according to job information, wherein the resource request amount is the total amount of machine resources required by executing the target job, and the machine resources can be memory resources, CPU resources, GPU resources and the like; and determining a target machine cluster from the candidate machine clusters according to the resource request quantity and the resource quantity of the candidate machine clusters.
That is, the embodiment determines the target machine cluster from the candidate machine clusters by the resource request amount in the job information, so that the determined target machine cluster has enough machine resources to execute the target job, and the success rate of executing the target job is improved.
In the embodiment, when S102 is executed to determine the target machine cluster in the candidate machine clusters according to the resource request amount and the resource amount of the candidate machine clusters, the candidate machine cluster whose resource amount can satisfy the resource request amount may be used as the target machine cluster; if there are a plurality of candidate machine clusters capable of satisfying the resource request amount, the candidate machine cluster having the lowest resource utilization rate may be used as the target machine cluster, or one candidate machine cluster selected at random may be used as the target machine cluster.
In the embodiment, when executing S102 to determine the target machine cluster in the candidate machine clusters in the private cloud environment according to the target job, the following may be included: the private cloud environment is subjected to authority verification, whether the private cloud environment can execute target operation or not can be verified according to the user name corresponding to the private cloud environment, whether the private cloud environment has the authority of using a cloud product corresponding to the target operation or not can be verified, whether a neural network model corresponding to the target operation exists in the private cloud environment or not can also be verified, and the like; in the case that the permission verification result is determined that the private cloud environment is capable of executing the target job (for example, the private cloud environment has permission to use the related cloud product, or a related neural network model exists in the private cloud environment), an operation of determining a target machine cluster in the candidate machine clusters in the private cloud environment according to the target job is executed.
That is, by means of verifying the private cloud environment, the embodiment ensures that the private cloud environment can execute the target job, avoids the problem that the job cannot be executed because the private cloud environment has no relevant authority after the determination of the target node is completed, and improves the success rate of job execution.
After executing S102 to determine a target machine cluster, executing S103 to obtain node information of nodes in the target machine cluster through a node query interface, and determining target nodes in the target machine cluster according to the node information and target jobs; the nodes in this embodiment are machines located in a cluster of machines.
In this embodiment, when executing S103 to obtain node information of a node in the target machine cluster through the node query interface, and determine a target node in the target machine cluster according to the node information and the target job, the optional implementation manner may be: acquiring the job information of the target job, wherein the acquired job information can be the resource request quantity of the job, the node identification information appointed by the job, the node label appointed by the job and the like; taking the node corresponding to the node information matched with the job information as a candidate node; a target node is determined from the candidate nodes.
That is, in this embodiment, node information of a node included in a target machine cluster is obtained through a node query interface connected with a private cloud environment, and then a target node is determined according to the node information and job information of a target job, so that the purpose of relying on node query permission in the cluster is achieved, and an additional component is not required to be added in the private cloud environment to invade the private cloud environment, thereby enhancing security of the private cloud environment.
In this embodiment, when executing S103 to use the node corresponding to the node information matched with the job information as the candidate node, the optional implementation manner may be: determining the resource request amount of the target job according to the job information; determining the available resource quantity of the node according to the node information; and taking the node with the available resource quantity larger than the resource request quantity as a candidate node.
That is, the present embodiment determines the candidate node through the resource request amount of the target job and the available resource amount of the node, so that the determined candidate node is ensured to have enough resources to process the target data corresponding to the target job, and further the success rate of job execution is improved.
In this embodiment, when executing S103, the node with the available resource amount greater than the resource request amount is used as the candidate node, the optional implementation manner may be: taking a node with the available resource amount larger than the resource request amount as an initial node; determining identification information (such as node name) and/or labels of the initial nodes according to the node information of the initial nodes; and taking the initial node corresponding to the identification information and/or the label matched with the job information as a candidate node.
That is, in this embodiment, after the node whose available resource amount can satisfy the resource request amount is taken as the initial node, the candidate node may be further determined from the initial node according to the identification information and/or the label of the node, so that the determined candidate node is more matched with the target job if the target job can be executed.
In this embodiment, when executing S103, the initial node corresponding to the identification information and/or the label matched with the job information is used as the candidate node, the initial node corresponding to the identification information and the label of the job-specified node may be used as the candidate node, and the initial node corresponding to the label of the job-specified node may be used as the candidate node.
In the embodiment, when determining the target node from the candidate nodes in S103, the candidate nodes may be ordered according to the resource utilization rate or the available resource amount of the candidate nodes, and then the candidate nodes ranked in the previous N bits are used as the target node, where N is a positive integer greater than or equal to 1; multiple nodes can also be randomly selected from the candidate nodes to serve as target nodes.
The present embodiment, after determining the target node in step S103, transmits target data corresponding to the target job to the target node through the job creation interface in step S104 for the target node to create the target job and process the target data.
In the embodiment, when executing S104, the target data corresponding to the target job may be first acquired through the job query interface, and then the acquired target data may be sent to the target node through the job creation interface.
In the embodiment, when executing S105 to send the target data corresponding to the target job to the target node through the job creation interface, the optional implementation manner may be: splitting the target job into a plurality of sub-target jobs; sub-target data corresponding to the plurality of sub-target jobs is sent to the plurality of target nodes through the job creation interface for the plurality of target nodes to create the sub-target jobs and process the sub-target data, respectively.
That is, in this embodiment, the target job may be split, and then the sub-target jobs obtained by splitting are executed by the plurality of target nodes, so as to achieve the purpose of parallel processing of the target job, thereby improving the execution efficiency of the target job.
In this embodiment, when executing S105 to send the target data corresponding to the target job to the target node through the job creation interface, if the target plugin corresponding to the target job is determined, in this embodiment, the target data corresponding to the target job is sent to the target node through the job creation interface and the target plugin when executing S105, specifically, the target data is sent to the target plugin through the job creation interface first, and then the target plugin sends the received target data to the target node.
Fig. 2 is a schematic diagram according to a second embodiment of the present disclosure. As shown in fig. 1, the method for executing a job in the private cloud environment of the present embodiment may further include the following:
s201, acquiring an operation request corresponding to the target job;
s202, the operation request is sent to the target node through a job management interface, so that the target node can process the target job according to the operation request.
That is, the present embodiment performs interaction with the target node through the job management interface, so that the purpose of operating the target job being executed in the target node can be achieved, and flexibility in executing the job in the private cloud environment can be improved.
The embodiment executes the operation request acquired in S201, which is an operation request made by the user for the target job, and the operation corresponding to the operation request may be deleting the target job, stopping the target job, updating the target job, or the like; in the case where there are a plurality of target jobs, the present embodiment can determine operation requests corresponding to different target jobs according to registration information or target plug-ins corresponding to the target jobs.
In this embodiment, when executing S202 to send an operation request to a target node through a job management interface, if a target plug-in corresponding to a target job is determined, the present embodiment may send the operation request to the target node through the job management interface and the target plug-in when executing S202, specifically, first send the operation request to the target plug-in through the job management interface, and then send the received operation request to the target node by the target plug-in.
It can be understood that the job query interface, the node query interface, the job creation interface, the job management interface, and the like in this embodiment may be a unified interface, that is, the unified interface can implement functions of job query, node query, job creation, job management, and the like.
Fig. 3 is a schematic diagram according to a third embodiment of the present disclosure. As shown in fig. 3, the method for executing a job in the private cloud environment of the present embodiment may further include the following:
s301, acquiring a job state sent by the target node through a job management interface;
s302, the job state is sent to a storage device through the job inquiry interface so as to be used for displaying the job state by the storage device.
That is, in this embodiment, the job query interface and the job management interface interact with the target node, so that the job state of the target node when executing the target job can be synchronized, so that the user can more clearly know the execution state of the target job, and further can perform corresponding operations on the target job according to the execution state, thereby improving convenience when executing the target job in the private cloud environment.
The job status received in S301 may be whether the target node completes execution of the target job, or whether the target node has a fault when executing the target job, or the like; in the case where there are a plurality of target jobs, the present embodiment can determine job states corresponding to different target jobs from registration information or job plug-ins corresponding to the target jobs.
In this embodiment, after executing S302 to send the job status to the storage device through the job query interface, the storage device may display the job status to the user, and the user may perform operations such as updating, deleting, stopping, etc. on the target job according to the displayed job status.
In this embodiment, when the job state sent by the target node is obtained through the job management interface in S301, if the target plug-in corresponding to the target job is determined, the job state sent by the target node in the job plug-in is obtained through the job management interface and the target plug-in S301, specifically, the job state sent by the target node in the job plug-in is obtained through the job management interface, that is, the target node will send the job state of the target job to the job plug-in corresponding to the target job, so as to achieve the purpose of obtaining the job state from the job plug-in through the job management interface.
Fig. 4 is a schematic diagram according to a fourth embodiment of the present disclosure. Fig. 4 is a schematic diagram of the dynamic registration of the job plug-in the present embodiment: the target job sends a registration request to an RPC server responsible for registration; the RPC server performs dynamic registration according to the registration request, and determines a target plug-in corresponding to the target job after the registration of the target job is completed, so that operations such as executing the target job, managing the target job, synchronizing the state of the target job and the like through the target plug-in are realized.
Fig. 5 is a schematic diagram according to a fifth embodiment of the present disclosure. Fig. 5 shows a structure diagram of executing a job in the private cloud environment according to the present embodiment: a device (such as a server or terminal equipment) for executing the operation in the private cloud environment is connected with the storage device through an operation inquiry interface and is used for acquiring target operation, acquiring target data, synchronizing operation states and the like; the device for executing the job in the private cloud environment realizes job creation, job operation (update, deletion, stop), job state synchronization and the like through a unified interface (comprising a node query interface, a job creation interface, a job management interface and the like); if there is a job plug-in corresponding to the target job, the present embodiment may connect with the job plug-in through a corresponding interface (a job creation interface, a job management interface, etc.), and further implement interaction with the target machine cluster through the job plug-in, for example, send the target data to the job plug-in through the job creation interface, and then send the target data to the target node by the job plug-in.
Fig. 6 is a schematic diagram according to a sixth embodiment of the present disclosure. Fig. 6 shows a flowchart of executing a job in the private cloud environment of the present embodiment: after creating a target job in the storage device, the device executing the job in the private cloud environment starts a scheduling period, wherein the scheduling period is a period of time for transmitting target data corresponding to the target job to a target node for execution; a device executing the job in the private cloud environment requests a job query interface in the unified interface to acquire the created target job; then determining a target machine cluster from the candidate machine clusters according to a judging result of whether the target job designates the machine cluster or not, and determining a target node from the target machine cluster through a request node query interface; after the determination of the target node is completed, the target data corresponding to the target job is transmitted to the target node by requesting the job creation interface in the unified interface, so that the target node processes the target data after creating the target job.
Fig. 7 is a schematic diagram according to a seventh embodiment of the present disclosure. As shown in fig. 7, an execution device 700 of a job in the private cloud environment of the present embodiment includes:
an acquiring unit 701, configured to acquire a target job through a job query interface;
a first determining unit 702, configured to determine, according to the target job, a target machine cluster in the candidate machine clusters in the private cloud environment;
a second determining unit 703, configured to obtain node information of a node in the target machine cluster through a node query interface, and determine a target node in the target machine cluster according to the node information and the target job;
an execution unit 704 is configured to send, through a job creation interface, target data corresponding to the target job to the target node, for the target node to create the target job and process the target data.
The acquiring unit 701 acquires a target job through the job query interface, where the acquired target job is a task to be executed in the machine cluster, for example, a model training job, a voice recognition job, and the like, and the job type of the target job is not limited in this embodiment.
The number of target jobs acquired by the acquisition unit 701 may be one, that is, one target job is executed in the private cloud environment; the method can also be used for executing a plurality of target jobs simultaneously in a private cloud environment, and the job types of the plurality of target jobs can be the same or different.
After the acquiring unit 701 acquires the target job through the job query interface, the acquired target job may be registered, so that operations such as executing the target job, managing the target job, synchronizing the state of the target job, and the like in the private cloud environment are completed according to the registration information of the target job; wherein different target jobs have different registration information.
When the acquisition unit 701 registers the target job, a static registration mode or a dynamic registration mode may be adopted; wherein, the static registration is local registration, and the registration is carried out through a configuration file located locally; dynamic registration is registered through a registration service provided by an RPC (Remote Procedure Call Protocol ) server.
After the acquisition unit 701 completes registration of the target job, the target job can be considered to be effective, and further operations such as executing the target job, managing the target job, synchronizing the state of the target job, and the like are realized according to registration information of the target job.
The acquiring unit 701 may further determine a target plug-in corresponding to the target job after acquiring the target job; the job plug-in the embodiment corresponds to different job types, and the target plug-in can be determined according to the job type of the target job, so that operations such as executing the target job, managing the target job, synchronizing the state of the target job and the like through the target plug-in are realized, and convenience in executing, managing or synchronizing the target job is improved.
The job plug-ins corresponding to different job types in this embodiment may support an in-tree mode (i.e., running together with source codes corresponding to the method provided in this embodiment) or an out-of-tree mode (i.e., running separately and independently from source codes corresponding to the method provided in this embodiment).
In the embodiment, after the acquisition unit 701 acquires the target job, the first determination unit 702 determines a target machine cluster in the candidate machine clusters in the private cloud environment according to the target job; in this embodiment, there are multiple candidate machine clusters in the private cloud environment, and different candidate machine clusters may have the same cluster type, or may have different cluster types, for example, kubernetes cluster, yarn cluster, and the like.
When determining a target machine cluster in the candidate machine clusters in the private cloud environment according to the target job, the first determining unit 702 may adopt the following alternative implementation manners: acquiring job information of a target job; and taking the machine cluster matched with the acquired operation information in the candidate machine clusters as a target machine cluster.
That is, the first determining unit 702 may determine the target machine cluster according to the job information of the target job, so that the determined target machine cluster is more matched with the target job, and accuracy of the determined target machine cluster is improved.
The first determining unit 702 may further employ, when a machine cluster matching the acquired job information among the candidate machine clusters is taken as the target machine cluster, the following manner: acquiring the resource request quantity of the target job according to the job information; and determining a target machine cluster from the candidate machine clusters according to the resource request quantity and the resource quantity of the candidate machine clusters.
That is, the first determining unit 702 determines the target machine cluster from among the candidate machine clusters by the resource request amount in the job information, so that the determined target machine cluster has sufficient machine resources to execute the target job, improving the success rate of execution of the target job.
Wherein, when determining the target machine cluster in the candidate machine clusters according to the resource request amount and the resource amount of the candidate machine clusters, the first determining unit 702 may use the candidate machine cluster whose resource amount can satisfy the resource request amount as the target machine cluster; if there are a plurality of candidate machine clusters capable of satisfying the resource request amount, the candidate machine cluster having the lowest resource utilization rate may be used as the target machine cluster, or one candidate machine cluster selected at random may be used as the target machine cluster.
The first determining unit 702 may further include, when determining a target machine cluster from among the candidate machine clusters in the private cloud environment according to the target job: performing authority verification on the private cloud environment; and under the condition that the permission verification result is that the private cloud environment can execute the target job, executing the operation of determining the target machine cluster in the candidate machine clusters in the private cloud environment according to the target job.
That is, the first determining unit 702 ensures that the private cloud environment can execute the target job by verifying the private cloud environment, so that the problem that the job cannot be executed because the private cloud environment has no relevant authority after the determination of the target node is completed is avoided, and the success rate of executing the job is improved.
In the embodiment, after the first determining unit 702 determines the target machine cluster, the second determining unit 703 obtains node information of nodes in the target machine cluster through a node query interface, and determines target nodes in the target machine cluster according to the node information and the target job; the nodes in this embodiment are machines located in a cluster of machines.
Specifically, when the second determining unit 703 obtains node information of a node in the target machine cluster through the node query interface, and determines a target node in the target machine cluster according to the node information and the target job, the optional implementation manner may be: acquiring job information of a target job; taking the node corresponding to the node information matched with the job information as a candidate node; a target node is determined from the candidate nodes.
That is, the second determining unit 703 obtains node information of a node included in the target machine cluster through a node query interface connected to the private cloud environment, and further determines the target node according to the node information and the job information, so as to achieve the purpose of relying only on the node query authority in the cluster, and does not need to add a large number of additional components in the private cloud environment to generate invasiveness to the private cloud environment, thereby enhancing the security of the private cloud environment.
The second determining unit 703 may adopt, when a node corresponding to the node information matched with the job information is taken as a candidate node, alternative implementation manners as follows: determining the resource request amount of the target job according to the job information; determining the available resource quantity of the node according to the node information; and taking the node with the available resource quantity larger than the resource request quantity as a candidate node.
That is, the second determining unit 703 determines the candidate node by the resource request amount of the target job and the available resource amount of the node, ensures that the determined candidate node has sufficient resources to process the target data corresponding to the target job, and further improves the success rate of job execution.
The second determining unit 703 may adopt, when a node whose available resource amount is greater than the resource request amount is used as a candidate node, the following alternative implementation modes: taking a node with the available resource amount larger than the resource request amount as an initial node; determining identification information and/or labels of the initial nodes according to the node information of the initial nodes; and taking the initial node corresponding to the identification information and/or the label matched with the job information as a candidate node.
When the initial node corresponding to the label and/or the identification information matching the job information is used as the candidate node, the second determining unit 703 may use the initial node corresponding to the label and the identification information of the job-specific node as the candidate node, or may use the initial node corresponding to the label and the label of the job-specific node as the candidate node.
The second determining unit 703 may sort the candidate nodes according to the resource utilization or the available resource amount of the candidate nodes when determining the target node from the candidate nodes, and further take the candidate node ranked in the previous N bits as the target node, where N is a positive integer greater than or equal to 1; multiple nodes can also be randomly selected from the candidate nodes to serve as target nodes.
The present embodiment, after the target node is determined by the second determination unit 703, transmits target data corresponding to the target job to the target node through the job creation interface by the execution unit 704 for the target node to create the target job and process the target data.
The execution unit 704 may first acquire target data corresponding to the target job through the job query interface, and then send the acquired target data to the target node through the job creation interface.
When the execution unit 704 sends the target data corresponding to the target job to the target node through the job creation interface, alternative implementation manners may be: splitting the target job into a plurality of sub-target jobs; sub-target data corresponding to the plurality of sub-target jobs is sent to the plurality of target nodes through the job creation interface for the plurality of target nodes to create the sub-target jobs and process the sub-target data, respectively.
That is, the execution unit 704 may further split the target job, and then execute the sub-target job obtained by splitting by the plurality of target nodes, so as to achieve the purpose of parallel processing of the target job, thereby improving the execution efficiency of the target job.
When the execution unit 704 sends the target data corresponding to the target job to the target node through the job creation interface, if the target plug-in corresponding to the target job is determined, the execution unit 704 sends the target data corresponding to the target job to the target node through the job creation interface and the target plug-in, specifically, first sends the target data to the target plug-in through the job creation interface, and then the target plug-in sends the received target data to the target node.
The execution apparatus 700 of the job in the private cloud environment of the present embodiment may further include an operation unit 705 for executing the following: acquiring an operation request corresponding to a target job; the operation request is sent to the target node through the job management interface for the target node to process the target job according to the operation request.
That is, the operation unit 705 performs interaction with the target node through the job management interface, can achieve the purpose of operating the target job being executed in the target node, and can promote flexibility in executing the job in the private cloud environment.
The operation request acquired by the operation unit 705 is an operation request made by the user for the target job, and the operation corresponding to the operation request may be deletion of the target job, stop of the target job, update of the target job, or the like.
When the operation unit 705 sends an operation request to the target node through the job management interface, if there is a job plug-in corresponding to the target job, the operation unit 705 may first send the operation request to the target plug-in through the job management interface, and then send the received operation request to the target node by the target plug-in.
It can be understood that the job query interface, the node query interface, the job creation interface, the job management interface, and the like in this embodiment may be a unified interface, that is, the unified interface can implement functions of job query, node query, job creation, job management, and the like.
The execution device 700 of the job in the private cloud environment of the present embodiment may further include a synchronization unit 706, configured to execute the following: acquiring a job state sent by a target node through a job management interface; and sending the job state to the storage device through the job inquiry interface so as to be used for displaying the job state by the storage device.
That is, the synchronization unit 706 performs interaction with the target node through the job query interface and the job management interface, so that the job state of the target node when executing the target job can be synchronized, so that the user can more clearly know the execution state of the target job, further, the corresponding operation can be performed on the target job according to the execution state, and convenience in executing the target job in the private cloud environment is improved.
The job status received by the synchronization unit 706 may be whether the target node completes execution of the target job, whether the target node has a failure when executing the target job, or the like.
After the job status is sent to the storage device through the job query interface, the synchronization unit 706 may display the job status to the user, and the user may perform operations such as updating, deleting, stopping, and the like, on the target job according to the displayed job status.
When the synchronization unit 706 obtains the job status sent by the target node through the job management interface, if there is a job plug-in corresponding to the target job, the synchronization unit 706 may obtain the job status sent by the target node in the job plug-in through the job management interface.
In the technical scheme of the disclosure, the acquisition, storage, application and the like of the related user personal information all conform to the regulations of related laws and regulations, and the public sequence is not violated.
According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
As shown in fig. 8, a block diagram of an electronic device of a method of executing a job in a private cloud environment according to an embodiment of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 8, the apparatus 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM802, and the RAM803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to the bus 804.
Various components in device 800 are connected to I/O interface 805, including: an input unit 806 such as a keyboard, mouse, etc.; an output unit 807 such as various types of displays, speakers, and the like;
a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809 such as a network card, a modem 5, a wireless communication transceiver, or the like. The communication unit 809 allows the device 800 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
The computing unit 801 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 801 include, but are not limited to, a Central Processing Unit (CPU), a graph
Shape processing units (GPUs), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine 0-ware learning model algorithms, digital Signal Processors (DSPs), and any suitable computing unit
A processor, a controller, a microcontroller, etc. The computing unit 801 performs the respective methods and processes described above, for example, an execution method of a job in a private cloud environment. For example, in some embodiments, the method of execution of a job in a private cloud environment may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 808.
5 in some embodiments, part or all of the computer program may be via ROM802 and +.
Or communication unit 809 is loaded and/or installed onto device 800. When the computer program is loaded into
The RAM 803, when executed by the computing unit 801, may perform one or more steps of the execution method of the job in the private cloud environment described above. Alternatively, in other embodiments, computing unit 801
The execution method configured to execute the job in the private 0 cloud environment may be configured by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here can be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments 5 may include: implemented in one or more computer programs, the one or more computer programs
The method may be performed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
0 program code for carrying out the methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable apparatus to perform a task in a private cloud environment, such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a presentation device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for presenting information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service ("Virtual Private Server" or simply "VPS") are overcome. The server may also be a server of a distributed system or a server that incorporates a blockchain.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel, sequentially, or in a different order, provided that the desired results of the disclosed aspects are achieved, and are not limited herein.
The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.
Claims (27)
1. An execution method of a job in a private cloud environment, comprising:
acquiring a target job through a job query interface;
determining a target machine cluster in the candidate machine clusters in the private cloud environment according to the target job;
acquiring node information of nodes in the target machine cluster through a node query interface, and determining target nodes in the target machine cluster according to the node information and the target operation;
and sending target data corresponding to the target job to the target node through a job creation interface, so that the target node creates the target job and processes the target data.
2. The method of claim 1, wherein the determining a target machine cluster from among the candidate machine clusters in the private cloud environment according to the target job comprises:
acquiring the operation information of the target operation;
and taking the machine cluster matched with the operation information in the candidate machine cluster as the target machine cluster.
3. The method of claim 1, wherein the determining a target machine cluster from among the candidate machine clusters in the private cloud environment according to the target job comprises:
Performing authority verification on the private cloud environment;
and under the condition that the permission verification result is that the private cloud environment can execute the target job, executing the operation of determining a target machine cluster in the candidate machine clusters in the private cloud environment according to the target job.
4. The method of claim 1, wherein the obtaining node information of nodes in the target machine cluster through a node query interface, and determining a target node in the target machine cluster according to the node information and the target job comprises:
acquiring the operation information of the target operation;
taking the node corresponding to the node information matched with the job information as a candidate node;
the target node is determined from the candidate nodes.
5. The method of claim 4, wherein the node corresponding to the node information matching the job information as a candidate node comprises:
determining the resource request amount of the target job according to the job information;
determining the available resource quantity of the node according to the node information;
and taking the node with the available resource quantity larger than the resource request quantity as the candidate node.
6. The method of claim 5, wherein the node having the amount of available resources greater than the amount of resource requests as the candidate node comprises:
taking the node with the available resource amount larger than the resource request amount as an initial node;
determining identification information and/or labels of the initial nodes according to the node information of the initial nodes;
and taking the initial node corresponding to the identification information and/or the label matched with the job information as the candidate node.
7. The method of claim 1, wherein the sending, through a job creation interface, target data corresponding to the target job to the target node comprises:
splitting the target job into a plurality of sub-target jobs;
and transmitting sub-target data corresponding to the plurality of sub-target jobs to a plurality of target nodes through the job creation interface, so that the plurality of target nodes respectively create the sub-target jobs and process the sub-target data.
8. The method of claim 1, further comprising,
acquiring an operation request corresponding to the target job;
and sending the operation request to the target node through a job management interface, so that the target node can process the target job according to the operation request.
9. The method of claim 1, further comprising,
acquiring an operation state sent by the target node through an operation management interface;
and sending the job state to a storage device through the job inquiry interface so as to be used for displaying the job state by the storage device.
10. The method of any one of claims 1-9, further comprising,
after the target job is acquired through the job query interface, determining a target plug-in corresponding to the target job.
11. The method of claim 10, wherein the sending, through a job creation interface, target data corresponding to the target job to the target node comprises:
and sending target data corresponding to the target job to the target node through the job creation interface and the target plug-in.
12. The method of claim 10, wherein the sending the operation request to the target node through a job management interface comprises:
and sending the operation request to the target node through the job management interface and the target plug-in.
13. The method of claim 10, wherein the obtaining, through a job management interface, a job status sent by the target node comprises:
And acquiring the job state sent by the target node through the job management interface and the target plug-in.
14. An execution device of a job in a private cloud environment, comprising:
the acquisition unit is used for acquiring the target operation through the operation inquiry interface;
the first determining unit is used for determining a target machine cluster in the candidate machine clusters in the private cloud environment according to the target job;
the second determining unit is used for obtaining node information of nodes in the target machine cluster through a node query interface and determining target nodes in the target machine cluster according to the node information and the target job;
and the execution unit is used for sending the target data corresponding to the target job to the target node through a job creation interface so as to be used for creating the target job by the target node and processing the target data.
15. The apparatus of claim 14, wherein the first determining unit, when determining a target machine cluster among candidate machine clusters in a private cloud environment according to the target job, specifically performs:
acquiring the operation information of the target operation;
and taking the machine cluster matched with the operation information in the candidate machine cluster as the target machine cluster.
16. The apparatus of claim 14, wherein the first determining unit, when determining a target machine cluster among candidate machine clusters in a private cloud environment according to the target job, specifically performs:
performing authority verification on the private cloud environment;
and under the condition that the permission verification result is that the private cloud environment can execute the target job, executing the operation of determining a target machine cluster in the candidate machine clusters in the private cloud environment according to the target job.
17. The apparatus of claim 14, wherein the second determining unit, when obtaining node information of a node in the target machine cluster through a node query interface, determines a target node in the target machine cluster according to the node information and the target job, specifically performs:
acquiring the operation information of the target operation;
taking the node corresponding to the node information matched with the job information as a candidate node;
the target node is determined from the candidate nodes.
18. The apparatus of claim 17, wherein the second determination unit includes, as candidate nodes, nodes corresponding to node information to be matched with the job information:
Determining the resource request amount of the target job according to the job information;
determining the available resource quantity of the node according to the node information;
and taking the node with the available resource quantity larger than the resource request quantity as the candidate node.
19. The apparatus of claim 18, wherein the second determining unit, when taking a node with the available resource amount greater than the resource request amount as the candidate node, specifically performs:
taking the node with the available resource amount larger than the resource request amount as an initial node;
determining identification information and/or labels of the initial nodes according to the node information of the initial nodes;
and taking the initial node corresponding to the identification information and/or the label matched with the job information as the candidate node.
20. The apparatus of claim 14, wherein the execution unit, when sending target data corresponding to the target job to the target node through a job creation interface, specifically performs:
splitting the target job into a plurality of sub-target jobs;
and transmitting sub-target data corresponding to the plurality of sub-target jobs to a plurality of target nodes through the job creation interface, so that the plurality of target nodes respectively create the sub-target jobs and process the sub-target data.
21. The apparatus according to claim 14, further comprising an operation unit for performing:
acquiring an operation request corresponding to the target job;
and sending the operation request to the target node through a job management interface, so that the target node can process the target job according to the operation request.
22. The apparatus of claim 14, further comprising a synchronization unit to perform:
acquiring an operation state sent by the target node through an operation management interface;
and sending the job state to a storage device through the job inquiry interface so as to be used for displaying the job state by the storage device.
23. The apparatus according to any of claims 14-22, the acquisition unit further to perform:
after the target job is acquired through the job query interface, determining a target plug-in corresponding to the target job.
24. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein, the liquid crystal display device comprises a liquid crystal display device,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-13.
25. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-13.
26. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any of claims 1-13.
27. An execution system of a job in a private cloud environment, comprising the electronic device of claim 24, a machine cluster in the private cloud environment, and a storage device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211720428.4A CN116028178A (en) | 2022-12-30 | 2022-12-30 | Execution method and device of job in private cloud environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211720428.4A CN116028178A (en) | 2022-12-30 | 2022-12-30 | Execution method and device of job in private cloud environment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116028178A true CN116028178A (en) | 2023-04-28 |
Family
ID=86077585
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211720428.4A Pending CN116028178A (en) | 2022-12-30 | 2022-12-30 | Execution method and device of job in private cloud environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116028178A (en) |
-
2022
- 2022-12-30 CN CN202211720428.4A patent/CN116028178A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2023109138A1 (en) | Method and apparatus for starting android application in linux system, and electronic device | |
| CN109388626B (en) | Method and apparatus for assigning numbers to services | |
| US8886787B2 (en) | Notification for a set of sessions using a single call issued from a connection pool | |
| US11165661B2 (en) | Dynamically grouping monitored resources in a cloud environment to collections representing a composite application | |
| US10924590B1 (en) | Virtual workspace experience visualization and optimization | |
| CN112866391A (en) | Message pushing method and device, electronic equipment and storage medium | |
| CN113766487B (en) | Cloud mobile phone information acquisition method, device, equipment and medium | |
| JP2018517982A (en) | Automatic recharge system, method and server | |
| WO2023082716A1 (en) | Method and apparatus for operating android application in linux system, and device | |
| CN114077367A (en) | Method and device for realizing shortcut of android application in Linux system | |
| CN114090113B (en) | Method, device, equipment and storage medium for dynamically loading data source processing plug-in | |
| CN111767149B (en) | Scheduling method, device, equipment and storage equipment | |
| CN107045452B (en) | Virtual machine scheduling method and device | |
| CN114070889B (en) | Configuration method, traffic forwarding device, storage medium, and program product | |
| US11662927B2 (en) | Redirecting access requests between access engines of respective disk management devices | |
| CN116028178A (en) | Execution method and device of job in private cloud environment | |
| CN111741097B (en) | Method for tenant to monopolize node, computer equipment and storage medium | |
| CN116319331A (en) | Method, device, medium and equipment for issuing DNS (Domain name System) resolution configuration information in batches | |
| US11310117B2 (en) | Pairing of a probe entity with another entity in a cloud computing environment | |
| CN113360689B (en) | Image retrieval system, method, related device and computer program product | |
| CN116319633A (en) | Instant messaging method, device, equipment and storage medium | |
| CN114528082A (en) | Task scheduling method and device, electronic equipment and storage medium | |
| CN116582317A (en) | Control authority acquisition method and device, electronic equipment and storage medium | |
| CN116627469A (en) | Method, device, equipment and medium for generating unique identifier of Android equipment | |
| CN117081939A (en) | Traffic data processing method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |