CN117119078A - Privacy task scheduling method and device based on Kubernetes cluster interconnection scheduling framework - Google Patents

Privacy task scheduling method and device based on Kubernetes cluster interconnection scheduling framework Download PDF

Info

Publication number
CN117119078A
CN117119078A CN202311079940.XA CN202311079940A CN117119078A CN 117119078 A CN117119078 A CN 117119078A CN 202311079940 A CN202311079940 A CN 202311079940A CN 117119078 A CN117119078 A CN 117119078A
Authority
CN
China
Prior art keywords
job
target
task
communication module
resource
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311079940.XA
Other languages
Chinese (zh)
Inventor
郭石磊
汪婷
曹辰
胡晓龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alipay Hangzhou Information Technology Co Ltd
Original Assignee
Alipay Hangzhou Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alipay Hangzhou Information Technology Co Ltd filed Critical Alipay Hangzhou Information Technology Co Ltd
Priority to CN202311079940.XA priority Critical patent/CN117119078A/en
Publication of CN117119078A publication Critical patent/CN117119078A/en
Pending legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/18Multiprotocol handlers, e.g. single devices capable of handling multiple protocols
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/48Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806Task transfer initiation or dispatching
    • G06F9/4843Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/133Protocols for remote procedure calls [RPC]

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer And Data Communications (AREA)

Abstract

The embodiment of the specification provides a privacy task scheduling method based on a Kubernetes cluster interconnection scheduling framework, and relates to a first scheduling system and a second scheduling system. The first scheduling system includes a Job module and a first communication module. The second scheduling system includes a second communication module. The method comprises the following steps: the Job module acquires Job resources and selects a target protocol for the Job resources; the first communication module calls a creation job interface of the target protocol by utilizing a target plug-in corresponding to the target protocol so as to inform the second communication module to create the target job. The Job module initiates Job resources in response to Job creation success. The first communication module monitors that the Job resource is started, and invokes a starting Job interface of the target protocol to inform the second communication module of starting the target Job. The Job module creates a Task resource in response to the Job launch success. The first communication module monitors that the Task resource is created, and invokes a starting Task interface of the target protocol to inform the second communication module to start the target privacy Task.

Description

Privacy task scheduling method and device based on Kubernetes cluster interconnection scheduling framework
Technical Field
One or more embodiments of the present disclosure relate to the field of computer technologies, and in particular, to a method and apparatus for scheduling a privacy task based on a Kubernetes cluster interconnection scheduling framework.
Background
The privacy computation task (privacy task for short) is usually done cooperatively by multiple participants, however, the multiple participants may correspond to scheduling systems based on different scheduling frameworks. For example, part of the scheduling system performs scheduling processing based on an interconnection scheduling framework of the Kubernetes cluster. Aiming at the scene, a privacy task scheduling method based on a Kubernetes cluster interconnection scheduling framework needs to be provided.
Disclosure of Invention
One or more embodiments of the present disclosure describe a method for scheduling private tasks based on a Kubernetes cluster interconnection scheduling framework, which can quickly and accurately implement scheduling private tasks across scheduling systems.
According to a first aspect, a privacy task scheduling method based on a Kubernetes cluster interconnection scheduling framework is provided, and relates to a first scheduling system and a second scheduling system which are respectively corresponding to a first party and a second party; the first scheduling system performs scheduling processing based on a Kubernetes cluster interconnection scheduling framework and comprises a Job module and a first communication module, wherein a plurality of plug-ins corresponding to a plurality of communication protocols are integrated in the first communication module; the second scheduling system comprises a second communication module; the method comprises the following steps:
The Job module acquires Job resources and selects a target protocol for the Job resources; the Job resources include target privacy tasks;
the first communication module invokes a creation job interface of the target protocol by utilizing a target plugin corresponding to the target protocol in the plugins so as to inform the second communication module to create a target job, wherein the target job comprises the target privacy task;
the Job module responds to the successful creation of the target Job in the second scheduling system and starts the Job resource;
the first communication module monitors that the Job resource is started, and invokes a starting Job interface of the target protocol to inform the second communication module of starting the target Job;
the Job module responds to the successful starting of the target Job in the second scheduling system, and creates Task resources for describing the target privacy Task;
the first communication module monitors that the Task resource is created, and calls a starting Task interface of the target protocol to inform the second communication module to start the target privacy Task.
According to a second aspect, a privacy task scheduling method based on a Kubernetes cluster interconnection scheduling framework is provided, and the privacy task scheduling method relates to a first scheduling system and a second scheduling system which are respectively corresponding to a first party and a second party; the second scheduling system performs scheduling processing based on a Kubernetes cluster interconnection scheduling framework and comprises a second communication module, wherein a plurality of plugins corresponding to a plurality of communication protocols are integrated in the second communication module; the first scheduling system comprises a first communication module; the method is performed by the second communication module and includes:
Receiving a first call to a create job interface of a target protocol from the first communication module;
responding to the first call, creating Job resources by utilizing a target plugin corresponding to the target protocol in the plurality of plugins, and returning a first notification of successful Job creation to the first communication module; the Job resources include target privacy tasks;
in response to receiving a second call from the first communication module to a start Job interface of the target protocol, starting the Job resource and returning a second notification to the first communication module that the start Job was successful;
responsive to receiving a third call from the first communication module to a start Task interface of the target protocol, a Task resource describing the target privacy Task is created.
According to a third aspect, a privacy task scheduling device based on a Kubernetes cluster interconnection scheduling framework is provided, and the privacy task scheduling device is arranged in a first scheduling system, and the first scheduling system performs scheduling processing based on the Kubernetes cluster interconnection scheduling framework; the device comprises:
the acquisition unit is used for acquiring Job resources and selecting a target protocol for the Job resources; the Job resources include target privacy tasks;
The calling unit is used for calling a creation job interface of the target protocol by utilizing a target plugin corresponding to the target protocol in a plurality of plugins so as to inform a second scheduling system to create a target job, wherein the target job comprises the target privacy task;
the starting unit is used for responding to the successful creation of the target Job in the second scheduling system and starting the Job resource;
the calling unit is further used for calling a starting Job interface of the target protocol to inform the second scheduling system of starting the target Job when the Job resource is monitored to be started;
the creating unit is used for responding to the successful starting of the target job in the second scheduling system and creating Task resources for describing the target privacy Task;
and the calling unit is also used for monitoring that the Task resource is created and calling a starting Task interface of the target protocol so as to inform the second scheduling system to start the target privacy Task.
According to a fourth aspect, a privacy task scheduling device based on a Kubernetes cluster interconnection scheduling framework is provided, and the privacy task scheduling device is arranged in a second scheduling system, and the second scheduling system performs scheduling processing based on the Kubernetes cluster interconnection scheduling framework; the device comprises:
A receiving unit for receiving a first call for creating a job interface of a target protocol from a first scheduling system;
the creating unit is used for responding to the first call, creating Job resources by utilizing a target plug-in corresponding to the target protocol in a plurality of plug-ins, and returning a first notification of successful Job creation to the first scheduling system; the Job resources include target privacy tasks;
a starting unit, configured to start the Job resource in response to receiving a second call for a start Job interface of the target protocol from the first scheduling system, and return a second notification that the start Job is successful to the first scheduling system;
the creating unit is further configured to create a Task resource for describing the target privacy Task in response to receiving a third call to the start Task interface of the target protocol from the first scheduling system.
According to a fifth aspect, there is provided a computer readable storage medium having stored thereon a computer program which, when executed in a computer, causes the computer to perform the method of the first or second aspect.
According to a sixth aspect, there is provided a computing device comprising a memory having executable code stored therein and a processor which when executing the executable code implements the method of the first or second aspect.
In the private task scheduling method based on the Kubernetes cluster interconnection scheduling framework provided in one embodiment of the present disclosure, a plurality of plugins corresponding to a plurality of communication protocols are integrated in a first scheduling system in advance. When the first scheduling system sends a target request to the second scheduling system, a target plug-in corresponding to a target protocol commonly supported by two scheduling systems in the plurality of communication protocols is utilized to construct a request parameter of the target request, so that the second scheduling system can identify the target request, and the private task scheduling across the scheduling systems can be rapidly and accurately realized.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present description, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present description, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic illustration of an implementation scenario of an embodiment disclosed herein;
FIG. 2 illustrates a schematic diagram of a privacy task scheduling method based on a Kubernetes cluster interworking scheduling framework, according to one embodiment;
Fig. 3 shows a schematic diagram of a privacy task scheduling method based on a Kubernetes cluster interworking scheduling framework according to another embodiment;
FIG. 4 illustrates a schematic diagram of a privacy task scheduling device based on a Kubernetes cluster interworking scheduling framework, according to one embodiment;
fig. 5 shows a schematic diagram of a privacy task scheduling device based on a Kubernetes cluster interworking scheduling framework according to another embodiment.
Detailed Description
The following describes the scheme provided in the present specification with reference to the drawings.
The privacy task is generally calculated by a plurality of participants in a cooperative manner under the condition that the respective data does not leave the domain, and a final calculation result is obtained, for example, the privacy task can be a multiparty cooperative modeling task, each participant holds part of training data, and finally each participant obtains a secret sharing slice of model parameters.
However, as described above, the scheduling system of some of the multiple participants participating in the privacy task may perform scheduling processing based on the interconnection scheduling framework of the Kubernetes cluster, and for this purpose, the embodiment of the present disclosure provides a privacy task scheduling method based on the interconnection scheduling framework of the Kubernetes cluster, and integrates multiple plugins corresponding to multiple communication protocols in the first scheduling system in advance. When the first scheduling system sends a target request to the second scheduling system, a target plug-in corresponding to a target protocol commonly supported by two scheduling systems in the plurality of communication protocols is utilized to construct a request parameter of the target request, so that the second scheduling system can identify the target request, and the private task scheduling across the scheduling systems can be rapidly and accurately realized. The scheduling method is described in detail below.
Fig. 1 is a schematic diagram of an implementation scenario of an embodiment disclosed in the present specification. In fig. 1, an initiator a corresponds to a first scheduling system, and a participant B corresponds to a second scheduling system. The first scheduling system performs scheduling processing based on a Kubernetes cluster interconnection scheduling framework, and may include a Job module and a first communication module, where a plurality of plugins corresponding to a plurality of communication protocols are integrated in the first communication module. In addition, the first scheduling system may further include a Task module and the like. The second scheduling system may include a second communication module, and the scheduling framework of the second scheduling system may be different from the first scheduling system.
In fig. 1, the Job module may receive Job resources submitted by a user and select for them a target protocol that is commonly supported by initiator a and participant B. After selecting the target protocol, the first communication module may send a create job request, a start Task request, and the like to the second communication module by using a target plug-in corresponding to the target protocol, so as to start a target privacy Task in the second scheduling system, where the target privacy Task corresponds to a Task resource created locally by the first scheduling system. Thus, the method for scheduling the private tasks across the scheduling system is realized.
It should be understood that fig. 1 is only an exemplary illustration, and in practical application, in Kubernetes clusters, namespaces naspace a and naspace B of an initiator a and a participant B may also be created respectively, and several communication protocols supported by the initiator a may be recorded under the namespaces naspace a, and several communication protocols supported by the participant B may be recorded under the namespaces naspace B, which is not limited in this specification.
Finally, the number of the above-mentioned participants is not limited to two, and may be three or more.
Fig. 2 illustrates a schematic diagram of a privacy task scheduling method based on a Kubernetes cluster interworking scheduling framework according to one embodiment. The method relates to a first scheduling system and a second scheduling system which are respectively corresponding to a first party (also called an initiator) and a second party (also called a participant), wherein the first scheduling system performs scheduling processing based on a Kubernetes cluster interconnection scheduling framework and comprises a Job module and a first communication module. The first communication module has a plurality of plug-ins integrated therein corresponding to a plurality of communication protocols. In addition, the first communication system further comprises a Task module. The second scheduling system includes a second communication module.
As shown in fig. 2, the method may include the steps of:
In step S202, the Job module acquires Job resources and selects a target protocol for them.
The Job module here may be, for example, a Job module in the first scheduling system corresponding to the initiator a in fig. 1.
In one embodiment, the Job resources described above may be created by a task presenter (e.g., a user) using a command line tool (kubenetel) of a Kubernetes cluster.
In another embodiment, the Job resource may be created by a task presenter (e.g., an application program) by invoking an application programming interface (Application Programming Interface, API) of the Kubernetes cluster.
In a more specific embodiment, the task presenter creates the Job resource by creating a CR instance of a custom resource (Custom Resource Definitions, CRD). The CRD herein may be used to extend the resources of the Kubernetes cluster.
In one embodiment, the Job resource may include several privacy tasks and description information thereof, where a single privacy task is identified by a TaskID. The Job resource may further include fields such as a protocol tag (labes), a Job stage (stage), and a first resource status (phase). Wherein the value of the job phase may include any one of the following: a create phase, a start phase, and a stop phase. The value of the first resource state may include any one of the following: pending (pending) state, running (running) state, success (success) state, and failed (failed) state.
In the embodiment of the present disclosure, the Job module may apply in advance to the control plane (API Server) of the Kubernetes cluster for listening to changes of Job resources (including Job resource creation, modification, and deletion) and changes of Task resources (including Task resource creation, modification, and deletion).
In one particular embodiment, the listening to changes to Job resources or Task resources may be implemented based on a list-watch mechanism of Kubernetes.
Specifically, the Job module monitors that a Job resource is created in the Kubernetes cluster, that is, the Job resource is acquired, and reads a plurality of communication protocols supported by a first party from a Label of meta-information of a first Namespace corresponding to the first party and reads a plurality of communication protocols supported by a second party from a Label of meta-information of a second Namespace corresponding to the second party in the Kubernetes cluster. And selecting a communication protocol commonly supported by the first party and the second party from a plurality of communication protocols supported by the first party and the second party respectively as a target protocol.
In one example, the target protocol may be an interworking silver (Beijing FinTech Industry Alliance, BFIA) protocol.
After selecting the target protocol, the Job module may set the value of the protocol label in the Job resource to the target protocol.
Of course, in practical application, before the Job module selects the target protocol for the Job resource, the Job resource may be preprocessed, where the preprocessing may include at least one of the following: check the validity of Job resources, populate default fields, generate the TaskID of the privacy task, and so forth.
Further, after the Job module selects the target protocol for the Job resource, the value of the Job phase in the Job resource may be set to the creation phase. Thus, the creation of Job resources in the Kubernetes cluster of the first scheduling system is completed.
In step S204, the first communication module invokes the creation job interface of the target protocol to notify the second communication module to create the target job, including the target privacy task, using the target plugin corresponding to the target protocol in the plurality of plugins.
The first communication module may monitor, through a list-watch mechanism, changes (including creation, modification and deletion of Job resources) and changes (including creation, modification and deletion of Task resources) of Job resources in the Kubernetes cluster.
Specifically, the first communication module monitors that the Job phase of the Job resource is changed to the creation phase, and the first communication module can select a target plug-in from a plurality of plug-ins integrated in the first communication module according to the value of the protocol label (i.e. the target protocol), and analyze each field in the Job resource by using the target plug-in to construct a request parameter of the creation Job interface of the target protocol. The creation job interface may then be invoked with the target plug-in based on the constructed request parameters to effect sending a creation job request (or creation job notification) to the second communication module. The second communication module creates a target job in the second scheduling system in response to the create job request. The target Job here includes several privacy tasks stored in the Job resource described above, and a single privacy task is identified by a TaskID.
Here, if the above-described target Job is successfully created in the second scheduling system, the second communication module may return a notification of the success of creating the Job to the first communication module, so that the first communication module updates the Job resource, for example, records in the Job resource information that the creation of the Job in the second scheduling system is successful. And if the target Job fails to be created in the second scheduling system, the second communication module may return a notification of the failure to create the Job to the first communication module, so that the first communication module updates the Job resource, for example, records information of the failure to create the Job in the second scheduling system in the Job resource.
It should be understood that, information that the creation Job is successful or information that the creation Job is failed is recorded in the Job resource, the Job resource is changed, and the Job module may monitor the change.
In step S206, the Job module initiates Job resources in response to the target Job being successfully created at the second dispatch system.
Specifically, if the Job module monitors for a change to a Job resource, and the change is specifically information recorded in the Job resource that the Job was successfully created in the second scheduling system, the Job module may update the value of the Job phase of the Job resource to the start phase, thereby starting the Job resource. Alternatively, the Job module monitors for a change to the Job resource, and the change is specifically information recorded in the Job resource that the creation of the Job in the second scheduling system failed, then the Job module may update the value of the Job phase of the Job resource to a stop phase, thereby stopping the Job resource.
It should be appreciated that when the Job phase of the Job resource is updated to a start phase or a stop phase, the Job resource changes, and the first communication module may monitor the changes.
In step S208, the first communication module monitors that the Job resource is started, and invokes the start Job interface of the target protocol to notify the second communication module to start the target Job.
Further, if the first communication module monitors that the Job resource is stopped, a stop Job interface of the target protocol may be invoked to notify the second communication module to stop the target Job.
In this embodiment of the present disclosure, for each call of the interfaces (including creating a Job interface, starting a Task interface, querying an operation state interface, stopping a Job interface, etc.) of the target protocol, the corresponding target plug-in may be used to parse the Job resource and/or the Task resource.
Specifically, when the above-mentioned start Job interface is called, the Job resource may be parsed by using the target plug-in to construct a request parameter of the start Job interface. In one example, the request parameters here include at least JobID for Job's resource. The start job interface may then be invoked based on the request parameters to effect sending a start job request (or start job notification) to the second communication module. The second communication module initiates the target job in the second scheduling system in response to the initiate job request.
Then, if the above target Job is successfully started in the second scheduling system, the second communication module may return a notification of the success of the start Job to the first communication module, so that the first communication module updates the Job resource, for example, records, in the Job resource, information of the success of the start Job in the second scheduling system. And if the target Job fails to start in the second scheduling system, the second communication module may return a notification of the failure to start the Job to the first communication module, so that the first communication module updates the Job resource, for example, records information of the failure to start the Job in the second scheduling system in the Job resource.
After the information that the Job is started successfully or the information that the Job is started failed is recorded in the Job resource, the Job resource is changed, and the Job module can monitor the change.
In step S210, the Job module creates a Task resource describing the target privacy Task in response to the target Job being successfully started in the second scheduling system.
Specifically, the Job module monitors the change of the Job resource, and the change is specifically that information that the Job is successfully started in the second scheduling system is recorded in the Job resource, and then the Job module creates a Task resource for describing a target privacy Task for the target privacy Task included in the Job resource in the Kubernetes cluster. In addition, the Task resource may include a participant Task running state (partyTaskStatus, abbreviated as Task running state) and a second resource state (phase).
Alternatively, if the Job module monitors for a change to a Job resource, and the change is specifically information recorded in the Job resource that the Job failed to be started in the second scheduling system, the Job module may update the value of the Job phase of the Job resource to a stop phase, thereby stopping the Job resource.
It should be appreciated that when the Job phase of the Job resource is updated by the Job module to a stop phase, the Job resource has changed, and the first communication module may monitor the change. Specifically, the first communication module monitors that the Job resource is stopped, and may call a stop Job interface of the target protocol to notify the second communication module to stop the target Job.
It should also be appreciated that after the Job module creates the Task resource, the Task resource has changed, and the first communication module may monitor the change.
In step S212, the first communication module monitors that the Task resource is created, and invokes the Task starting interface of the target protocol to notify the second communication module to start the target privacy Task.
Specifically, when the above-mentioned starting Task interface is called, the Task resource may be parsed by using the target plug-in to construct a request parameter of the starting Task interface. In one example, the request parameters here include at least the taskID of the target privacy task. The launch task interface may then be invoked based on the request parameters to effect sending a launch task request (or launch task notification) to the second communication module. The second communication module initiates a target privacy task in the second scheduling system in response to the initiation task request.
Then, if the target privacy Task is successfully started in the second scheduling system, the second communication module may return a notification that the starting Task is successful to the first communication module, so that the first communication module updates the Task resource, for example, records information in the Task resource that the starting Task is successful in the second scheduling system. And if the target privacy Task fails to be started in the second scheduling system, the second communication module may return a notification of the failed start Task to the first communication module, so that the first communication module updates the Task resource, for example, records information of failed start job in the second scheduling system in the Task resource.
After the information of successful Task starting or the information of failed Task starting is recorded in the Task resource, the Task resource is changed, and the first communication module and the Task module can monitor the change. Here, the Task module applies in advance to the control plane of the Kubernetes cluster for monitoring the change of the Task resource.
Specifically, the first communication module responds to successful starting of the target privacy task in the second scheduling system, and periodically invokes a query running state interface of the target protocol to query the running state of the target privacy task. And then, the first communication module sets a value corresponding to the Task running state of the second party in the Task resource according to the running state of the target privacy Task. For example, when the running state of the target privacy Task in the second scheduling system is a failure state, the Task module updates the value of the Task running state corresponding to the second party in the Task resource to be the failure state; the Task module aggregates Task running states of all the participants and updates a second resource state (phase) in the Task resource to a failure state. Similarly, when the running state of the target privacy Task in the second scheduling system is a successful state, the Task module updates the value of the Task running state corresponding to the second party in the Task resource to be the successful state; the Task module aggregates the Task running states of the participants and updates the second resource state (phase) in the Task resource to a successful state.
Of course, in practical application, after the target privacy task is started successfully in the second scheduling system, the second communication module may periodically send the running state of the target privacy task to the first communication module, which is not limited in this specification.
It should be appreciated that after setting the second resource state in the Task resource to the failed state or the successful state, the Task resource is changed, and the Job module may monitor the change.
In one embodiment, the Job module listens to the change of the second resource state in the Task resource to the failed state, and will stop the Job resource. The first communication module monitors that the Job resource is stopped, and invokes a stop Job interface of the target protocol to inform the second communication module to stop the target Job.
In addition, the first communication module may also call a stop task interface of the target protocol to notify the second communication module to stop the target privacy task.
In view of the foregoing, in the method for scheduling private tasks based on the Kubernetes trunking interconnection scheduling framework provided in the embodiments of the present disclosure, when a scheduling system (i.e., a first scheduling system) corresponding to an initiator performs scheduling processing based on the Kubernetes trunking interconnection scheduling framework, multiple plug-ins corresponding to multiple communication protocols may be integrated in the first scheduling system in advance. When the first scheduling system sends a target request to a scheduling system (namely a second scheduling system) corresponding to a participant, a target plug-in corresponding to a target protocol commonly supported by two scheduling systems in the plurality of communication protocols is utilized to construct a request parameter of the target request, so that the second scheduling system can identify the target request, and the private task scheduling across the scheduling systems can be rapidly and accurately realized.
It should be understood that the above is an explanation of the method for scheduling a privacy task in the case where the scheduling system corresponding to the initiator performs scheduling processing based on the Kubernetes cluster interconnection scheduling framework. The following describes a scheduling method of a privacy task when a scheduling system corresponding to a participant performs scheduling processing based on a Kubernetes cluster interconnection scheduling framework.
Fig. 3 shows a schematic diagram of a privacy task scheduling method based on a Kubernetes cluster interworking scheduling framework according to another embodiment. The method relates to a first scheduling system and a second scheduling system which are respectively corresponding to a first party (also called an initiator) and a second party (also called a participant), wherein the second scheduling system performs scheduling processing based on a Kubernetes cluster interconnection scheduling framework and comprises a second communication module. The second communication module has a plurality of plug-ins integrated therein corresponding to a plurality of communication protocols. In addition, the second communication system further comprises a Job module and a Task module. The first scheduling system includes a first communication module.
As shown in fig. 3, the method may include the steps of:
in step S302, the second communication module receives a first call to create a job interface for a target protocol from the first communication module.
The target protocol may be a commonly supported communication protocol selected by the first communication module from a plurality of communication protocols supported by the first and second parties, respectively.
In one embodiment, the first communication module may first create a target job in the first scheduling system, the target job including a target privacy task. Thereafter, the first communication module may invoke the create job interface of the selected target protocol. It should be appreciated that the first call described above may include a first parameter (i.e., a request parameter to create a job interface).
In step S304, the second communication module responds to the first call, creates Job resources with a target plug-in corresponding to the target protocol in the plurality of plug-ins, and returns a first notification to the first communication module that the creation Job is successful.
Specifically, the second communication module parses the first parameter by using the target plug-in to determine a field value of each field in the Job resource. Here, various fields included in the Job resource may be referred to above, including at least the target privacy task. Job resources may then be created in the Kubernetes cluster based on the determined field values for the respective fields.
As described herein, if the target plugin is successful in resolving the first parameter and creating the Job resource is successful, the second communication module may return a notification to the first communication module that the creation Job was successful (i.e., the first notification). Conversely, if the target plugin fails to parse for the first parameter or fails to create the Job resource, the second communication module may return a notification of the creation Job failure to the first communication module, so that the first scheduling system stops task scheduling.
In step S306, the second communication module starts Job resources in response to receiving a second call to the start Job interface of the target protocol from the first communication module, and returns a second notification to the first communication module that the start Job is successful.
It should be appreciated that the second call herein may be sent by the first communication module in response to the first notification described above.
In the embodiment of the present specification, the second communication module may parse each interface call (including the first call and the second call described above, etc.) received from the first communication module by using the target plugin. Specifically, for the second call, after the target plugin is utilized to parse the second call, jobsid can be obtained, so that the second communication module can search and start Job resources based on the jobsid.
In one embodiment, the above-described starting the Job resource may include updating the Job resource's Job phase value to the start phase.
If the second communication module successfully starts Job resources, it may return a notification to the first communication module that the start Job was successful (i.e., the second notification). Conversely, if the second communication module fails to start the Job resource, it may return a notification of the failure to start the Job to the first communication module, so that the first scheduling system stops task scheduling.
In step S308, the second communication module creates a Task resource for describing the target privacy Task in response to receiving a third call to the start Task interface of the target protocol from the first communication module.
It should be appreciated that the third call herein may be sent by the first communication module in response to the second notification described above. Specifically, after the second communication module analyzes the third call by using the target plug-in, jobsid of the target Job and Name (Name) and TaskID of the target privacy task can be obtained, and the second communication module updates the TaskID to the target privacy task under the corresponding Job resource. After the Job module monitors the change of the Job resource, the Task resource corresponding to the target privacy Task corresponding to the Task ID is created. The fields included in the Task resource may be referred to above, and are not described herein.
Specifically, after the second communication module creates the Task resource successfully, a notification of success of the start Task (hereinafter referred to as third notification) may be returned to the first communication module. And after the second communication module fails to create the Task resource, a notification of a failure to start the Task may be returned to the first communication module, so that the first scheduling system stops Task scheduling.
It should be appreciated that after the second communication module successfully creates the Task resource, it may also receive a call from the first communication module to query the running state interface for the target protocol, and in response to the call, the second communication module may return the running state of the target privacy Task to the first communication module.
Or after the second communication module successfully creates the Task resource, the second communication module may periodically send the running state of the target privacy Task to the first communication module.
Of course, in practical application, the second communication module may also receive a fourth call to the stop job interface of the target protocol from the first communication module. The second communication module stops the Job resource in response to the fourth call. I.e., the value of the Job phase in the Job resource is updated to the stop phase.
Wherein if the second communication module succeeds in updating the Job resource, a notification of success of stopping the Job is returned to the first communication module. Here, updating the Job resource means updating the value of the Job phase in the Job resource to the stop phase. Otherwise, if the second communication module fails to update the Job resource, a notification of a stop Job failure is returned to the first communication module.
In addition, the second communication module may also call a push job status interface of the target protocol to notify the first communication module to query the job status of the target job. The value of the first resource state in the Job resource is set based on the received Job state.
Finally, the second communication module may also receive a fifth call to the stop task interface of the target protocol from the first communication module. The second communication module stops the Task resource in response to the fifth call.
And if the second communication module succeeds in stopping the Task resource, returning a notification of successful Task stopping to the first communication module. Otherwise, if the second communication module fails to stop the Task resource, a notification of the failure of stopping the Task is returned to the first communication module.
In view of the foregoing, in the method for scheduling private tasks based on the Kubernetes trunking interconnection scheduling framework provided in the embodiments of the present disclosure, when a scheduling system (i.e., a second scheduling system) corresponding to a participant performs scheduling processing based on the Kubernetes trunking interconnection scheduling framework, multiple plug-ins corresponding to multiple communication protocols may be integrated in the second scheduling system in advance. And then, when the second scheduling system receives the target request from the scheduling system (namely the first scheduling system) corresponding to the initiator, the target plug-in corresponding to the target protocol commonly supported by the two scheduling systems in the plurality of communication protocols is utilized to analyze and identify the request parameters of the target request, so that the private task scheduling of the cross-scheduling system can be rapidly and accurately realized.
In summary, the scheme has the following advantages:
1) The communication module can integrate a plurality of plugins corresponding to different communication protocols, and the scheduling of privacy tasks among scheduling systems based on different scheduling frames can be rapidly and conveniently realized through the plugins.
2) The scheduling system based on the Kubernetes cluster scheduling framework is communicated with an external scheduling system through a form of a job or task API, so that processing details of internal complex jobs and tasks are shielded.
3) The Job module and the Task module are used for processing Job resources and Task resources in the Kubernetes cluster, and can be used for completing scheduling of private tasks across a scheduling system in cooperation with the communication module.
Corresponding to the above private task scheduling method based on the Kubernetes cluster interconnection scheduling framework, the private task scheduling device based on the Kubernetes cluster interconnection scheduling framework provided in an embodiment of the present disclosure is set in a first scheduling system, where the first scheduling system performs scheduling processing based on the Kubernetes cluster interconnection scheduling framework. As shown in fig. 4, the apparatus may include:
an obtaining unit 402, configured to obtain a Job resource, and select a target protocol for the Job resource, where the Job resource includes a target privacy task.
And a calling unit 404, configured to call, by using a target plug-in corresponding to the target protocol in the plurality of plug-ins, a creation job interface of the target protocol to notify the second scheduling system to create the target job, where the target job includes a target privacy task.
And a starting unit 406, configured to start the Job resource in response to the creation success of the target Job in the second scheduling system.
The calling unit 404 is further configured to monitor that the Job resource is started, and call a start Job interface of the target protocol to notify the second scheduling system to start the target Job.
A creating unit 408, configured to create a Task resource for describing the target privacy Task in response to the target job being started successfully in the second scheduling system.
The calling unit 404 is further configured to monitor that the Task resource is created, and call a startup Task interface of the target protocol to notify the second scheduling system to start the target privacy Task.
In one embodiment, the second scheduling system has a different scheduling framework from the first scheduling system
In one embodiment, the obtaining unit 402 is specifically configured to:
reading a plurality of communication protocols supported by the first and second parties respectively from the labels of the meta information of the first and second namespaces corresponding to the first and second parties respectively in the Kubernetes cluster;
And selecting a communication protocol commonly supported by the first and second parties from a plurality of communication protocols supported by the first and second parties respectively as a target protocol.
In one embodiment, the apparatus further comprises:
a stopping unit 410, configured to stop the Job resource in response to a failure of the target Job to be created in the second scheduling system;
the calling unit 404 is further configured to monitor that the Job resource is stopped, and call a stop Job interface of the target protocol to notify the second scheduling system to stop the target Job.
In one embodiment, stop unit 410 is further configured to stop the Job resource in response to a failure of the target Job to start at the second dispatch system;
the calling unit 404 is further configured to monitor that the Job resource is stopped, and call a stop Job interface of the target protocol to notify the second scheduling system to stop the target Job.
In one embodiment, the Task resource includes, a Task running state; the apparatus further includes a first setting unit 412;
the calling unit 404 is further configured to, in response to successful start-up of the target privacy task in the second scheduling system, call a query running state interface of the target protocol to query a running state of the target privacy task;
a first setting unit 412, configured to set a value corresponding to the Task running state of the second party in the Task resource according to the running state of the target privacy Task.
In one embodiment, the first scheduling system further comprises a Task module; the Job resource also includes a first resource state, and the Task resource also includes a second resource state; the apparatus further comprises: a second setting unit 414;
a second setting unit 414, configured to monitor that the second resource status is changed to a failure status, set the first resource status to the failure status, and stop the Job resource; wherein the second resource state is determined by the Task module by aggregating Task running states of the participants;
the calling unit 404 is further configured to monitor that the Job resource is stopped, and call a stop Job interface of the target protocol to notify the second scheduling system to stop the target Job.
In one embodiment, the calling unit 404 is further configured to call a stop task interface of the target protocol to notify the second scheduling system to stop the target privacy task.
The functions of the functional units of the apparatus in the foregoing embodiments of the present disclosure may be implemented by the steps of the foregoing method embodiments, so that the specific working process of the apparatus provided in one embodiment of the present disclosure is not repeated herein.
The privacy task scheduling device based on the Kubernetes cluster interconnection scheduling framework provided by the embodiment of the specification can rapidly and accurately realize the privacy task scheduling of a cross-scheduling system.
Corresponding to the above private task scheduling method based on the Kubernetes cluster interconnection scheduling framework, an embodiment of the present disclosure further provides a private task scheduling device based on the Kubernetes cluster interconnection scheduling framework, which is configured in a second scheduling system, where the second scheduling system performs scheduling processing based on the Kubernetes cluster interconnection scheduling framework. As shown in fig. 5, the apparatus may include:
a receiving unit 502 is configured to receive a first call for creating a job interface of a target protocol from a first scheduling system.
A creating unit 504, configured to create, in response to the first call, a Job resource with a target plugin corresponding to the target protocol in the plurality of plugins, and return a first notification that the creation Job is successful to the first scheduling system, where the Job resource includes the target privacy task.
A starting unit 506, configured to, in response to receiving a second call to the target protocol to start the Job interface from the first scheduling system, start the Job resource, and return a second notification to the first scheduling system that the start Job is successful.
The creating unit 504 is further configured to create a Task resource for describing the target privacy Task in response to receiving a third call to the start Task interface of the target protocol from the first scheduling system.
In one embodiment, the apparatus further comprises:
a stopping unit 508 for stopping the Job resource in response to receiving a fourth call from the first dispatch system to stop the Job interface for the target protocol.
In one embodiment, the apparatus further comprises:
and the sending unit 510 is configured to send the running state of the target privacy task to the first scheduling system periodically.
In one embodiment, the first call is sent by the first scheduling system after the creation of the target job; job resources also include a first resource state; the apparatus further comprises:
a calling unit 512, configured to call a push job status interface of the target protocol to notify the first scheduling system to query the job status of the target job;
a setting unit 514 is configured to set a value of the first resource status according to the received job status.
The functions of the functional units of the apparatus in the foregoing embodiments of the present disclosure may be implemented by the steps of the foregoing method embodiments, so that the specific working process of the apparatus provided in one embodiment of the present disclosure is not repeated herein.
The privacy task scheduling device based on the Kubernetes cluster interconnection scheduling framework provided by the embodiment of the specification can rapidly and accurately realize the privacy task scheduling of a cross-scheduling system.
According to an embodiment of another aspect, there is also provided a computer-readable storage medium having stored thereon a computer program which, when executed in a computer, causes the computer to perform the method described in connection with fig. 2 or 3.
According to an embodiment of yet another aspect, there is also provided a computing device including a memory having executable code stored therein and a processor that, when executing the executable code, implements the method described in connection with fig. 2 or 3.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for medium or device embodiments, the description is relatively simple as it is substantially similar to method embodiments, with reference to the description of method embodiments in part.
The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
The foregoing detailed description of the embodiments has further described the objects, technical solutions and advantages of the present specification, and it should be understood that the foregoing description is only a detailed description of the embodiments of the present specification, and is not intended to limit the scope of the present specification, but any modifications, equivalents, improvements, etc. made on the basis of the technical solutions of the present specification should be included in the scope of the present specification.

Claims (16)

1. A privacy task scheduling method based on a Kubernetes cluster interconnection scheduling framework relates to a first scheduling system and a second scheduling system which are respectively corresponding to a first party and a second party; the first scheduling system performs scheduling processing based on a Kubernetes cluster interconnection scheduling framework and comprises a Job module and a first communication module, wherein a plurality of plug-ins corresponding to a plurality of communication protocols are integrated in the first communication module; the second scheduling system comprises a second communication module; the method comprises the following steps:
the Job module acquires Job resources and selects a target protocol for the Job resources; the Job resources include target privacy tasks;
the first communication module invokes a creation job interface of the target protocol by utilizing a target plugin corresponding to the target protocol in the plugins so as to inform the second communication module to create a target job, wherein the target job comprises the target privacy task;
The Job module responds to the successful creation of the target Job in the second scheduling system and starts the Job resource;
the first communication module monitors that the Job resource is started, and invokes a starting Job interface of the target protocol to inform the second communication module of starting the target Job;
the Job module responds to the successful starting of the target Job in the second scheduling system, and creates Task resources for describing the target privacy Task;
the first communication module monitors that the Task resource is created, and calls a starting Task interface of the target protocol to inform the second communication module to start the target privacy Task.
2. The method of claim 1, wherein a scheduling framework of the second scheduling system is different from the first scheduling system.
3. The method of claim 1, wherein the selecting the target protocol for it comprises:
reading a plurality of communication protocols supported by each of the first and second parties from the labels of the meta information of each of the first and second namespaces corresponding to the first and second parties in the Kubernetes cluster;
and selecting a communication protocol commonly supported by the first and second parties from a plurality of communication protocols supported by the first and second parties respectively as the target protocol.
4. The method of claim 1, further comprising:
the Job module stopping the Job resource in response to the failure of the target Job to be created in the second scheduling system;
the first communication module monitors that the Job resource is stopped, and invokes a stop Job interface of the target protocol to inform the second communication module to stop the target Job.
5. The method of claim 1, further comprising:
the Job module responds to the failure of the target Job to start in the second scheduling system and stops the Job resource;
the first communication module monitors that the Job resource is stopped, and invokes a stop Job interface of the target protocol to inform the second communication module to stop the target Job.
6. The method of claim 1, wherein the Task resource comprises a Task running state; the method further comprises the steps of:
the first communication module responds to the successful starting of the target privacy task in the second scheduling system, and invokes a query running state interface of the target protocol to query the running state of the target privacy task;
and the first communication module sets a value corresponding to the Task running state of the second party in the Task resource according to the running state of the target privacy Task.
7. The method of claim 6, wherein the first scheduling system further comprises a Task module; the Job resource further comprises a first resource state, and the Task resource further comprises a second resource state; the method further comprises the steps of:
the Job module monitors that the second resource state is changed into a failure state, sets the first resource state into the failure state, and stops the Job resource; wherein the second resource state is determined by the Task module by aggregating Task running states of the participants;
the first communication module monitors that the Job resource is stopped, and invokes a stop Job interface of the target protocol to inform the second communication module to stop the target Job.
8. The method of claim 1, further comprising:
the first communication module invokes a stop task interface of the target protocol to inform the second communication module to stop the target privacy task.
9. A privacy task scheduling method based on a Kubernetes cluster interconnection scheduling framework relates to a first scheduling system and a second scheduling system which are respectively corresponding to a first party and a second party; the second scheduling system performs scheduling processing based on a Kubernetes cluster interconnection scheduling framework and comprises a second communication module, wherein a plurality of plugins corresponding to a plurality of communication protocols are integrated in the second communication module; the first scheduling system comprises a first communication module; the method is performed by the second communication module and includes:
Receiving a first call to a create job interface of a target protocol from the first communication module;
responding to the first call, creating Job resources by utilizing a target plugin corresponding to the target protocol in the plurality of plugins, and returning a first notification of successful Job creation to the first communication module; the Job resources include target privacy tasks;
in response to receiving a second call from the first communication module to a start Job interface of the target protocol, starting the Job resource and returning a second notification to the first communication module that the start Job was successful;
responsive to receiving a third call from the first communication module to a start Task interface of the target protocol, a Task resource describing the target privacy Task is created.
10. The method of claim 9, further comprising:
responsive to receiving a fourth call from the first communication module to stop the Job interface for the target protocol, stopping the Job resource.
11. The method of claim 9, further comprising:
and periodically sending the running state of the target privacy task to the first communication module.
12. The method of claim 9, wherein the first call is sent by the first communication module after creation of a target job; the Job resource further comprises a first resource state; the method further comprises the steps of:
Invoking a push job status interface of the target protocol to inform the first communication module to query the job status of the target job;
and setting the value of the first resource state according to the received job state.
13. The privacy task scheduling device based on the Kubernetes cluster interconnection scheduling framework is arranged in a first scheduling system, and the first scheduling system performs scheduling processing based on the Kubernetes cluster interconnection scheduling framework; the device comprises:
the acquisition unit is used for acquiring Job resources and selecting a target protocol for the Job resources; the Job resources include target privacy tasks;
the calling unit is used for calling a creation job interface of the target protocol by utilizing a target plugin corresponding to the target protocol in a plurality of plugins so as to inform a second scheduling system to create a target job, wherein the target job comprises the target privacy task;
the starting unit is used for responding to the successful creation of the target Job in the second scheduling system and starting the Job resource;
the calling unit is further used for calling a starting Job interface of the target protocol to inform the second scheduling system of starting the target Job when the Job resource is monitored to be started;
The creating unit is used for responding to the successful starting of the target job in the second scheduling system and creating Task resources for describing the target privacy Task;
and the calling unit is also used for monitoring that the Task resource is created and calling a starting Task interface of the target protocol so as to inform the second scheduling system to start the target privacy Task.
14. The privacy task scheduling device based on the Kubernetes cluster interconnection scheduling framework is arranged in a second scheduling system, and the second scheduling system performs scheduling processing based on the Kubernetes cluster interconnection scheduling framework; the device comprises:
a receiving unit for receiving a first call for creating a job interface of a target protocol from a first scheduling system;
the creating unit is used for responding to the first call, creating Job resources by utilizing a target plug-in corresponding to the target protocol in a plurality of plug-ins, and returning a first notification of successful Job creation to the first scheduling system; the Job resources include target privacy tasks;
a starting unit, configured to start the Job resource in response to receiving a second call for a start Job interface of the target protocol from the first scheduling system, and return a second notification that the start Job is successful to the first scheduling system;
The creating unit is further configured to create a Task resource for describing the target privacy Task in response to receiving a third call to the start Task interface of the target protocol from the first scheduling system.
15. A computer readable storage medium having stored thereon a computer program, wherein the computer program, when executed in a computer, causes the computer to perform the method of any of claims 1-12.
16. A computing device comprising a memory and a processor, wherein the memory has executable code stored therein, which when executed by the processor, implements the method of any of claims 1-12.
CN202311079940.XA 2023-08-24 2023-08-24 Privacy task scheduling method and device based on Kubernetes cluster interconnection scheduling framework Pending CN117119078A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311079940.XA CN117119078A (en) 2023-08-24 2023-08-24 Privacy task scheduling method and device based on Kubernetes cluster interconnection scheduling framework

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311079940.XA CN117119078A (en) 2023-08-24 2023-08-24 Privacy task scheduling method and device based on Kubernetes cluster interconnection scheduling framework

Publications (1)

Publication Number Publication Date
CN117119078A true CN117119078A (en) 2023-11-24

Family

ID=88797844

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311079940.XA Pending CN117119078A (en) 2023-08-24 2023-08-24 Privacy task scheduling method and device based on Kubernetes cluster interconnection scheduling framework

Country Status (1)

Country Link
CN (1) CN117119078A (en)

Similar Documents

Publication Publication Date Title
CN111176858A (en) Data request processing method and device
CN107241315B (en) Access method and device of bank gateway interface and computer readable storage medium
CN111694797B (en) File uploading and analyzing method, device, server and medium
CN113691635B (en) Method and device for calling microservice, electronic equipment and readable storage medium
CN111381940B (en) Distributed data processing method and device
CN103107921A (en) Monitoring method and system
CN114237994A (en) Test method and system for distributed system, electronic device and storage medium
CN111858083A (en) Remote service calling method and device, electronic equipment and storage medium
CN107463390B (en) Software upgrading method and upgrading server
CN111580948A (en) Task scheduling method and device and computer equipment
CN113342503B (en) Real-time progress feedback method, device, equipment and storage medium
CN110661851A (en) Data exchange method and device
CN113010306A (en) Service data processing method and device, computer equipment and storage medium
CN111427634A (en) Atomic service scheduling method and device
CN117119078A (en) Privacy task scheduling method and device based on Kubernetes cluster interconnection scheduling framework
CN114003384B (en) Task management method, device and equipment
CN115361382A (en) Data processing method, device, equipment and storage medium based on data group
CN113010377B (en) Method and device for collecting operation log of job
CN114020368A (en) Information processing method and device based on state machine and storage medium
CN114666411A (en) Request processing method, device, server, storage medium and product
CN110768855B (en) Method and device for testing linkmzation performance
CN108170536B (en) Expanding method and device of WEB program
CN110958240A (en) Message subscription system and method
CN104298750B (en) Renewal processing method and processing device for real-time system communication
CN112598323B (en) Cross-business-domain process monitoring method and device, computer equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination