CN112783653A - Resource scheduling method and device based on containerization - Google Patents
Resource scheduling method and device based on containerization Download PDFInfo
- Publication number
- CN112783653A CN112783653A CN202110095621.2A CN202110095621A CN112783653A CN 112783653 A CN112783653 A CN 112783653A CN 202110095621 A CN202110095621 A CN 202110095621A CN 112783653 A CN112783653 A CN 112783653A
- Authority
- CN
- China
- Prior art keywords
- task
- container
- resources
- pipeline
- pipeline template
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/508—Monitor
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a resource scheduling method and device based on containerization, and belongs to the technical field of cloud computing. The resource scheduling method based on containerization comprises the following steps: receiving a pipeline template issuing request, and creating a pipeline template according to the pipeline template issuing request; sequentially converting each task into a corresponding container according to a task scheduling sequence in the pipeline template; mounting resources of corresponding tasks in a container; and deploying the container with the resources mounted on a predetermined target node to realize resource scheduling. The invention can evenly schedule resources and ensure the full utilization of the resources.
Description
Technical Field
The invention relates to the technical field of cloud computing, in particular to a resource scheduling method and device based on containerization.
Background
Software Development goes through different Development stages in sequence from waterfall type method to agile delivery and then to Development (which is a combined word of Development and Operations and a general term of process, method and system), and the software Development never stops advancing. As enterprises step toward the DevOps approach, the ability to promote continuous integration and continuous delivery (CI/CD) will help enterprise products develop quickly, come online quickly, try and error quickly, and recover quickly.
The traditional construction and distribution system at present has the following two problems: the first is that when the CI/CD system does not build a task, the physical machine or virtual machine of the CI/CD system is in an idle state, and the resources are not completely released, so that a large amount of machines are idle, and the resources are wasted. The second is that when there are a large number of building tasks, the task resource scheduling is unbalanced, so that tasks on some building nodes are queued to be executed, and some building nodes are idle or only a small number of tasks run on them.
In summary, there is a need for a scheme capable of dynamically and reasonably utilizing resources and load balancing task scheduling. The problem that the resource is unreasonably used in the use process of the CI/CD pipeline is solved.
Disclosure of Invention
The embodiments of the present invention mainly aim to provide a resource scheduling method and apparatus based on containerization, so as to balance scheduling resources and ensure full utilization of resources.
In order to achieve the above object, an embodiment of the present invention provides a resource scheduling method based on containerization, including:
receiving a pipeline template issuing request, and creating a pipeline template according to the pipeline template issuing request;
sequentially converting each task into a corresponding container according to a task scheduling sequence in the pipeline template;
mounting resources of corresponding tasks in a container;
and deploying the container with the resources mounted on a predetermined target node to realize resource scheduling.
An embodiment of the present invention further provides a resource scheduling apparatus based on containerization, including:
the pipeline template module is used for receiving a pipeline template issuing request and creating a pipeline template according to the pipeline template issuing request;
the container generation module is used for sequentially converting each task into a corresponding container according to the task scheduling sequence in the pipeline template;
the resource mounting module is used for mounting resources of the corresponding tasks in the container;
and the container deployment module is used for deploying the container with the resources mounted on a predetermined target node so as to realize resource scheduling.
The embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements the steps of the resource scheduling method based on containerization when executing the computer program.
The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the method for scheduling resources based on containerization.
The resource scheduling method and device based on containerization firstly create the pipeline template according to the pipeline template issuing request, then sequentially convert each task into the corresponding container according to the task scheduling sequence in the pipeline template, then mount the resources of the corresponding tasks in the containers, and deploy the containers with the resources mounted on the target nodes, so that the resources can be scheduled in a balanced manner, and the full utilization of the resources is ensured.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without creative efforts.
FIG. 1 is a flow chart of a method for resource scheduling based on containerization in an embodiment of the present invention;
FIG. 2 is a flow chart of a method for resource scheduling based on containerization in another embodiment of the present invention;
FIG. 3 is a flow diagram of creating a pipeline template in an embodiment of the invention;
FIG. 4 is a block diagram of a resource scheduling apparatus based on containerization according to an embodiment of the present invention;
FIG. 5 is a block diagram of an apparatus for customizing resource templates in another embodiment of the present invention;
FIG. 6 is a block diagram of a custom resource controller in another embodiment of the present invention;
FIG. 7 is a block diagram of a monitoring device according to another embodiment of the present invention;
FIG. 8 is a block diagram showing the structure of a log device according to another embodiment of the present invention;
fig. 9 is a block diagram of a computer device in the embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As will be appreciated by one skilled in the art, embodiments of the present invention may be embodied as a system, apparatus, device, method, or computer program product. Accordingly, the present disclosure may be embodied in the form of: entirely hardware, entirely software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software.
In view of the unreasonable resource usage and unbalanced resource allocation in the prior art, embodiments of the present invention provide a resource scheduling method based on containerization, which can schedule resources in a balanced manner and ensure full utilization of resources. The present invention will be described in detail below with reference to the accompanying drawings.
In the field of cloud computing, the container provides independent operating system, operating environment, file storage and other capabilities for a user, and the technical implementation of the Kubernetes container arrangement scheme based on the Google open source guarantees load balance, life cycle management, elastic expansion and the like of the container during use. Therefore, the invention provides a CI/CD (continuous integration and continuous delivery) system based on Kubernetes protogenesis based on the characteristics of Kubernetes and containerization technology. CRD resources of various CI/CDs, such as Task, Pipeline resource, Pipeline and Pipeline run, are designed through Kubernets self-defining resource (CRD). The method comprises the steps of compiling a pipeline template, arranging and combining tasks and resources to form a required pipeline, generating a corresponding container (pod) for the tasks of the pipeline to run in a Kubernetes cluster when the pipeline runs, collecting a running log and deleting the container when the tasks run, and achieving the technical scheme that the running resources are generated immediately after the running resources are used and the tasks are destroyed after the running of the tasks is finished. Meanwhile, based on a load balancing strategy of Kubernetes, a container for running tasks can be reasonably scheduled to each idle node, and the full utilization of resources of each virtual machine is guaranteed. The invention combines Kubernetes protogenesis and CI/CD technology, creates a new pipeline operation scheme, and realizes the concept of cloud protogenesis.
FIG. 1 is a flowchart of a method for resource scheduling based on containerization according to an embodiment of the present invention. FIG. 2 is a flowchart of a method for scheduling resources based on containerization according to another embodiment of the present invention. As shown in fig. 1-2, the method for scheduling resources based on containerization includes:
s101: and receiving a pipeline template issuing request, and creating a pipeline template according to the pipeline template issuing request.
FIG. 3 is a flow diagram of creating a pipeline template in an embodiment of the invention. As shown in FIG. 3, creating a pipeline template includes:
s201: and issuing a request according to the pipeline template to obtain a task scheduling sequence.
Before executing S201, the method further includes: and installing the custom resource controller. The custom resource controller is pre-installed in a Kubernets cluster and is mainly used for analyzing and managing the custom resource. The user-defined resource controller is a core device for containerization operation of tasks, needs to be operated in a Kubernetes cluster all the time, can verify and analyze CRD template information of a production line when receiving user-defined resource template information (CRD template information of a CI/CD production line, a production line template), can analyze the production line template (production line) when the template format and the content pass the verification, and sequentially starts corresponding containers to complete the execution of the tasks according to the task sequence arranged in the production line template.
The custom resource Controller comprises a custom resource checking unit (Pipeline-Webhook) and a custom resource control unit (Pipeline-Controller). The user-defined resource control unit is used for monitoring the analysis, creation, deletion, modification and query events of the CRD resource.
S202: and acquiring each task and the resource of each task according to the task scheduling sequence.
Before executing S202, the method further includes: and defining tasks, resources corresponding to the tasks and task scheduling sequences.
When the method is concretely implemented, the CRD template information of the CI/CD assembly line can be stored by the self-defined resource template device, and the composition, the function and the input and output information (resources) of the assembly line template are defined. Wherein the CRD resource types of the template include: task, Pipeline resource, Pipeline, and Pipeline run.
Tasks to be executed (such as compiling code, building images, pushing images and the like) are defined through task units, the tasks are minimum configurable units in a pipeline, and each task actually runs in a kubernets cluster in a container mode. If the task is successfully executed, the corresponding container status will become Completed, otherwise Failed.
The input and output information of the pipeline resources is defined through the pipeline resource management unit, and the pipeline can be used by introducing a required resource name.
S203: and creating a pipeline template according to the task scheduling sequence, each task and the resource of each task.
A pipeline template (pipeline) is a combination of one or more tasks, resources, and custom parameters. The names of tasks and resources to be scheduled are defined in the Pipeline template, and after a user-defined resource control unit (Pipeline-Controller) analyzes the template, corresponding tasks are created according to the task names and the resource names.
During specific implementation, a user integrates Task, Pipeline resource, Pipeline and Pipeline running by compiling a Pipeline template, the Pipeline template is issued to a Kubernets cluster, a user-defined resource controller in the Kubernets cluster receives an issuing request of the Pipeline template, and Pipeline running operation is started by a CRD resource of the Pipeline running until the operation of the last Task container of the Pipeline is finished, so that a Pipeline operation Task is completed.
S102: and sequentially converting each task into a corresponding container according to the task scheduling sequence in the pipeline template.
Before executing S102, the method further includes: and verifying the correctness of the streamline template format by the self-defined resource verification unit. And when the format of the pipeline template is correct, executing S102-S104, triggering the specified pipeline to run through PipelineRun, analyzing the task scheduling sequence set in the pipeline template through the self-defined resource control unit, sequentially converting each task into a corresponding container, and simultaneously mounting the resources into the container.
S103: and mounting the resources of the corresponding tasks in the container.
S104: and deploying the container with the resources mounted on a predetermined target node to realize resource scheduling.
In an embodiment, before performing S104, the method further includes: and determining a target node according to the load balancing strategy.
In specific implementation, a scheduler (kube-scheduler) component of the kubernets cluster monitors resource operation conditions on each virtual machine node (node), and deploys a task container to a selected target node to operate through a preselected and preferred load balancing strategy, so that the node resources are fully used.
In one embodiment, after executing S104, the method further includes:
1. and running the container on the target node to obtain a task execution result.
2. When a user initiates a request for issuing and deploying, the task execution result is a mirror image, and the mirror image is uploaded to a mirror image warehouse for subsequent issuing work.
3. And receiving an application starting instruction, pulling a corresponding mirror image from the mirror image warehouse according to the application starting instruction, and running to start the application program.
For example, a user initiates a release deployment request, issues a released pipeline template to a Kubernetes cluster, and a custom resource controller in the cluster analyzes the format of the released pipeline template and deploys the template to a corresponding cluster environment according to a load balancing strategy. In the deployment process, according to the practical situation of the application, the corresponding mirror image is pulled from the mirror image warehouse to start the application program.
In one embodiment, the method further comprises: collecting log information in a task running process in real time; and storing the collected log information and displaying the log information to a user.
In one embodiment, the method further comprises: monitoring the running state of the pipeline in real time (such as successful running or failed running of the task) and monitoring the log information of the running of the task. And deleting the container after the log collection is finished and the task operation is finished, releasing the constructed resources to be used by other tasks, and further achieving the reasonable utilization of the resources.
The execution subject of the containerization-based resource scheduling method shown in fig. 1 may be a computer. As can be seen from the process shown in fig. 1, in the resource scheduling method based on containerization according to the embodiment of the present invention, a pipeline template is created according to a pipeline template issuing request, then each task is sequentially converted into a corresponding container according to a task scheduling sequence in the pipeline template, then resources of the corresponding task are mounted in the containers, and the containers with the mounted resources are deployed on a target node, so that the resources can be scheduled in a balanced manner, and the full utilization of the resources is ensured.
The specific process of the embodiment of the invention is as follows:
1. and receiving a pipeline template issuing request, and acquiring a task scheduling sequence according to the pipeline template issuing request.
2. And acquiring each task and the resource of each task according to the task scheduling sequence.
3. And creating a pipeline template according to the task scheduling sequence, each task and the resource of each task.
4. And sequentially converting each task into a corresponding container according to the task scheduling sequence in the pipeline template.
5. And mounting the resources of the corresponding tasks in the container.
6. And deploying the container with the resources mounted on a predetermined target node to realize resource scheduling.
7. Running a container on the target node to obtain a task execution result; and when the task execution result is the mirror image, uploading the mirror image to a mirror image warehouse and receiving an application starting instruction.
8. And pulling the corresponding mirror image from the mirror image warehouse according to the application starting instruction and running to start the application program.
In summary, the resource scheduling method based on containerization provided by the embodiment of the invention has the following beneficial effects:
1. the tasks of the pipeline can be operated in a Kubernets cluster in a container mode, the capability of a Kubernets scheduler (kube-scheduler) for pre-selecting and optimizing strategies is utilized, so that each task can be distributed to idle nodes to be executed, and the computing resources of each node are fully used, and the CI/CD pipeline tasks are completed.
2. The user only needs the Kubernets cluster environment to conveniently install and operate the CI/CD assembly line, the work of applying and configuring an execution environment in the traditional CI/CD flow is avoided, and the cloud ecology is tightly combined.
3. The invention defines the task to be executed by the self-defined resource template device, and then organizes the task arrangement into a production line, has the characteristics of flexible arrangement and repeated use of the task, and is convenient for users to share and modify.
4. The invention monitors the running state and the log information of the task, so that a user can check the running information of the task in real time, and can find and repair the problem of the abnormal task in time. Meanwhile, the invention can clear up the tasks finished by operation, further release the computing resources and ensure the full utilization of the resources.
5. Containerization: the core of the invention is to containerize the constructed task and combine the containerized task with a Kubernetes primitive, the installation and the task operation of the CI/CD process are completed in the Kubernetes cluster, and the CI/CD flow operation can be realized only by owning the Kubernetes cluster without constructing an additional constructed environment. Meanwhile, compared with the traditional construction assembly line, the system provided by the invention can destroy the running tasks as soon as possible, and can not leave the historical running task resources.
6. Light weight: the construction Task in the invention does not need extra plug-ins, and only needs to introduce construction mirror images suitable for application when a user writes Task (Task) resources. If the user prepares to compile the code under the java environment, the java:1.8 mirror image is selected as the basic mirror image; when the precompiled environment is a golang environment, the golang to 1.4 can be selected, and the running of each task has a mirror image environment of the task, so that the tasks cannot interfere with each other.
7. The stability is strong: the user-defined resource controller operates in a manner of depolyment (a resource of Kubernetes can start a corresponding number of containers according to information filled by a user) in the Kubernetes cluster, and when the containers of the controller are deleted or abnormal, the Kubernetes can automatically create new copies, so that the number of the copies is consistent with the set number.
Based on the same inventive concept, the embodiment of the present invention further provides a resource scheduling device based on containerization, and because the principle of the device for solving the problem is similar to the resource scheduling method based on containerization, the implementation of the device can refer to the implementation of the method, and repeated parts are not described again.
Fig. 4 is a block diagram of a resource scheduling apparatus based on containerization according to an embodiment of the present invention. FIG. 5 is a block diagram of an apparatus for customizing resource templates according to another embodiment of the present invention. FIG. 6 is a block diagram of a custom resource controller in another embodiment of the invention. Fig. 7 is a block diagram of a monitoring apparatus according to another embodiment of the present invention. Fig. 8 is a block diagram showing the structure of a log device according to another embodiment of the present invention. As shown in fig. 4 to 8, the containerization-based resource scheduling apparatus includes:
the pipeline template module is used for receiving a pipeline template issuing request and creating a pipeline template according to the pipeline template issuing request;
the container generation module is used for sequentially converting each task into a corresponding container according to the task scheduling sequence in the pipeline template;
the resource mounting module is used for mounting resources of the corresponding tasks in the container;
and the container deployment module is used for deploying the container with the resources mounted on a predetermined target node so as to realize resource scheduling.
In one embodiment, the pipeline template module comprises:
the task scheduling sequence unit is used for acquiring a task scheduling sequence according to the pipeline template issuing request;
the task resource acquiring unit is used for acquiring each task and the resource of each task according to the task scheduling sequence;
and the pipeline template creating unit is used for creating a pipeline template according to the task scheduling sequence, each task and the resource of each task.
In one embodiment, the method further comprises the following steps:
and the container operation module is used for operating the container on the target node to obtain a task execution result.
In one embodiment, the method further comprises the following steps:
and the mirror image uploading module is used for uploading the mirror image to the mirror image warehouse when the task execution result is the mirror image.
In one embodiment, the method further comprises the following steps:
and the mirror image pulling module is used for receiving the application starting instruction, pulling the corresponding mirror image from the mirror image warehouse according to the application starting instruction and running to start the application program.
In one embodiment, the method further comprises the following steps:
and the container deleting module is used for deleting the container corresponding to each task so as to release the resource corresponding to each task.
In one embodiment, the method further comprises the following steps:
and the target node determining module is used for determining the target node according to the load balancing strategy.
As shown in fig. 5-8, in practical applications, the resource scheduling apparatus based on containerization includes a custom resource template apparatus, a custom resource controller, a monitoring apparatus and a log apparatus.
The user-defined resource template device comprises a production line resource management unit, a task unit, a production line unit and a production line operation unit.
The task unit defines tasks to be executed (such as compiling code, building an image, pushing an image and the like), the tasks are the smallest configurable units in the pipeline, and each task actually runs in a kubernets cluster in a container mode. If the task is successfully executed, the corresponding container status will become Completed, otherwise Failed.
The pipeline resource management unit defines input and output information of the pipeline resource, and the pipeline can be used by introducing a required resource name.
A pipeline unit is a combination of one or more tasks, resources, and custom parameters, including a pipeline template module. The names of tasks and resources to be scheduled are defined in the Pipeline template, and after a user-defined resource control unit (Pipeline-Controller) analyzes the template, corresponding tasks are created according to the task names and the resource names.
The pipeline running unit is used for triggering the specified pipeline to run and outputting a running result of the pipeline.
The user-defined resource controller comprises a user-defined resource checking unit and a user-defined resource control unit.
The self-defined resource checking unit is used for checking the correctness of the format of the pipeline template.
The user-defined resource control unit comprises a container generation module, a resource mounting module, a container deployment module, a container operation module, a mirror image uploading module, a mirror image pulling module and a target node determination module, and is used for monitoring analysis, creation, deletion, modification and query events of CRD resources. When the user-defined resource checking unit passes the checking, a task scheduling sequence set in the pipeline template is analyzed, corresponding containers are sequentially generated for each task, meanwhile, resources are mounted to a scheduler (club-scheduler) component of a Kubernet cluster in each container, resource operation conditions on each virtual machine node (node) can be monitored, the task container is deployed to a selected target node to operate through a preselected and preferred load balancing strategy, and the full use of node resources is guaranteed.
The monitoring device comprises a monitoring unit and a deleting unit, and is used for monitoring the running state of the pipeline (such as the running success or the running failure of the task) and monitoring the log information of the running of the task in real time. And deleting the container after the log collection is finished and the task operation is finished, releasing the constructed resources to be used by other tasks, and further achieving the reasonable utilization of the resources.
The monitoring unit is used for monitoring the running state of the production line in real time.
The deleting unit comprises a container deleting module used for deleting the container after the task operation is completed.
The log device comprises a log acquisition unit and a log storage unit, and is used for acquiring and storing log information when the production line runs. The log information of the pipeline running each time is collected by the log collection unit, and the collected log information is stored by the log storage unit.
To sum up, the resource scheduling device based on containerization according to the embodiment of the present invention first creates a pipeline template according to a pipeline template issuing request, then sequentially converts each task into a corresponding container according to a task scheduling sequence in the pipeline template, then mounts resources of the corresponding tasks in the containers, and deploys the containers with the resources mounted on target nodes, so that resources can be scheduled in a balanced manner, and full utilization of the resources is ensured.
The embodiment of the present invention further provides a specific implementation manner of a computer device, which can implement all the steps in the resource scheduling method based on containerization in the foregoing embodiment. Fig. 9 is a block diagram of a computer device in an embodiment of the present invention, and referring to fig. 9, the computer device specifically includes the following:
a processor (processor)901 and a memory (memory) 902.
The processor 901 is configured to call a computer program in the memory 902, and the processor executes the computer program to implement all the steps in the containerization-based resource scheduling method in the above embodiments, for example, the processor executes the computer program to implement the following steps:
receiving a pipeline template issuing request, and creating a pipeline template according to the pipeline template issuing request;
sequentially converting each task into a corresponding container according to a task scheduling sequence in the pipeline template;
mounting resources of corresponding tasks in a container;
and deploying the container with the resources mounted on a predetermined target node to realize resource scheduling.
To sum up, the computer device of the embodiment of the present invention first issues a request to create a pipeline template according to the pipeline template, then sequentially converts each task into a corresponding container according to a task scheduling sequence in the pipeline template, then mounts resources of the corresponding task in the container, and deploys the container with the resources mounted on a target node, so that the resources can be scheduled in a balanced manner, and the full utilization of the resources is ensured.
An embodiment of the present invention further provides a computer-readable storage medium capable of implementing all the steps in the resource scheduling method based on containerization in the foregoing embodiment, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements all the steps of the resource scheduling method based on containerization in the foregoing embodiment, for example, when the processor executes the computer program, the processor implements the following steps:
receiving a pipeline template issuing request, and creating a pipeline template according to the pipeline template issuing request;
sequentially converting each task into a corresponding container according to a task scheduling sequence in the pipeline template;
mounting resources of corresponding tasks in a container;
and deploying the container with the resources mounted on a predetermined target node to realize resource scheduling.
To sum up, the computer-readable storage medium according to the embodiment of the present invention first issues a request to create a pipeline template according to the pipeline template, then sequentially converts each task into a corresponding container according to a task scheduling sequence in the pipeline template, then mounts resources of the corresponding task in the container, and deploys the container with the resources mounted on a target node, so as to balance scheduling of the resources and ensure full utilization of the resources.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Those of skill in the art will further appreciate that the various illustrative logical blocks, units, and steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate the interchangeability of hardware and software, various illustrative components, elements, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design requirements of the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present embodiments.
The various illustrative logical blocks, or elements, or devices described in connection with the embodiments disclosed herein may be implemented or performed with a general purpose processor, a digital signal processor, an Application Specific Integrated Circuit (ASIC), a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other similar configuration.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. For example, a storage medium may be coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC, which may be located in a user terminal. In the alternative, the processor and the storage medium may reside in different components in a user terminal.
In one or more exemplary designs, the functions described above in connection with the embodiments of the invention may be implemented in hardware, software, firmware, or any combination of the three. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media that facilitate transfer of a computer program from one place to another. Storage media may be any available media that can be accessed by a general purpose or special purpose computer. For example, such computer-readable media can include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store program code in the form of instructions or data structures and which can be read by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Additionally, any connection is properly termed a computer-readable medium, and, thus, is included if the software is transmitted from a website, server, or other remote source via a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wirelessly, e.g., infrared, radio, and microwave. Such discs (disk) and disks (disc) include compact disks, laser disks, optical disks, DVDs, floppy disks and blu-ray disks where disks usually reproduce data magnetically, while disks usually reproduce data optically with lasers. Combinations of the above may also be included in the computer-readable medium.
Claims (10)
1. A resource scheduling method based on containerization is characterized by comprising the following steps:
receiving a pipeline template issuing request, and creating a pipeline template according to the pipeline template issuing request;
converting each task into a corresponding container in sequence according to the task scheduling sequence in the pipeline template;
mounting resources of corresponding tasks in a container;
and deploying the container with the resources mounted on a predetermined target node to realize resource scheduling.
2. The method of claim 1, wherein the creating of the pipeline template according to the pipeline template issuing request comprises:
acquiring the task scheduling sequence according to the pipeline template issuing request;
acquiring each task and the resource of each task according to the task scheduling sequence;
and creating a pipeline template according to the task scheduling sequence, the tasks and the resources of the tasks.
3. The method of claim 2, further comprising:
and running the container on the target node to obtain a task execution result.
4. The method of claim 3, further comprising:
and when the task execution result is a mirror image, uploading the mirror image to a mirror image warehouse.
5. The method of claim 4, wherein uploading the image to an image repository further comprises:
and receiving an application starting instruction, pulling a corresponding mirror image from the mirror image warehouse according to the application starting instruction, and running to start an application program.
6. The method of claim 3, wherein obtaining the task execution result further comprises:
and deleting the container corresponding to each task to release the resource corresponding to each task.
7. The method of claim 2, further comprising:
and determining a target node according to the load balancing strategy.
8. A resource scheduling apparatus based on containerization, comprising:
the pipeline template module is used for receiving a pipeline template issuing request and creating a pipeline template according to the pipeline template issuing request;
the container generation module is used for sequentially converting each task into a corresponding container according to the task scheduling sequence in the pipeline template;
the resource mounting module is used for mounting resources of the corresponding tasks in the container;
and the container deployment module is used for deploying the container with the resources mounted on a predetermined target node so as to realize resource scheduling.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and running on the processor, characterized in that the processor implements the steps of the containerization-based resource scheduling method of any of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the containerization-based resource scheduling method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110095621.2A CN112783653A (en) | 2021-01-25 | 2021-01-25 | Resource scheduling method and device based on containerization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110095621.2A CN112783653A (en) | 2021-01-25 | 2021-01-25 | Resource scheduling method and device based on containerization |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112783653A true CN112783653A (en) | 2021-05-11 |
Family
ID=75758896
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110095621.2A Pending CN112783653A (en) | 2021-01-25 | 2021-01-25 | Resource scheduling method and device based on containerization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112783653A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113190328A (en) * | 2021-05-22 | 2021-07-30 | 北京理工大学 | System identification-oriented containerized cloud workflow processing system and method |
-
2021
- 2021-01-25 CN CN202110095621.2A patent/CN112783653A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113190328A (en) * | 2021-05-22 | 2021-07-30 | 北京理工大学 | System identification-oriented containerized cloud workflow processing system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10216509B2 (en) | Continuous and automatic application development and deployment | |
| US10761899B2 (en) | Framework to improve parallel job workflow | |
| CN108196915B (en) | Code processing method and device based on application container engine and storage medium | |
| CN108964968B (en) | Service access management method and system in container cloud environment | |
| Yang et al. | A profile-based approach to just-in-time scalability for cloud applications | |
| WO2017074414A1 (en) | Software kit release management | |
| CN105359147A (en) | Online database migration | |
| CN110990026B (en) | Automatic deployment method and system for PaaS platform | |
| CN111897541B (en) | Software interaction platform and method for automatically deploying resources in cloud environment | |
| CN102520972A (en) | Video game development system and method thereof | |
| CN112486466B (en) | Method for realizing quick universal basic framework based on micro-service architecture | |
| CN102541609A (en) | Installation program manufacturing method, device and network device | |
| CN115860143A (en) | Operator model generation method, device and equipment | |
| CN112015371A (en) | Software development device under non-embedded software platform | |
| CN114912897A (en) | Workflow execution method, workflow arrangement method and electronic equipment | |
| CN112783653A (en) | Resource scheduling method and device based on containerization | |
| CN113687927A (en) | Method, device, equipment and storage medium for scheduling and configuring flash tasks | |
| CN110795105A (en) | Automatic compiling and deploying method and system for power distribution automation master station system | |
| CN110011827A (en) | Towards doctor conjuncted multi-user's big data analysis service system and method | |
| US20140047447A1 (en) | Work scheduling method and system implemented via cloud platform | |
| CN115357198B (en) | Mounting method and device of storage volume, storage medium and electronic equipment | |
| Thomas et al. | Simulation factory: Taming application configuration and workflow on high-end resources | |
| CN115480785A (en) | Container-based service deployment method and device and server | |
| CN118056183A (en) | Optimizing just-in-time compilation processes | |
| CN111309297B (en) | Script development system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |