CN113934515A - Container group scheduling method and device based on data domain and calculation domain - Google Patents
Container group scheduling method and device based on data domain and calculation domain Download PDFInfo
- Publication number
- CN113934515A CN113934515A CN202111551581.4A CN202111551581A CN113934515A CN 113934515 A CN113934515 A CN 113934515A CN 202111551581 A CN202111551581 A CN 202111551581A CN 113934515 A CN113934515 A CN 113934515A
- Authority
- CN
- China
- Prior art keywords
- node
- computing
- nodes
- computing node
- network delay
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/465—Distributed object oriented systems
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The embodiment of the invention provides a container group scheduling method and a device based on a data domain and a calculation domain, wherein the method comprises the following steps: when a newly created container group which is not scheduled to a computing node is detected, determining the task type and the resource requirement of the container group, determining an adjustable computing node which meets the task type and the resource requirement from the computing nodes, acquiring network delay information between the adjustable computing node and a data storage node, determining a target computing node from the adjustable computing node according to the network delay information, and scheduling the container group to the target computing node. By applying the embodiment of the invention, after the adjustable computing node is determined, the adjustable computing node with low network delay is determined as the target computing node according to the network delay information between the adjustable computing node and the data storage node, so that the problem of data response delay during edge computing caused by high network delay of the determined target computing node when a traditional container group scheduling mode is adopted is solved.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and an apparatus for scheduling container groups based on a data domain and a computation domain, a device and an electronic apparatus for scheduling container groups based on a data domain and a computation domain, and a computer readable medium.
Background
At present, the container scheduling scheme of the mainstream container scheduling platform such as Kubernetes and Swarm is scheduled by a central scheduler in a centralized way, and the scheduling process is as follows: traversing all nodes (nodes) of a container group (Pod) needing to be scheduled, finding out nodes meeting the resource requirement of the container group, then scoring the nodes according to a scheduling strategy, and selecting the Node with the highest score to deploy the container group.
In a traditional central cloud operating system, because a high-speed network is used among central cloud computing nodes and network delay is small, in the process of task scheduling by taking a container group as granularity, the factor of network delay is ignored, and scheduling is mainly performed according to available computing power of the nodes. However, when the calculation is extended to the edge scene, because the edge nodes are often dispersed and the network delay between the nodes is large, the traditional container group scheduling mode is adopted, which causes the data response delay during the edge calculation and cannot satisfy the service scene emphasizing the low delay of the calculation.
Disclosure of Invention
The embodiment of the invention provides a container group scheduling method, a container group scheduling device, electronic equipment and a computer-readable storage medium based on a data domain and a calculation domain, and aims to solve the problem of data response delay during edge calculation caused by the adoption of a traditional container group scheduling mode when calculation extends to an edge scene.
The embodiment of the invention discloses a container group scheduling method based on a data domain and a computing domain, which is applied to a distributed container cluster frame, wherein the distributed container cluster frame is configured with computing nodes and data storage nodes, the computing nodes are used for operating a container group, and when the computing nodes execute tasks, the computing nodes and the data storage nodes are connected through a network for data transmission, and the container group scheduling method comprises the following steps:
when a newly created container group which is not scheduled to the computing node is detected, determining the task type and the resource requirement of the container group;
determining an adjustable computing node which meets the task type and the resource requirement from the computing nodes;
acquiring network delay information between the adjustable computing node and the data storage node;
determining a target computing node from the adjustable computing nodes according to the network delay information;
scheduling the group of containers into the target compute node.
Optionally, the distributed container cluster framework is configured with a work node, and before determining the task type and resource requirement of the container group when detecting a newly created container group that is not scheduled to the computing node, the method further includes:
marking a data storage node from the working node;
and taking other working nodes except the data storage node of the working node as computing nodes.
Optionally, the distributed container cluster framework is configured with a database, and further includes, before the obtaining of the network latency information between the scalable computation node and the data storage node:
detecting network delay information between the data storage node and the computing node every preset interval time;
storing the network delay information in a database;
the obtaining network delay information between the schedulable computing node and the data storage node includes:
and inquiring network delay information between the adjustable computing node and the data storage node stored in the database.
Optionally, the determining, according to the network delay information, a target computing node from the tunable computing node includes:
and determining the adjustable computing node with the minimum network delay as a target computing node of the container group according to the network delay information.
Optionally, the determining, according to the network delay information, a target computing node from the tunable computing node includes:
determining at least one candidate adjustable computing node with the network delay smaller than a first preset threshold value from the adjustable computing nodes according to the network delay information;
scoring the candidate adjustability calculation nodes according to a scoring rule to obtain scores corresponding to the candidate adjustability calculation nodes; the scoring rules at least comprise scoring including node resource balanced distribution, mirror image positions, container group affinity, node resource minimum distribution, node affinity, node tendency avoidance container groups, default container group topology expansion and stain tolerance;
determining the candidate schedulable computing node with the highest score as a target computing node.
Optionally, after the determining, according to the network delay information, a target computing node from the schedulable computing node, the method further includes:
if the target computing node is determined to be failed from the adjustable computing node according to the network delay information;
scoring the schedulable computing nodes according to a scoring rule to obtain scores corresponding to the schedulable computing nodes; the scoring rules at least comprise scoring including node resource balanced distribution, mirror image positions, container group affinity, node resource minimum distribution, node affinity, node tendency avoidance container groups, default container group topology expansion and stain tolerance;
determining the schedulable computing node with the highest score as a target computing node.
Optionally, if determining the target computing node from the scalable computing node fails according to the network delay information, the determining may include:
and if the network delay of each schedulable computing node and the data storage node is larger than a second preset threshold, the representation fails to determine a target computing node from the schedulable computing node according to the network delay information.
The embodiment of the invention discloses a container group scheduling device based on a data domain and a computing domain, which is applied to a distributed container cluster frame, wherein the distributed container cluster frame is configured with computing nodes and data storage nodes, the computing nodes are used for operating a container group, and when the computing nodes execute tasks, the computing nodes and the data storage nodes are connected through a network for data transmission, and the device comprises:
an information determination module, configured to determine a task type and a resource requirement of a newly created container group that is not scheduled to the computing node when the container group is detected;
the first node determining module is used for determining an adjustable computing node which meets the task type and the resource requirement from the computing nodes;
the information acquisition module is used for acquiring network delay information between the adjustable computing node and the data storage node;
the second node determining module is used for determining a target computing node from the adjustable computing nodes according to the network delay information;
a container group scheduling module to schedule the container group to the target compute node.
Optionally, the distributed container cluster framework is configured with a work node, and further includes:
the node marking module is used for marking the data storage nodes from the working nodes;
and the third node determining module is used for taking other working nodes except the data storage node as the computing nodes.
Optionally, the distributed container cluster framework is configured with a database, and further includes:
the information detection module is used for detecting network delay information between the data storage node and the computing node once every preset interval time;
the information sending module is used for storing the network delay information in a database;
the information acquisition module includes:
and the information query submodule is used for querying the network delay information between the adjustable computing node and the data storage node stored in the database.
Optionally, the second node determining module includes:
and the node determining submodule is used for determining the adjustable computing node with the minimum network delay as a target computing node of the container group according to the network delay information.
Optionally, the second node determining module includes:
the candidate node determining submodule is used for determining at least one candidate adjustable computing node with the network delay smaller than a first preset threshold value from the adjustable computing nodes according to the network delay information;
the node scoring submodule is used for scoring the candidate adjustability computing nodes according to a scoring rule to obtain scores corresponding to the candidate adjustability computing nodes; the scoring rules at least comprise scoring including node resource balanced distribution, mirror image positions, container group affinity, node resource minimum distribution, node affinity, node tendency avoidance container groups, default container group topology expansion and stain tolerance;
and the node determining submodule is also used for determining the candidate schedulable computing node with the highest score as a target computing node.
Optionally, the method further comprises:
a failure determination module, configured to determine that a target computing node fails from the adjustable computing node according to the network delay information;
the node scoring module is used for scoring the schedulable computing nodes according to a scoring rule to obtain scores corresponding to the schedulable computing nodes; the scoring rules at least comprise scoring including node resource balanced distribution, mirror image positions, container group affinity, node resource minimum distribution, node affinity, node tendency avoidance container groups, default container group topology expansion and stain tolerance;
a fourth node determining module, configured to determine the schedulable computing node with the highest score as a target computing node.
Optionally, the failure determination module includes:
and the failure determining submodule is used for characterizing that the target computing node is determined to be failed from the schedulable computing node according to the network delay information if the network delay of each schedulable computing node and the data storage node is larger than a second preset threshold value.
The embodiment of the invention also discloses electronic equipment which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory finish mutual communication through the communication bus;
the memory is used for storing a computer program;
the processor is configured to implement the method according to the embodiment of the present invention when executing the program stored in the memory.
Also disclosed are one or more computer-readable media having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform a method according to an embodiment of the invention.
The embodiment of the invention has the following advantages: when a newly created container group which is not scheduled to a computing node is detected, determining the task type and the resource requirement of the container group, determining an adjustable computing node which meets the task type and the resource requirement from the computing nodes, acquiring network delay information between the adjustable computing node and a data storage node, determining a target computing node from the adjustable computing node according to the network delay information, and scheduling the container group to the target computing node. By applying the embodiment of the invention, after the schedulable computing node is determined according to the task type and the resource requirement of the container group, the schedulable computing node with low network delay during data transmission with the data storage node can be determined as the target computing node according to the network delay information between the schedulable computing node and the data storage node, so that the problem of data response delay during edge computing caused by high network delay of the determined target computing node when the traditional container group scheduling mode is adopted is solved.
Drawings
FIG. 1 is a flowchart illustrating steps of a method for scheduling a container group based on a data domain and a computation domain according to an embodiment of the present invention;
FIG. 2 is a flow chart illustrating steps of another method for scheduling a group of containers based on data fields and computation fields according to an embodiment of the present invention;
fig. 3 is a block diagram of a container group scheduling apparatus based on a data domain and a computation domain according to an embodiment of the present invention;
FIG. 4 is a block diagram of an electronic device provided in an embodiment of the invention;
fig. 5 is a schematic diagram of a computer-readable medium provided in an embodiment of the invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
The embodiment of the invention discloses a container group scheduling method based on a data domain and a computing domain, which fuses the technologies of the data domain and the computing domain, determines a schedulable computing node according to the task type and the resource requirement of a container group, and then determines a target computing node from the schedulable computing node with low network delay during data transmission with a data storage node according to network delay information between the schedulable computing node and the data storage node, so as to solve the problems that when the computation extends to an edge scene, the data response delay during the edge computation is caused by adopting a traditional container group scheduling mode, and the service scene emphasizing the low delay cannot be satisfied.
The container is a virtualization technology, a plurality of isolated operating system environments are provided on one host, the computing capacity of the whole computing center is pooled, an operating environment with high availability, high reliability and flexible scalability is provided for applications, and a container arrangement engine is responsible for managing all hosts and containers of a single data center and even multiple data centers, so that a cloud computing platform is formed. Container layout engines include principally Swarm, Mesos, Kubernets, etc., but Kubernets has now become a de facto container layout standard.
The host (which can be a physical machine or a virtual machine) in Kubernetes is mainly divided into a Master Node (Master) and a working Node (Node), wherein the working Node is a host running a specific container group (Pod) and is responsible for providing specific services after the provision, and the working Node has self-repairing capability; the main node is used for managing the working nodes and controlling what container group the working nodes specifically operate.
Referring to fig. 1, a flowchart illustrating steps of a container group scheduling method based on a data domain and a computation domain provided in an embodiment of the present invention is shown, and is applied to a distributed container cluster framework, where the distributed container cluster framework is configured with computation nodes and data storage nodes, the computation nodes are used for running a container group, and when the computation nodes execute tasks, the computation nodes and the data storage nodes perform data transmission through network connection, where the method specifically includes the following steps:
step 101: when a newly created container group is detected that is not scheduled to the compute node, the task type and resource requirements of the container group are determined.
The working nodes (nodes) comprise computing nodes and data storage nodes, the computing nodes comprise central cloud computing nodes and edge computing nodes, the computing nodes are used for operating a container group (Pod), the data storage nodes are usually nodes for operating databases such as MySQL and MongoDB, and when the computing nodes execute tasks, the computing nodes and the data storage nodes are connected through a network to perform data transmission.
The newly created container group has corresponding task types and resource requirements, the task types comprise Web application, online transaction, video/graphic processing, data analysis, machine learning, network processing and the like, and different types of processing computing nodes such as a CPU (Central processing Unit), a GPU (graphics processing Unit), an SNIC (network interface) and the like are required to be adopted for scheduling corresponding to different task types; resource requirements include, but are not limited to, at least one of: processor resources, memory resources.
Specifically, when a newly created container group is detected that is not scheduled to a compute node, the task type and resource requirements of the container group are determined.
Step 102: and determining the adjustable computing nodes meeting the task types and the resource requirements from the computing nodes.
In particular, each container within a group of containers has a different demand for resources, and the group of containers itself has a different resource demand. Thus, the container groups in the cluster need to be filtered once before they are dispatched to the compute nodes, depending on these particular resource requirements and task types. For example, all the computing nodes in the distributed container cluster framework are queried through the scheduler, the computing nodes meeting the task type and the resource requirement of the container group are determined, and the computing nodes are used as the adjustable computing nodes. When no one compute node can meet the resource requirements and task type of the container group, then the container group will stay unscheduled until the scheduler can find a suitable compute node.
Step 103: and acquiring network delay information between the adjustable computing node and the data storage node.
Step 104: and determining a target computing node from the adjustable computing nodes according to the network delay information.
The central cloud computing nodes use a high-speed network, network delay is small, when a scheduler performs task scheduling, the factor of network delay is ignored, and scheduling is performed mainly according to available computing power of the computing nodes. When the calculation is extended to the edge scene, because the edge calculation nodes are often dispersed, the network delay between the edge calculation nodes and the data storage is large, and at the moment, the network delay information between the network nodes needs to be considered; the network delay refers to the time delay existing when data is transmitted between the adjustable computing node and the data storage node.
Specifically, the method includes the steps of determining the adjustable computing nodes meeting task types and resource requirements, obtaining network delay information between each adjustable computing node and the data storage nodes, and determining target computing nodes from the adjustable computing nodes according to the network delay information, for example, determining a schedulable computing node with the lowest network delay from the adjustable computing nodes as the target computing node.
Step 105: scheduling the group of containers into the target compute node.
Specifically, after the target computing node is determined, the container group is scheduled to the target computing node for operation.
In the embodiment of the invention, after the schedulable computing node is determined according to the task type and the resource requirement of the container group, the schedulable computing node with low network delay during data transmission with the data storage node can be determined as the target computing node according to the network delay information between the schedulable computing node and the data storage node, so that the problem of data response delay during edge computing caused by high network delay of the determined target computing node when the traditional container group scheduling mode is adopted is solved.
Referring to fig. 2, a flowchart illustrating steps of another container group scheduling method based on a data domain and a computation domain provided in an embodiment of the present invention is shown, and is applied to a distributed container cluster framework, where the distributed container cluster framework is configured with computation nodes and data storage nodes, the computation nodes are used for running a container group, and when the computation nodes execute tasks, the computation nodes and the data storage nodes perform data transmission through network connection, where the method specifically includes the following steps:
step 201: when a newly created container group is detected that is not scheduled to the compute node, the task type and resource requirements of the container group are determined.
In an embodiment of the present invention, the distributed container cluster framework is configured with a work node, and before the step 201, the method further includes: marking a data storage node from the working node; and taking other working nodes except the data storage node of the working node as computing nodes.
Specifically, the working nodes (nodes) do not distinguish the computing nodes from the data storage nodes, and therefore, nodes in the working nodes for running databases such as MySQL, MongoDB, and the like are marked, and whether the labels of the data storage nodes are marked is detected through the kubecect gel nodes. And the other working nodes except the data storage node of the working node are used as the computing nodes, so that the data storage node and the computing nodes are distinguished.
Step 202: and determining the adjustable computing nodes meeting the task types and the resource requirements from the computing nodes.
Step 203: and acquiring network delay information between the adjustable computing node and the data storage node.
In an embodiment of the present invention, the distributed container cluster framework is configured with a database, and before the step 203, the method further includes: detecting network delay information between the data storage node and the computing node every preset interval time; storing the network delay information in a database; the step 203 includes: and inquiring network delay information between the adjustable computing node and the data storage node stored in the database.
Wherein the database (etcd) is a distributed, highly available, consistent key-value storage database.
Specifically, when each data storage node runs at a timing Job, the data storage node sends a test data packet to other computing nodes, confirms whether the computing nodes respond or not and counts response time, obtains average delay of the acquired data storage node and other computing nodes, acquires network delay information of the data storage node and other computing nodes, reports the delay to a host node (master) of Kubernetes, and stores the network delay information in a database (etcd) through the host node.
After determining the schedulable computing node, querying a database for network delay information between the schedulable computing node and the data storage node.
Step 204: and determining the adjustable computing node with the minimum network delay as a target computing node of the container group according to the network delay information.
Specifically, after the schedulable computing node is determined, the schedulable computing node with the minimum network delay is determined as the target computing node of the container group from the schedulable computing node according to the network delay information.
In an embodiment of the present invention, the determining a target computing node from the scalable computing node according to the network delay information includes: determining at least one candidate adjustable computing node with the network delay smaller than a first preset threshold value from the adjustable computing nodes according to the network delay information; scoring the candidate adjustability calculation nodes according to a scoring rule to obtain scores corresponding to the candidate adjustability calculation nodes; the scoring rules at least comprise scoring including node resource balanced distribution, mirror image positions, container group affinity, node resource minimum distribution, node affinity, node tendency avoidance container groups, default container group topology expansion and stain tolerance; determining the candidate schedulable computing node with the highest score as a target computing node.
The first preset threshold is smaller than or equal to the second preset threshold.
Specifically, after the schedulable computing nodes are determined, candidate schedulable computing nodes with the network delay smaller than a first preset threshold are determined from the schedulable computing nodes according to the network delay information, and if the number of the candidate schedulable computing nodes is more than one, the candidate schedulable computing nodes are scored by adopting a container group scheduling process of kubernets default, the candidate schedulable computing node with the highest score is determined as a target computing node, and if a plurality of candidate schedulable computing nodes with the highest score exist, one candidate schedulable computing node is randomly selected as the target computing node.
And if one candidate schedulable computing node is available, determining the candidate schedulable computing node as the target schedulable computing node.
In an embodiment of the present invention, after determining a target computing node from the scalable computing node according to the network delay information, the method further includes: if the target computing node is determined to be failed from the adjustable computing node according to the network delay information; scoring the schedulable computing nodes according to a scoring rule to obtain scores corresponding to the schedulable computing nodes; the scoring rules at least comprise scoring including node resource balanced distribution, mirror image positions, container group affinity, node resource minimum distribution, node affinity, node tendency avoidance container groups, default container group topology expansion and stain tolerance; determining the schedulable computing node with the highest score as a target computing node.
Specifically, when a target computing node of a container group cannot be determined from the adjustable computing nodes according to network delay information, scoring the adjustable computing nodes by adopting a kubernetes default container group scheduling process, wherein the scoring comprises node resource balanced distribution, mirror image positions, container group affinity, node resource minimum distribution, node affinity, node tendency avoidance container groups, default container group topology expansion, stain tolerance and the like, obtaining scores corresponding to the adjustable computing nodes, determining the schedulable computing node with the highest score as the target computing node, and if a plurality of schedulable computing nodes with the highest score exist, randomly selecting one node from the schedulable computing nodes as the target computing node.
In an embodiment of the present invention, if determining the target computing node from the scalable computing node fails according to the network delay information, the determining includes: and if the network delay of each schedulable computing node and the data storage node is larger than a second preset threshold, the representation fails to determine a target computing node from the schedulable computing node according to the network delay information.
Specifically, when the network delays of all the schedulable computing nodes and the data storage nodes are small and can be ignored, for example, the network delays of the schedulable computing nodes and the data storage nodes are both greater than a second preset threshold, it is not necessary to determine the target computing node according to the network delay information at this time, and it is characterized that the determination of the target computing node from the schedulable computing nodes fails according to the network delay information.
Step 205: scheduling the group of containers into the target compute node.
In the embodiment of the invention, after the schedulable computing node is determined according to the task type and the resource requirement of the container group, the schedulable computing node with low network delay during data transmission with the data storage node can be determined as the target computing node according to the network delay information between the schedulable computing node and the data storage node, so that the problem of data response delay during edge computing caused by high network delay of the determined target computing node when the traditional container group scheduling mode is adopted is solved.
In addition, network delay between all schedulable computing nodes and data storage nodes is small, and the untimely network delay can be ignored, so that the target computing node is confirmed without determining the target computing node from the schedulable computing nodes according to the network delay information, and the target computing node is confirmed by adopting a container group scheduling process of kubernets default, so that the target computing node matched with the container group is confirmed.
It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.
Referring to fig. 3, a block diagram of a structure of a container group scheduling apparatus based on a data domain and a computation domain, which is provided in an embodiment of the present invention, and is applied to a distributed container cluster framework, where the distributed container cluster framework is configured with a computation node and a data storage node, the computation node is used to run a container group, and when the computation node executes a task, the computation node and the data storage node perform data transmission through network connection, and specifically includes the following modules:
an information determining module 301, configured to determine a task type and a resource requirement of a newly created container group that is not scheduled to the computing node when the container group is detected;
a first node determining module 302, configured to determine, from the computing nodes, an adjustable computing node that meets the task type and the resource requirement;
an information obtaining module 303, configured to obtain network delay information between the scalable computation node and the data storage node;
a second node determining module 304, configured to determine a target computing node from the scalable computing nodes according to the network delay information;
a container group scheduling module 305 for scheduling the container group into the target compute node.
In an embodiment of the present invention, the distributed container cluster framework is configured with a work node, and further includes:
the node marking module is used for marking the data storage nodes from the working nodes;
and the third node determining module is used for taking other working nodes except the data storage node as the computing nodes.
In an embodiment of the present invention, the distributed container cluster framework is configured with a database, and further includes:
the information detection module is used for detecting network delay information between the data storage node and the computing node once every preset interval time;
the information sending module is used for storing the network delay information in a database;
the information acquisition module includes:
and the information query submodule is used for querying the network delay information between the adjustable computing node and the data storage node stored in the database.
In an embodiment of the present invention, the second node determining module 304 includes:
and the node determining submodule is used for determining the adjustable computing node with the minimum network delay as a target computing node of the container group according to the network delay information.
In an embodiment of the present invention, the second node determining module 304 includes:
the candidate node determining submodule is used for determining at least one candidate adjustable computing node with the network delay smaller than a first preset threshold value from the adjustable computing nodes according to the network delay information;
the node scoring submodule is used for scoring the candidate adjustability computing nodes according to a scoring rule to obtain scores corresponding to the candidate adjustability computing nodes; the scoring rules at least comprise scoring including node resource balanced distribution, mirror image positions, container group affinity, node resource minimum distribution, node affinity, node tendency avoidance container groups, default container group topology expansion and stain tolerance;
and the node determining submodule is also used for determining the candidate schedulable computing node with the highest score as a target computing node.
In an embodiment of the present invention, the method further includes:
a failure determination module, configured to determine that a target computing node fails from the adjustable computing node according to the network delay information;
the node scoring module is used for scoring the schedulable computing nodes according to a scoring rule to obtain scores corresponding to the schedulable computing nodes; the scoring rules at least comprise scoring including node resource balanced distribution, mirror image positions, container group affinity, node resource minimum distribution, node affinity, node tendency avoidance container groups, default container group topology expansion and stain tolerance;
a fourth node determining module, configured to determine the schedulable computing node with the highest score as a target computing node.
In an embodiment of the present invention, the failure determining module includes:
and the failure determining submodule is used for characterizing that the target computing node is determined to be failed from the schedulable computing node according to the network delay information if the network delay of each schedulable computing node and the data storage node is larger than a second preset threshold value.
For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.
In addition, an electronic device is further provided in the embodiments of the present invention, as shown in fig. 4, and includes a processor 401, a communication interface 402, a memory 403, and a communication bus 404, where the processor 401, the communication interface 402, and the memory 403 complete mutual communication through the communication bus 404,
a memory 403 for storing a computer program;
the processor 401, when executing the program stored in the memory 403, is configured to implement the container group scheduling method based on the data domain and the computation domain as described in the foregoing embodiments.
The communication bus mentioned in the above terminal may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the terminal and other equipment.
The Memory may include a Random Access Memory (RAM) or a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the Integrated Circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component.
In yet another embodiment provided by the present invention, as shown in fig. 5, there is further provided a computer-readable storage medium 501, which stores instructions that, when executed on a computer, cause the computer to execute the container group scheduling method based on data domain and computation domain described in the above embodiments.
In yet another embodiment of the present invention, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the container group scheduling method based on data domain and computation domain as described in the above embodiments.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (10)
1. A container group scheduling method based on a data domain and a computing domain is applied to a distributed container cluster framework, the distributed container cluster framework is configured with computing nodes and data storage nodes, the computing nodes are used for operating a container group, and when the computing nodes execute tasks, the computing nodes and the data storage nodes perform data transmission through network connection, and the method comprises the following steps:
when a newly created container group which is not scheduled to the computing node is detected, determining the task type and the resource requirement of the container group;
determining an adjustable computing node which meets the task type and the resource requirement from the computing nodes;
acquiring network delay information between the adjustable computing node and the data storage node;
determining a target computing node from the adjustable computing nodes according to the network delay information;
scheduling the group of containers into the target compute node.
2. The method of claim 1, wherein the distributed container cluster framework is configured with working nodes, and further comprising, prior to said determining a task type and resource requirements of a container group when a newly created container group is detected that is not scheduled to the computing node, the step of:
marking a data storage node from the working node;
and taking other working nodes except the data storage node of the working node as computing nodes.
3. The method of claim 1, wherein the distributed container cluster framework is configured with a database, further comprising, prior to said obtaining network latency information between the scaleable computing node and the data storage node:
detecting network delay information between the data storage node and the computing node every preset interval time;
storing the network delay information in a database;
the obtaining network delay information between the schedulable computing node and the data storage node includes:
and inquiring network delay information between the adjustable computing node and the data storage node stored in the database.
4. The method of claim 1, wherein determining a target compute node from the tunable compute node based on the network delay information comprises:
and determining the adjustable computing node with the minimum network delay as a target computing node of the container group according to the network delay information.
5. The method of claim 1, wherein determining a target compute node from the tunable compute node based on the network delay information comprises:
determining at least one candidate adjustable computing node with the network delay smaller than a first preset threshold value from the adjustable computing nodes according to the network delay information;
scoring the candidate adjustability calculation nodes according to a scoring rule to obtain scores corresponding to the candidate adjustability calculation nodes; the scoring rules at least comprise scoring including node resource balanced distribution, mirror image positions, container group affinity, node resource minimum distribution, node affinity, node tendency avoidance container groups, default container group topology expansion and stain tolerance;
determining the candidate schedulable computing node with the highest score as a target computing node.
6. The method of claim 1, after determining a target computing node from the tunable computing node based on the network delay information, further comprising:
if the target computing node is determined to be failed from the adjustable computing node according to the network delay information;
scoring the schedulable computing nodes according to a scoring rule to obtain scores corresponding to the schedulable computing nodes; the scoring rules at least comprise scoring including node resource balanced distribution, mirror image positions, container group affinity, node resource minimum distribution, node affinity, node tendency avoidance container groups, default container group topology expansion and stain tolerance;
determining the schedulable computing node with the highest score as a target computing node.
7. The method of claim 6, wherein determining a target computing node from the scaleable computing nodes that fails based on the network delay information comprises:
and if the network delay of each schedulable computing node and the data storage node is larger than a second preset threshold, the representation fails to determine a target computing node from the schedulable computing node according to the network delay information.
8. A container group scheduling device based on a data domain and a computation domain is applied to a distributed container cluster framework, the distributed container cluster framework is configured with computation nodes and data storage nodes, the computation nodes are used for running a container group, and when the computation nodes execute tasks, the computation nodes and the data storage nodes perform data transmission through network connection, the device comprises:
an information determination module, configured to determine a task type and a resource requirement of a newly created container group that is not scheduled to the computing node when the container group is detected;
the first node determining module is used for determining an adjustable computing node which meets the task type and the resource requirement from the computing nodes;
the information acquisition module is used for acquiring network delay information between the adjustable computing node and the data storage node;
the second node determining module is used for determining a target computing node from the adjustable computing nodes according to the network delay information;
a container group scheduling module to schedule the container group to the target compute node.
9. An electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;
the memory is used for storing a computer program;
the processor, when executing a program stored on the memory, implementing the method of any of claims 1-7.
10. One or more computer-readable media having instructions stored thereon that, when executed by one or more processors, cause the processors to perform the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111551581.4A CN113934515A (en) | 2021-12-17 | 2021-12-17 | Container group scheduling method and device based on data domain and calculation domain |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111551581.4A CN113934515A (en) | 2021-12-17 | 2021-12-17 | Container group scheduling method and device based on data domain and calculation domain |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113934515A true CN113934515A (en) | 2022-01-14 |
Family
ID=79289271
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111551581.4A Pending CN113934515A (en) | 2021-12-17 | 2021-12-17 | Container group scheduling method and device based on data domain and calculation domain |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113934515A (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160357610A1 (en) * | 2014-02-27 | 2016-12-08 | Intel Corporation | Techniques to allocate configurable computing resources |
| CN109582452A (en) * | 2018-11-27 | 2019-04-05 | 北京邮电大学 | A kind of container dispatching method, dispatching device and electronic equipment |
| CN109819037A (en) * | 2019-01-29 | 2019-05-28 | 武汉鸿瑞达信息技术有限公司 | A kind of method and system of adaptive polo placement and communication |
| CN110380891A (en) * | 2019-06-13 | 2019-10-25 | 中国人民解放军国防科技大学 | Edge computing service resource allocation method and device and electronic equipment |
| CN111371603A (en) * | 2020-02-27 | 2020-07-03 | 长沙市源本信息科技有限公司 | Service instance deployment method and device applied to edge computing |
| CN111522639A (en) * | 2020-04-16 | 2020-08-11 | 南京邮电大学 | Multidimensional resource scheduling method under Kubernetes cluster architecture system |
| CN112084026A (en) * | 2020-09-02 | 2020-12-15 | 国网河北省电力有限公司石家庄供电分公司 | Low-energy-consumption edge computing resource deployment system and method based on particle swarm |
| CN113037800A (en) * | 2019-12-09 | 2021-06-25 | 华为技术有限公司 | Job scheduling method and job scheduling device |
| CN113190358A (en) * | 2021-05-25 | 2021-07-30 | 曙光信息产业(北京)有限公司 | Job distribution method and device, electronic equipment and readable storage medium |
-
2021
- 2021-12-17 CN CN202111551581.4A patent/CN113934515A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160357610A1 (en) * | 2014-02-27 | 2016-12-08 | Intel Corporation | Techniques to allocate configurable computing resources |
| CN109582452A (en) * | 2018-11-27 | 2019-04-05 | 北京邮电大学 | A kind of container dispatching method, dispatching device and electronic equipment |
| CN109819037A (en) * | 2019-01-29 | 2019-05-28 | 武汉鸿瑞达信息技术有限公司 | A kind of method and system of adaptive polo placement and communication |
| CN110380891A (en) * | 2019-06-13 | 2019-10-25 | 中国人民解放军国防科技大学 | Edge computing service resource allocation method and device and electronic equipment |
| CN113037800A (en) * | 2019-12-09 | 2021-06-25 | 华为技术有限公司 | Job scheduling method and job scheduling device |
| CN111371603A (en) * | 2020-02-27 | 2020-07-03 | 长沙市源本信息科技有限公司 | Service instance deployment method and device applied to edge computing |
| CN111522639A (en) * | 2020-04-16 | 2020-08-11 | 南京邮电大学 | Multidimensional resource scheduling method under Kubernetes cluster architecture system |
| CN112084026A (en) * | 2020-09-02 | 2020-12-15 | 国网河北省电力有限公司石家庄供电分公司 | Low-energy-consumption edge computing resource deployment system and method based on particle swarm |
| CN113190358A (en) * | 2021-05-25 | 2021-07-30 | 曙光信息产业(北京)有限公司 | Job distribution method and device, electronic equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11943104B2 (en) | Application migration system | |
| US8185905B2 (en) | Resource allocation in computing systems according to permissible flexibilities in the recommended resource requirements | |
| US8112546B2 (en) | Routing users to receive online services based on online behavior | |
| US20170373932A1 (en) | Configuration discovery service data visualization | |
| JP2020150545A (en) | Intelligent configuration detection techniques | |
| US10505832B2 (en) | Resource coordinate system for data centers | |
| JP5245711B2 (en) | Distributed data processing system, distributed data processing method, and distributed data processing program | |
| US7738504B1 (en) | Method of establishing and updating master node in computer network | |
| US20100306162A1 (en) | Proximity data import | |
| CN107491340B (en) | Method for realizing huge virtual machine crossing physical machines | |
| US8291092B2 (en) | Managing IPMI sessions | |
| CN115277566B (en) | Load balancing method and device for data access, computer equipment and medium | |
| US8903871B2 (en) | Dynamic management of log persistence | |
| US20060210051A1 (en) | Method and system for managing computer resource in system | |
| CN111418187A (en) | Scalable statistics and analysis mechanism in cloud networks | |
| US9245232B1 (en) | Machine generated service cache | |
| US7743140B2 (en) | Binding processes in a non-uniform memory access system | |
| CN113934515A (en) | Container group scheduling method and device based on data domain and calculation domain | |
| CN107491270B (en) | Resource access method and device of multi-control storage system | |
| CN115878333A (en) | Method, device and equipment for judging consistency between process groups | |
| US20130318102A1 (en) | Data Handling in a Cloud Computing Environment | |
| CN109614242B (en) | Computing capacity sharing method, device, equipment and medium | |
| CN110928679B (en) | Resource allocation method and device | |
| CN113553314A (en) | Service processing method, device, equipment and medium of super-convergence system | |
| US10904093B2 (en) | Cluster system self-organizing method, device, and cluster system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220114 |