CN114265677A

CN114265677A - Scheduling method and device for load balancing and computing equipment

Info

Publication number: CN114265677A
Application number: CN202111561467.XA
Authority: CN
Inventors: 周鹏; 叶中玉; 胡翔; 余昇锦
Original assignee: Xi'an Tongxin Software Technology Co ltd
Current assignee: Uniontech Software Technology Co Ltd
Priority date: 2021-12-15
Filing date: 2021-12-15
Publication date: 2022-04-01

Abstract

The invention discloses a scheduling method and device for load balancing and computing equipment. The method comprises the following steps: determining a migrating process CPU and one or more migration target CPUs in CPUs of all nodes of the multi-core system at least based on the load of the CPUs; traversing all processes on the CPU of the migrated process, and determining the process to be migrated at least according to the CPU affinity of each process; determining the corresponding relation between the process to be migrated and the migration target CPU according to the CPU affinity of the process to be migrated; and migrating the process to be migrated to the corresponding migration target CPU according to the corresponding relation. The invention also discloses a corresponding device and a computing device.

Description

Scheduling method and device for load balancing and computing equipment

Technical Field

The invention relates to the technical field of data scheduling, in particular to a scheduling method, a scheduling device and a computing device for load balancing.

Background

At present, scenes such as big data, AI (Artificial Intelligence) and the like generally have a multi-thread parallel operation condition, so that a scene that ten or more processes run on one CPU can appear on a NUMA (Non Uniform Memory Access) architecture server, at this time, when a load balancing mechanism of a system is relied on to migrate processes to other CPUs, load balancing can only be performed between two CPUs each time, and when NODE-crossing scheduling is performed, load balancing interval time is long, and performance is poor. Therefore, the current load balancing scheme has a low balancing speed, and the parallel operation performance in a short time is poor.

Therefore, for a complex system with a multi-layer scheduling domain, a feasible method for rapidly distributing the process to each CPU is needed.

Disclosure of Invention

To this end, the present invention provides a scheduling method, apparatus and computing device for load balancing in an attempt to solve or at least alleviate at least one of the problems presented above.

According to an aspect of the present invention, there is provided a scheduling method for load balancing, comprising the steps of: determining a migrating process CPU and one or more migration target CPUs in CPUs of all nodes of the multi-core system at least based on the load of the CPUs; traversing all processes on the CPU of the migrated process, and determining the process to be migrated at least according to the CPU affinity of each process; determining the corresponding relation between the process to be migrated and the migration target CPU according to the CPU affinity of the process to be migrated; and migrating the process to be migrated to the corresponding migration target CPU according to the corresponding relation.

Optionally, in the scheduling method according to the present invention, the step of determining, based on at least the size of the CPU load, the migrated process CPU and the one or more migrated target CPUs includes: selecting at least part of all the nodes as a first candidate node, and determining the CPU with the maximum load or the CPU with the load larger than or equal to a preset emigration threshold value as an emigration process CPU in CPUs corresponding to the first candidate node; and selecting at least part of all the nodes as second candidate nodes, and determining N CPUs with the minimum load or N CPUs with the load smaller than or equal to a preset immigration threshold value as migration target CPUs in CPUs corresponding to the second candidate nodes, wherein N is a positive integer larger than or equal to 1.

Optionally, in the scheduling method according to the present invention, the first candidate node is a node with a largest load among all the nodes; the second candidate node is a node which is less than or equal to a distance threshold value from the node where the migrating process CPU is located in all the nodes, or one or more nodes with the minimum load.

Optionally, in the scheduling method according to the present invention, before the step of determining, based on at least the size of the CPU load, the migrating process CPU and the one or more migration target CPUs, the method further includes: periodically detecting the load condition of each CPU in all the nodes; or detecting the load condition of each CPU in all nodes when the state of a process changes.

Optionally, in the scheduling method according to the present invention, traversing all processes on the CPU of the migrated process, and determining the process to be migrated according to at least the CPU affinity of each process includes: for each process, determining whether the process allows migration according to the CPU affinity of the process, and determining the migration allowed range of the process under the condition that the process allows migration; when the process is allowed to be migrated and at least one of the one or more migration target CPUs is within the allowed migration range of the process, determining the process as a process to be migrated.

Optionally, in the scheduling method according to the present invention, the step of determining the correspondence between the process to be migrated and the migration target CPU according to the CPU affinity of the process to be migrated includes: determining the corresponding relation according to the CPU affinity of the processes to be migrated so as to minimize the quantity difference of the processes to be migrated among all the migration target CPUs after migration; or, determining the corresponding relation according to the CPU affinity of the process to be migrated, so that the load difference of the process to be migrated among all migration target CPUs can reach the minimum after migration; or, determining the corresponding relation according to the CPU affinity of the processes to be migrated, so that the sequence of the processes to be migrated on the running queue of the same migration target CPU after migration is consistent with the sequence of the processes to be migrated on the running queue of the CPU of the migrated process; or, determining the corresponding relation according to the CPU affinity of the process to be migrated, so that the closer to the CPU of the migrated process, the more the number of the processes to be migrated corresponding to the migrated target CPU; or, determining the corresponding relation according to the CPU affinity of the process to be migrated, so that the smaller the load, the larger the number of the processes to be migrated corresponding to the migration target CPU.

According to another aspect of the present invention, there is also provided a scheduling apparatus for load balancing, including: the searching module is suitable for determining a migrating process CPU and one or more migrating target CPUs in CPUs of all nodes of the multi-core system at least based on the load of the CPUs; the traversal module is suitable for traversing all processes on the CPU of the migrated process and determining the process to be migrated at least according to the CPU affinity of each process; the matching module is suitable for determining the corresponding relation between the process to be migrated and the migration target CPU according to the CPU affinity of the process to be migrated; and the migration module is suitable for migrating the process to be migrated to the corresponding migration target CPU according to the corresponding relation.

Optionally, in the scheduling method according to the present invention, the lookup module determines, based on at least the CPU load size, that the migrating process CPU and the one or more migration target CPUs are adapted to: selecting at least part of all the nodes as a first candidate node, and determining the CPU with the maximum load or the CPU with the load larger than or equal to a preset emigration threshold value as an emigration process CPU in CPUs corresponding to the first candidate node; and selecting at least part of all the nodes as second candidate nodes, and determining N CPUs with the minimum load or N CPUs with the load smaller than or equal to a preset immigration threshold value as migration target CPUs in CPUs corresponding to the second candidate nodes, wherein N is a positive integer larger than or equal to 1.

According to another aspect of the present invention, there is also provided a computing device comprising: at least one processor and a memory storing program instructions; the program instructions, when read and executed by a processor, cause the computing device to perform the scheduling method for load balancing as described above.

According to still another aspect of the present invention, there is also provided a readable storage medium storing program instructions, which when read and executed by a computing device, cause the computing device to execute the scheduling method for load balancing as above.

According to the scheduling method, the scheduling device and the computing equipment for load balancing, the following beneficial effects can be realized:

according to the method and the device, a large number of processes on the migrated process CPU can be balanced to other idle CPUs through one-time rapid balancing, time overhead caused by load balancing is reduced, and therefore in a big data operation scene, if a large number of processes are instantly created, the performance of a test process can be remarkably improved, and through testing on a domestic Feiteng S2500 server, the load balancing time which is originally needed for more than 5 seconds in a part of scenes can be shortened to within 1 second.

Drawings

To the accomplishment of the foregoing and related ends, certain illustrative aspects are described herein in connection with the following description and the annexed drawings, which are indicative of various ways in which the principles disclosed herein may be practiced, and all aspects and equivalents thereof are intended to be within the scope of the claimed subject matter. The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description read in conjunction with the accompanying drawings. Throughout this disclosure, like reference numerals generally refer to like parts or elements.

FIG. 1 shows a schematic structural diagram of a NUMA architecture according to one embodiment of the invention;

FIG. 2 shows a schematic diagram of NUMA architecture load balancing according to one embodiment of the invention;

FIG. 3 shows a schematic diagram of NUMA architecture load balancing according to one embodiment of the invention;

FIG. 4 shows a schematic diagram of a computing device 400 according to one embodiment of the invention;

FIG. 5 illustrates a flow diagram of a scheduling method 500 for load balancing, according to one embodiment of the invention;

fig. 6 is a schematic structural diagram illustrating a scheduling apparatus 600 for load balancing according to an embodiment of the present invention;

FIG. 7 is a schematic block diagram of a first computer-readable storage medium according to an embodiment of the present application;

fig. 8 is a schematic block diagram of a second computer-readable storage medium according to an embodiment of the present application.

Detailed Description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

FIG. 1 is a framework diagram of a NUMA architecture of an embodiment of the invention; the NUMA architecture includes 16 NODES, NODES 0-15, each NODE including 8 CPUs, CPU 0-7, assuming that processes P0-P15 run on CPU 0 of NODE 0, the remaining CPUs on NODE 0 are idle, and all CPUs on NODE 15 are idle. The load balancing mechanism aims at balancing processes to other CPUs respectively, in the prior art, multiple times of load balancing are needed, load balancing needs to be performed in NODE 0 at first, because idle CPUs pull tasks from busy CPUs in the load balancing process, at this time, 7 idle CPUs on NODE 0 need to be performed for load balancing to achieve load balancing in NODE 0, as shown in FIG. 2, at this time, 8 idle CPUs are on NODE 15, other NODE are in busy states, at this time, the load balancing aim is to transfer the processes from NODE 0 to NODE 15, at this time, at least 8 times of load balancing are needed to achieve load balancing of NODE 0 and NODE 15, as shown in FIG. 3, the load balancing interval time across NODE is long, and performance is poor. The load balancing efficiency in the prior art is not high, tasks cannot be rapidly balanced on all CPUs, and meanwhile, because the load balancing can only be performed between two CPUs every time in the prior art, the tasks are migrated from one CPU to the other CPU, when a large number of tasks exist on a single CPU, the tasks cannot be balanced in time, and the performance loss caused by too slow balancing is caused.

When the method is used for load balancing scheduling, a busy CPU actively pushes tasks to a low-load CPU, and a plurality of tasks can be pushed to a plurality of CPUs at one time, so that the effect of one-to-many is achieved. The range of selecting a low-load CPU is also extended to the entire multi-NODE system.

The scheduling method for load balancing of the present invention is executed in a computing device. The computing device may be any device with storage and computing capabilities, and may be implemented as, for example, a server, a workstation, or the like, or may be implemented as a personal computer such as a desktop computer or a notebook computer, or may be implemented as a terminal device such as a mobile phone, a tablet computer, a smart wearable device, or an internet of things device, but is not limited thereto.

FIG. 4 shows a schematic diagram of a computing device 400 according to one embodiment of the invention. It should be noted that the computing device 400 shown in fig. 4 is only an example, and in practice, the computing device for implementing the scheduling method for load balancing of the present invention may be any type of device, and the hardware configuration thereof may be the same as the computing device 400 shown in fig. 4 or different from the computing device 400 shown in fig. 4. In practice, the computing device implementing the scheduling method for load balancing of the present invention may add or delete hardware components of the computing device 400 shown in fig. 4, and the present invention does not limit the specific hardware configuration of the computing device.

As shown in FIG. 4, in a basic configuration 402, a computing device 400 typically includes a system memory 406 and one or more processors 404. A memory bus 408 may be used for communicating between the processor 404 and the system memory 406.

Depending on the desired configuration, processor 404 may be any type of processing, including but not limited to: a microprocessor (UP), a microcontroller (UC), a digital information processor (DSP), or any combination thereof. Processor 404 may include one or more levels of cache, such as a level one cache 410 and a level two cache 412, a processor core 414, and registers 416. The example processor core 414 may include an Arithmetic Logic Unit (ALU), a Floating Point Unit (FPU), a digital signal processing core (DSP core), or any combination thereof. The example memory controller 418 may be used with the processor 404, or in some implementations the memory controller 418 may be an internal part of the processor 404.

Depending on the desired configuration, system memory 406 may be any type of memory including, but not limited to: volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. System memory 406 may include an operating system 420, one or more applications 422, and program data 424. The application 422 is actually a plurality of program instructions that direct the processor 404 to perform corresponding operations. In some implementations, the application 422 can be arranged to cause the processor 404 to operate with the program data 424 on an operating system.

Computing device 400 may also include a storage interface bus 434. A storage interface bus 434 enables communication from the storage devices 432 (e.g., removable storage 436 and non-removable storage 438) to the basic configuration 402 via the bus/interface controller 430. At least a portion of the operating system 420, applications 422, and data 424 can be stored on removable storage 436 and/or non-removable storage 438, and loaded into system memory 406 via storage interface bus 434 and executed by the one or more processors 404 when the computing device 400 is powered on or the applications 422 are to be executed.

Computing device 400 may also include an interface bus 440 that facilitates communication from various interface devices (e.g., output devices 442, peripheral interfaces 444, and communication devices 446) to the basic configuration 402 via bus/interface controller 430. The example output device 442 includes a graphics processing unit 448 and an audio processing unit 450. They may be configured to facilitate communication with various external devices, such as a display or speakers, via one or more a/V ports 452. Example peripheral interfaces 444 may include a serial interface controller 454 and a parallel interface controller 456, which may be configured to facilitate communications with external devices such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device) or other peripherals (e.g., printer, scanner, etc.) via one or more I/O ports 458. An example communication device 446 may include a network controller 460, which may be arranged to facilitate communications with one or more other computing devices 462 over a network communication link via one or more communication ports 464.

A network communication link may be one example of a communication medium. Communication media may typically be embodied by computer readable instructions, data structures, program modules, and may include any information delivery media, such as carrier waves or other transport mechanisms, in a modulated data signal. A "modulated data signal" may be a signal that has one or more of its data sets or changes that are made in the signal in a manner that encodes information. By way of non-limiting example, communication media may include wired media such as a wired network or private-wired network, and various wireless media such as acoustic, Radio Frequency (RF), microwave, Infrared (IR), or other wireless media. The term computer readable media as used herein may include both storage media and communication media.

In a computing device 400 according to the invention, the application 422 includes a plurality of program instructions that execute the scheduling method for load balancing 500 that may instruct the processor 404 to perform the scheduling method for load balancing 500 of the invention such that the computing device 400 performs the scheduling method for load balancing 500 of the invention.

Fig. 5 shows a flow diagram of a scheduling method 500 for load balancing according to one embodiment of the invention. The scheduling method 500 for load balancing is executed in a computing device (such as the computing device 400) and the scheduling method 500 for load balancing of the present invention is used to solve the performance loss problem caused by too slow balancing in the above-mentioned scheme. As shown in fig. 5, the scheduling method 500 for load balancing may include steps S510 to S530.

In step S510, among the CPUs of all the nodes of the multi-core system, a migration process CPU and one or more migration target CPUs are determined based on at least the CPU load size.

According to one implementation, in step S510, a migration process CPU and one or more migration target CPUs are determined according to the load size of each CPU in all NODEs in the multi-core system. In the embodiment of the invention, the NODE is composed of all CPUs with equal memory distance accessing the same block.

The embodiment of the invention is applied to the situation that the process belongs to a CPU type process, the performance bottleneck of the process is not in a memory, the memory bandwidth is not limited, the process does not depend on file cache, the process is mainly limited by CPU resources, the process is balanced and utilized on the CPU resources, and the process performance is mainly limited by the CPU resources and is not limited by the memory resources. Processes are less memory dependent, but are not memory dependent.

According to one implementation, the step of determining the migrated process CPUs and the one or more migration target CPUs based on at least the CPU load size in step S310 includes: selecting at least part of all the nodes as a first candidate node, and determining the CPU with the maximum load or the CPU with the load larger than or equal to a preset emigration threshold value as an emigration process CPU in CPUs corresponding to the first candidate node; and selecting at least part of all the nodes as second candidate nodes, and determining N CPUs with the minimum load or N CPUs with the load smaller than or equal to a preset immigration threshold value as migration target CPUs in CPUs corresponding to the second candidate nodes, wherein N is a positive integer larger than or equal to 1.

In the embodiment of the present invention, at least one of all nodes is selected as a first candidate node, and at least one of all nodes is selected as a second candidate node, where the first candidate node may include all nodes, or several nodes, or only a node with the highest load among all nodes, and the second candidate node may include all nodes, or several nodes, or only a node with the lowest load among all nodes.

For the first candidate node comprising all nodes and the second candidate node comprising all nodes, the determined migrating process CPUs and the one or more migration target CPUs are: and taking the CPU with the largest load in all the nodes as an migrating process CPU or taking the CPU with the load larger than or equal to a preset migrating threshold value as the migrating process CPU, and determining N CPUs with the smallest load in all the nodes or N CPUs with the load smaller than or equal to the preset migrating threshold value as migrating target CPUs, wherein N is a positive integer larger than or equal to 1.

For the first candidate nodes including all nodes, the second candidate node is a node (also may be several nodes with minimum load or any partial node) in all nodes whose distance from the node where the migrating process CPU is located is less than or equal to the distance threshold, and the determined migrating process CPU and one or more migration target CPUs are: one CPU with the largest load in all nodes is taken as an emigration process CPU or a CPU with the load larger than or equal to a preset emigration threshold value is taken as an emigration process CPU, N CPUs with the smallest load are searched in nodes (which can also be a plurality of nodes with the smallest load or any part of nodes) with the distance smaller than or equal to the distance threshold value from the node where the emigration process CPU is located, and the N CPUs with the smallest load are taken as migration target CPUs or N CPUs with the load smaller than or equal to the preset emigration threshold value and are positive integers larger than or equal to 1.

For the first candidate node including all nodes, the second candidate node including one of the nodes (or any one of the nodes) with the smallest load, the determined migrated process CPUs and the migration target CPUs are: one CPU with the largest load in all nodes is used as an emigration process CPU or a CPU with the load larger than or equal to a preset emigration threshold value is used as an emigration process CPU, N CPUs with the smallest load are searched in one node (or any one node) with the smallest load to be used as migration target CPUs or N CPUs with the load smaller than or equal to the preset emigration threshold value to be used as migration target CPUs, and N is a positive integer larger than or equal to 1.

For a plurality of nodes (or any part of nodes) with the highest load among all the nodes, the first candidate node includes all the nodes, and the determined migrating process CPUs and one or more migration target CPUs are: one CPU with the largest load in a plurality of nodes (or any part of nodes) with the largest load is taken as an emigration process CPU or a CPU with the load larger than or equal to a preset emigration threshold value is taken as an emigration process CPU, N CPUs with the smallest load or N CPUs with the load smaller than or equal to the preset emigration threshold value in all nodes are determined as migration target CPUs, and N is a positive integer larger than or equal to 1.

For a plurality of nodes (or any part of nodes) with the largest load in all nodes, the first candidate node is a node (or several nodes or any part of nodes with the smallest load) with a distance threshold value less than or equal to the distance threshold value from the node where the migrated process CPU is located in all nodes, and the determined migrated process CPU and one or more migration target CPUs are: one CPU with the largest load in a plurality of nodes (which can also be any partial node) with the largest load is taken as an emigration process CPU or a CPU with the load larger than or equal to a preset emigration threshold value is taken as an emigration process CPU, N CPUs with the smallest load are searched in nodes (which can also be a plurality of nodes with the smallest load or any partial node) with the distance smaller than or equal to the distance threshold value from the node where the emigration process CPU is located, and the N CPUs with the smallest load are taken as migration target CPUs or N with the load smaller than or equal to the preset emigration threshold value are taken as migration target CPUs, wherein N is a positive integer larger than or equal to 1.

For a plurality of nodes (or any part of nodes) with the highest load among all the nodes of the first candidate node, the second candidate node includes a node (or any one of the nodes) with the lowest load among all the nodes, and the determined migrating process CPUs and the migration target CPUs are: one CPU with the largest load in a plurality of nodes (or any part of nodes) with the largest load is taken as an emigration process CPU or a CPU with the load larger than or equal to a preset emigration threshold value is taken as an emigration process CPU, N CPUs with the smallest load are searched in one node (or any node) with the smallest load as migration target CPUs or N CPUs with the load smaller than or equal to the preset emigration threshold value are taken as migration target CPUs, and N is a positive integer larger than or equal to 1.

For the first candidate node including one of all nodes with the highest load (or any one of the nodes), the second candidate node including all nodes, the determined migrated process CPUs and one or more migration target CPUs are: one CPU with the largest load in one node (or any one node) with the largest load is taken as an migrating process CPU or a CPU with the load larger than or equal to a preset migrating threshold value is taken as a migrating process CPU, N CPUs with the smallest load or N CPUs with the load smaller than or equal to the preset migrating threshold value in all nodes are determined as migrating target CPUs, and N is a positive integer larger than or equal to 1.

For the first candidate node including one node (may also be any one node) with the largest load among all nodes, the second candidate node being a node (may also be several nodes or any partial node with the smallest load) with a distance threshold value less than or equal to the distance threshold value from the node where the migrated process CPU is located among all nodes, the determined migrated process CPU and one or more migration target CPUs are: one CPU with the largest load in one node (or any one node) with the largest load in all nodes is used as an emigration process CPU or a CPU with the load larger than or equal to a preset emigration threshold value is used as an emigration process CPU, N CPUs with the smallest load are searched from the nodes (or a plurality of nodes with the smallest load or any part of nodes) with the distance smaller than or equal to the distance threshold value from the node where the emigration process CPU is located and used as migration target CPUs or N CPUs with the load smaller than or equal to the preset emigration threshold value are used as migration target CPUs, and N is a positive integer larger than or equal to 1.

For the first candidate node including the node (or any one of the nodes) with the highest load among all the nodes, the second candidate node including the node (or any one of the nodes) with the lowest load among all the nodes, the determined migrating process CPUs and the migration target CPUs are: one CPU with the largest load in one node (or any one node) with the largest load in all nodes is used as an emigration process CPU or a CPU with the load larger than or equal to a preset emigration threshold value is used as an emigration process CPU, N CPUs with the smallest load are searched in one node (or any one node) with the smallest load to be used as migration target CPUs or N CPUs with the load smaller than or equal to the preset emigration threshold value to be used as migration target CPUs, and N is a positive integer larger than or equal to 1.

In the detection process, the number of the CPUs with the detected loads smaller than or equal to the preset immigration threshold value may be less than N or more than N, in the embodiment of the present invention, when the detected number is greater than or equal to N, the N with the smallest loads may be selected, or the first N may be selected, that is, load detection is performed on all CPUs in the second candidate node according to a certain sequence, after the N CPUs meeting the requirements are detected, load detection on subsequent CPUs is stopped, the N migration target CPUs are obtained, load detection on all CPUs in the second candidate node may also be completed, and the N CPUs with the smallest loads are selected from all the obtained CPUs meeting the requirements. When the detected number is smaller than N, the number of actually detected CPUs satisfying the requirement may be used as the standard, or the number of migration target CPUs may be made up to N in the order of the load from small to large.

According to one implementation, the step of determining the migrated process CPUs and the migration target CPUs based on at least the CPU load size in step S310 includes: periodically detecting the load condition of each CPU in all NODEs; or detecting the load condition of each CPU in all nodes when the state of a process changes.

The method for detecting the load condition of each CPU in all the nodes when the state of the process is changed comprises the following steps: and triggering and detecting the load condition of each CPU in all the nodes when a certain process is finished or a new process is started. The state change of the process in the embodiment of the invention may include: when a process is started from executing to finishing or a new process is started, the load condition of each CPU in all the NODEs is triggered to be detected, or the load condition of each CPU in the first candidate NODE and the second candidate NODE is triggered to be detected. In the embodiment of the invention, the process state change not only comprises the starting or the ending of the process, but also comprises the sleep of the process, the awakening of the process or the migration of the process, and the process state change can be the process adding or quitting of a CPU running queue. When the process changes, the load information of each CPU is updated periodically or when the process state changes, and the updated CPU load condition is obtained.

In the embodiment of the present invention, the load condition of each CPU in all NODEs may be periodically detected, for example, the detected period may be 4ms, where the detected period may be set in an actual application environment, and may be modified to 1ms, 3ms, 6ms, 10ms, and the like by configuration modification, and according to experience, if the time is too short, the system scheduling may be too frequent, and if the time is too long, the difference between the loads of the CPUs may be large, and the system performance may be deteriorated, so that the selection is performed according to the actual application environment. When the state of a process of a certain CPU changes, for example, a new process is started or an original process is ended, the load condition of each CPU in all NODEs may be detected or the load condition of each CPU in the first candidate NODE and the second candidate NODE may be detected. And when the detection time is reached, finding out the migrating process CPU and the migrating target CPU respectively.

In the embodiment of the present invention, when finding a migrated process CPU and a migrated target CPU, the first candidate NODE and the second candidate NODE may both be all NODEs, and when finding a migrated process CPU, the busiest CPU may be found in the busiest NODE, or the busiest CPU in all NODEs of the entire NUMA system may be found, and when finding a migrated target CPU, the busiest N CPUs may be found in the idlest one or more NODEs, or the idlest N CPUs in all NODEs of the entire NUMA system may be found, and in some cases, the busiest CPU in the busiest NODE is not the busiest CPU in the entire NUMA system, and the idlest CPU in the idlest NODE is not the idlest CPU in the entire NUMA system. Compared with the two modes, the busy or idle NODE is found firstly, then the busy or idle CPU in the corresponding NODE is found, and the searching is carried out in two stages, so that the load of each CPU is not required to be obtained, the searching time can be saved, and the searching speed is improved; the busiest CPU and the most idle CPU are directly found, and the CPU which needs to be adjusted most can be accurately found at the moment, so that the load balancing is more effective.

The load balancing conditions in the embodiment of the invention are ideally equal in load of each CPU, and in the practical application process, because the load of the CPU changes at any time, the ideal load balancing conditions are difficult to achieve, so that the load balancing conditions in the scheduling process make the difference between the loads of each CPU close to the minimum as much as possible.

In the embodiment of the invention, the CPU currently carrying out load detection can be used as the CPU of the migration process, and if the load detected by the current CPU is greater than or equal to the preset migration threshold value, the CPU of the migration target is determined according to the result of the load detection. In the embodiment of the present invention, the detection cycle of each CPU may be different, and a corresponding load detection cycle is set for each CPU according to the storage space, the execution content requirement, and the like of each CPU, at this time, if the load of the CPU performing load detection is greater than or equal to a preset emigration threshold, the CPU is taken as an emigration process CPU, then N CPUs with the minimum load in all nodes (or a part of nodes or nodes with the minimum load) or N CPUs with the minimum load less than or equal to a preset emigration threshold are searched for as migration target CPUs, or one or more nodes with the minimum load in all nodes are searched for, and N CPUs with the minimum load in the selected nodes are searched for as migration target CPUs. For example, the detection period of the CPU a is 6ms, the detection period of the CPU B is 4ms, the time starts from 0, when the 4 th ms is reached, the CPU B performs load detection, the load of the CPU B is greater than or equal to a preset emigration threshold, the CPU in the emigration process is determined to be the CPU B at this time, then the load conditions of other CPUs except the CPU B are detected, the migration target CPU is determined according to the detected load conditions, and further the process migration is performed according to the determination result. And when the load reaches the 6 th ms, the CPU A carries out load detection, the load of the CPU A is greater than or equal to a preset emigration threshold value, the CPU in the emigration process is determined to be the CPU A, then the load conditions of other CPUs except the CPU A are detected, the migration target CPU is determined according to the detected load conditions, and the process migration is further carried out according to the judgment result. And when the load reaches the 8 th ms, the CPU B performs load detection, and if the load of the CPU B is greater than or equal to the preset emigration threshold value, the load balancing process of the CPU B as the CPU in the emigration process is repeated. When the load reaches 12ms, the CPU A and the CPU B simultaneously carry out load detection, at the moment, the loads of the CPU A and the CPU B are both larger than or equal to a preset migration threshold value, the migration process CPU can be determined to be the CPU A or the CPU B, the CPU A and the CPU B can also be simultaneously used as the migration process CPU, and then migration target CPUs are respectively searched. Or when the load reaches the 12 th ms, the CPU a and the CPU B perform load detection simultaneously, at this time, the CPU a or the CPU B is not determined as the migration process CPU, one CPU with the largest load in all nodes may be determined as the migration process CPU or the CPU with the largest load in the nodes with the largest load in all nodes may be determined as the migration process CPU, the CPU with the load greater than or equal to the preset migration threshold may also be determined as the migration process CPU, and then the migration target CPU is searched. Because each CPU can periodically or in a triggering mode carry out load detection, whether the load of any CPU exceeds a preset migration threshold value or not can be judged, and the process number of the CPU is obtained, the embodiment of the invention can rapidly determine the CPU of the migration process and one or more migration target CPUs. .

In step S520, all processes on the migrated process CPU are traversed, and a process to be migrated is determined at least according to the CPU affinity of each process.

Before the process migration, it is necessary to determine whether the process meets the migration standard according to the CPU affinity of the process, as shown in fig. 4, in step S520, all processes on the CPU of the migrated process are traversed, and the step of determining the process to be migrated according to at least the CPU affinity of each process may include: for each process, determining whether the process allows migration according to the CPU affinity of the process, and determining the migration allowed range of the process under the condition that the process allows migration; when the process is allowed to be migrated and at least one of the one or more migration target CPUs is within the allowed migration range of the process, determining the process as a process to be migrated.

In the embodiment of the invention, aiming at each process, whether the process is allowed to be migrated is determined firstly according to the CPU affinity of the process; when the process allows migration, determining whether the allowed migration range of the process overlaps with the one or more migration target CPUs, and when at least one of the one or more migration target CPUs is within the allowed migration range of the process and the allowed migration range of the process includes at least one of the one or more migration target CPUs, determining the process as the process to be migrated.

In the embodiment of the invention, each process has a CPU _ allowed field for managing the CPU affinity of the process, the CPU affinity of the process refers to which CPUs the process is allowed to run, and the user-mode process can be run by all CPUs under the default condition. If the CPU affinity of the process has requirements, the CPU affinity of a certain process can be generally set in a user mode through a taskset interface, namely, the CPU _ allowed attribute value is changed. Therefore, when a process is migrated, it is necessary to consider whether a migration target CPU is within the CPU _ allowed range of the process, and if not, it indicates that the process is not allowed to run on the migration target CPU, and at this time, the migration target CPU needs to be replaced, and if all the migration target CPUs are not within the migration allowed range of the process, it indicates that no replaceable CPU exists, and at this time, the process is not migrated.

In step S530, a corresponding relationship between the process to be migrated and the migration target CPU is determined according to the CPU affinity of the process to be migrated.

In the embodiment of the present invention, the step of determining the correspondence between the process to be migrated and the migration target CPU according to the CPU affinity of the process to be migrated in step S530 includes: determining the corresponding relation according to the CPU affinity of the processes to be migrated so as to minimize the quantity difference of the processes to be migrated among all the migration target CPUs after migration; or, determining the corresponding relation according to the CPU affinity of the process to be migrated, so that the load difference of the process to be migrated among all migration target CPUs can reach the minimum after migration; or, determining the corresponding relation according to the CPU affinity of the processes to be migrated, so that the sequence of the processes to be migrated on the running queue of the same migration target CPU after migration is consistent with the sequence of the processes to be migrated on the running queue of the CPU of the migrated process; or, determining the corresponding relation according to the CPU affinity of the process to be migrated, so that the closer to the CPU of the migrated process, the more the number of the processes to be migrated corresponding to the migrated target CPU; or, determining the corresponding relation according to the CPU affinity of the process to be migrated, so that the smaller the load, the larger the number of the processes to be migrated corresponding to the migration target CPU.

In the embodiment of the present invention, the number x of the determined processes to be migrated and the number y of the determined migration target CPUs are not related, where x may be greater than y, and x may be equal to y, and x may be less than y, in the embodiment of the present invention, the processes to be migrated may be allocated to all migration target CPUs as much as possible, and due to the CPU affinity requirement of the processes to be migrated, a plurality of processes to be migrated may be allocated or correspond to one migration target CPU, and some migration target CPUs may be allocated or may not correspond to the processes to be migrated, in the embodiment of the present invention, when the correspondence is determined, the processes to be migrated may be allocated or correspond as evenly as possible, for example, the number is allocated or the load is as evenly as possible, and when the correspondence is not possible, the allocation or the correspondence of the processes to be migrated may be performed according to a principle that the sequence between the processes to be migrated on the operation queue of the same migration target CPU after migration is consistent with the sequence of the processes to be migrated on the operation queue of the migration target CPU, so that the process to be migrated meets the first-in first-out sequence after being migrated into the migration target CPU. The process to be migrated may also be allocated or corresponded according to the principle that "the number of the processes to be migrated corresponding to the migration target CPU closer to the migration process CPU is larger", so that the distance between the node where the file cache accessed by the process to be migrated is located and the migration target CPU is as small as possible after the process to be migrated is migrated into the migration target CPU. The distribution or correspondence of the processes to be migrated can also be performed according to the principle that the number of the processes to be migrated corresponding to the migration target CPU with the smaller load is larger, so that the difference between the CPU load of the migration process and the CPU load of the migration target CPU is as small as possible after the processes to be migrated are migrated into the migration target CPU, and load balancing is achieved as possible.

In the embodiment of the present invention, when determining the correspondence between the process to be migrated and the migration target CPU according to the CPU affinity of the process to be migrated in step S530, each process to be migrated needs to meet the requirement of the allowable migration range thereof, and the processes to be migrated are distributed to each migration target CPU as evenly as possible. If the number or the load of the processes to be migrated cannot be averaged to each migration target CPU, the processes are distributed as far as possible according to a first-in first-out sequence, or the number or the load of the processes is distributed more when the distance between the migration target CPU and the migration process CPU is smaller, or the number or the load of the processes is distributed more when the load of the migration target CPU is smaller, according to the sequence from the small to the large of the load of the migration target CPU. The finally determined corresponding relation between the process to be migrated and the migration target CPU in the embodiment of the invention may meet the above requirement, and may also meet the above requirements.

The embodiment of the invention can determine the corresponding relation between the process to be migrated and the migration target CPU by adopting the following modes:

and in the first mode, the CPU affinity requirement of the process is met, and the processes to be migrated are averagely corresponding to each migration target CPU as much as possible according to the number.

And secondly, meeting the CPU affinity requirement of the process, and enabling the process to be migrated to correspond to each migration target CPU as averagely as possible according to the load.

And thirdly, meeting the CPU affinity requirement of the process, and corresponding the processes to be migrated to each migration target CPU one by one according to a first-in first-out sequence.

And fourthly, meeting the CPU affinity requirement of the process, and corresponding the processes to be migrated to each migration target CPU one by one according to the sequence from small to large of the distance between the migration target CPU and the migrated process CPU.

And fifthly, meeting the CPU affinity requirement of the process, and corresponding the processes to be migrated to each migration target CPU one by one according to the sequence of the load of the migration target CPUs from small to large.

The corresponding manner of the process to be migrated and the migration target CPU in the first manner is described with reference to fig. 1, where P0 is running on CPU 0 of NODE 0 and processes P1-P15 waiting to run are running, the rest of CPUs on NODE 0 are idle, all CPUs on NODE 15 are idle, CPU 0 of NODE 0 is taken as the migration process CPU, the rest of CPUs of NODE 0 and all CPUs on NODE 15 are taken as the migration target CPUs, if the CPU affinities of P0-P15 are all default NODEs, P0 (running and not migrating) is retained in CPU 0 of NODE 0 according to the corresponding manner of the first manner, and the rest of P1-P15 are allocated to the idle CPUs (migration target CPUs) on NODE 0 and NODE 15 in number average, each CPU corresponds to 1 process, as shown in fig. 3. If the permitted migration range of P4 is NODE 0, P15 does not permit migration, P0 (running and not performing migration) and P15 are reserved in CPU 0 of NODE 0, P4 corresponds to any idle CPU (for example, CPU 2) of NODE 0, the rest P1-P3 and P5-P14 are distributed to idle CPUs (migration target CPUs) on NODE 0 and NODE 15 according to the number average, and one CPU idle occurs after the correspondence, and the CPU idle (for example, CPU 7) on NODE 15 (farther from the migrated process CPU) is generally the case. For the requirement that more processes have CPU affinity, in a similar way, the CPU affinity requirements of the processes are met, and meanwhile, the migration target CPUs are distributed equally according to the number.

The corresponding mode of the process to be migrated and the migration target CPU in the second mode is described with reference to fig. 1, where P0 and the process to be executed P1-P15 are running on CPU 0 of NODE 0, the rest of CPUs on NODE 0 are idle, all CPUs on NODE 15 are idle, CPU 0 of NODE 0 is taken as a migration process CPU, the rest of CPUs of NODE 0 and all CPUs on NODE 15 are taken as migration target CPUs, the load of P0 is 160, the loads of P1-P15 are 10, 20 and 30, respectively, the load is 150, if the allowed migration range of P1-P15 is NODE 0, P0 (running and not migrating) and P1(P0 with the largest load, P1 with the smallest load and the load being the average as much as possible) are retained on CPU 0 of NODE in the second mode, P2 and P15 are corresponding to CPU 1 of NODE 0, and P3, P14 to CPU 582 of NODE 0 are corresponding to NODE 0, p4 and P13 correspond to CPU 3 of NODE 0, P5 and P12 correspond to CPU 4 of NODE 0, P6 and P11 correspond to CPU 5 of NODE 0, P7 and P10 correspond to CPU 6 of NODE 0, and P8 and P9 correspond to CPU 7 of NODE 0. If the migration is not allowed by P1-P7, the CPU affinities of P8-P15 are all default NODEs, P0-P7 is reserved in CPU 0 of NODE 0, the rest P8-P15 is evenly distributed to idle CPUs (migration target CPUs) on NODE 0 and NODE 15 according to loads, and multiple CPU idleness occurs after correspondence, and the CPU idleness on NODE 15 (farther from the CPU of the migration process) is generally the case. If more processes have the requirement of CPU affinity, the P1-P15 distribute the migration target CPUs of the processes to be migrated to the greatest extent according to the load average while meeting the requirement of CPU affinity of the processes.

Explaining the corresponding manner of the processes to be migrated and the migration target CPUs in the third manner with reference to fig. 1, P0 and the processes to be waited to run P1-P15 are running on the CPU 0 of NODE 0, the remaining CPUs on NODE 0 are idle, all the CPUs on NODE 15 are idle, the CPU 0 of NODE 0 is taken as the migration process CPU, the remaining CPUs of NODE 0 and all the CPUs on NODE 15 are taken as the migration target CPUs, if the allowable migration range of P1-P15 is NODE 0, in the third manner, P0 (running without migration) and P1 are retained in the CPU 0 of NODE 0 and P0 is arranged before P1, P2, P15 are corresponding to the CPU 1 of NODE 0 and P2 is arranged before P15, P3, P14 are corresponding to the CPU 2 of NODE 0 and P3 is arranged before P14, P4, P13 is corresponding to NODE 460 and P583 is arranged before P4, P5 is arranged before P4, P12 corresponds to CPU 4 of NODE 0 and P5 is ranked before P12, P6, P11 correspond to CPU 5 of NODE 0 and P6 is ranked before P11, P7, P10 correspond to CPU 6 of NODE 0 and P7 is ranked before P10, P8, P9 correspond to CPU 7 of NODE 0 and P8 is ranked before P9. If more processes have the CPU affinity requirement, the P1-P15 distribute the process to be migrated to each migration target CPU in the order of first-in first-out as much as possible while meeting the CPU affinity requirement of the process.

Explaining the corresponding modes of the processes to be migrated and the migration target CPUs in the fourth mode in combination with fig. 1, wherein P0 and the processes waiting to run P1-P15 are running on CPU 0 of NODE 0, CPUs 1-3 on NODE 0 are idle, all CPUs on NODE 15 are idle, CPU 0 of NODE 0 is taken as a migration process CPU, CPUs 1-3 of NODE 0 and all CPUs on NODE 15 are taken as migration target CPUs, if the CPU affinities of P0-P15 are default all NODEs, P0 (running and not migrating) and P12 are retained in CPU 0 of NODE 0 according to the fourth mode, P1 and P13 are corresponding to CPU 1 of NODE 0, P2 and P14 are corresponding to CPU 2 of NODE 0, P3 and P15 are corresponding to CPU 3 of NODE 0, P4 is corresponding to CPU 0 of NODE 15, P5 is corresponding to CPU 1 of NODE 3615, P6 is corresponding to CPU 2 of NODE 15, p7 corresponds to CPU 3 of NODE 15, P8 corresponds to CPU 4 of NODE 15, P9 corresponds to CPU 5 of NODE 15, P10 corresponds to CPU 6 of NODE 15, P11 corresponds to CPU 7 of NODE 15, processes P1-P15 are distributed to CPUs 1-3 of NODE 0 and idle CPUs (migration target CPUs) on the NODE 15 one by one according to the corresponding mode of the fourth mode, each CPU is distributed with one process, at the moment, the remaining 4 processes P12-P15 do not correspond, and the remaining 4 processes P12-P15 are preferentially distributed on the NODE 0 because the distance between the CPU on the NODE 0 and the CPU on the NODE 15 is smaller than that between the CPU on the NODE 15 and the CPU of the migration process CPU. For the requirement that more processes have CPU affinity, according to a similar mode, the CPU affinity requirements of the processes are met, and meanwhile, the migration target CPUs are distributed as far as possible according to the sequence from small to large of the distance between the migration process CPUs.

Explaining a corresponding mode of a process to be migrated and a migration target CPU in a mode five with reference to fig. 1, where P0 is running on CPU 0 of NODE 0 and processes P1-P15 waiting to run, all CPUs 1-3 and 15 on NODE 0 are CPUs smaller than a preset migration threshold, CPU 0 of NODE 0 is taken as a migration process CPU, CPUs 1-3 and 1-3 of NODE 0 and 15 on NODE 15 are taken as migration target CPUs, if the CPU affinities of P0-P15 are all default NODEs, where the CPU 0 load of NODE 0 is 160, the loads of the original processes on CPUs 1-3 and 1-7 of NODE 0 and 15 are 110, 100 and 90, respectively, and P0 (running and not migrating) is retained on CPU 0 of NODE 0 in the mode five corresponding mode, and P1 is corresponding to CPU 1 of NODE 0, corresponding P2 to CPU 2 of NODE 0, P3 to CPU 3 of NODE 0, P4 to CPU 0 of NODE 15, P5 to CPU 1 of NODE 15, P6 to CPU 2 of NODE 15, P7 to CPU 3 of NODE 15, P8 and P15 to CPU 4 of NODE 15, P9 and P14 to CPU 5 of NODE 15, P10 and P13 to CPU 6 of NODE 15, P11 and P12 to CPU 7 of NODE 15, in the corresponding manner of the fifth manner, the processes P1-P15 are allocated to the CPUs 1-3 of NODE 0 and the idle CPUs (migration target CPUs) on NODE 15 one by one, each CPU is allocated one process, and at this time, the remaining 4 processes P12-P15 do not correspond, since the loads of the CPUs 4-7 on NODE 15 are the smallest, the remaining 4 processes P12-P15 are preferentially allocated on CPUs 4-7 of NODE 15. For the requirement that more processes have CPU affinity, in a similar way, the CPU affinity requirements of the processes are met, and meanwhile, the migration target CPUs are distributed as much as possible according to the order from small to large of the load of the migration target CPUs.

The mode in the embodiment of the invention can be mainly a certain mode, and the rest modes are used as assistance, or the combination of the modes.

In step S540, the process to be migrated is migrated to the corresponding migration target CPU according to the correspondence.

According to the migration target CPU and the process to be migrated determined in steps S510 to S530 in the embodiment of the present invention, the process to be migrated is migrated to the corresponding migration target CPU according to the determined correspondence, and added to the migration target CPU running queue.

As shown in fig. 6, an embodiment of the present invention further provides a scheduling apparatus for load balancing, including: a lookup module 610, a traversal module 620, a matching module 630, and a migration module 640.

The searching module 610 is adapted to determine, among CPUs of all nodes of the multi-core system, a migration process CPU and one or more migration target CPUs based on at least a CPU load size; a traversal module 620, adapted to traverse all processes on the migrated process CPU, and determine a process to be migrated at least according to the CPU affinity of each process; a matching module 630, adapted to determine a corresponding relationship between the process to be migrated and the migration target CPU according to the CPU affinity of the process to be migrated; and the migration module 640 is adapted to migrate the process to be migrated to the corresponding migration target CPU according to the correspondence.

According to one implementation, the lookup module 610 determines, based at least on the CPU load size, that the migrating process CPU and the one or more migration target CPUs are adapted to: selecting at least part of all the nodes as a first candidate node, and determining the CPU with the maximum load or the CPU with the load larger than or equal to a preset emigration threshold value as an emigration process CPU in CPUs corresponding to the first candidate node; and selecting at least part of all the nodes as second candidate nodes, and determining N CPUs with the minimum load or N CPUs with the load smaller than or equal to a preset immigration threshold value as migration target CPUs in CPUs corresponding to the second candidate nodes, wherein N is a positive integer larger than or equal to 1.

According to one implementation, the first candidate node is a node with the highest load among the total nodes; the second candidate node is a node which is less than or equal to a distance threshold value from the node where the migrating process CPU is located in all the nodes, or one or more nodes with the minimum load.

Wherein the determining, by the lookup module 610, the migrated process CPU and the one or more migration target CPUs based at least on the CPU load size comprises: periodically detecting the load condition of each CPU in all the nodes; or detecting the load condition of each CPU in all nodes when the state of a process changes.

According to one implementation, the traversal module 620 traverses all processes on the migrated process CPU, and determines that the process to be migrated is suitable for, at least according to the CPU affinity of each process: for each process, determining whether the process allows migration according to the CPU affinity of the process, and determining the migration allowed range of the process under the condition that the process allows migration; when the process is allowed to be migrated and at least one of the one or more migration target CPUs is within the allowed migration range of the process, determining the process as a process to be migrated.

According to one implementation, the matching module 630 determines, according to the CPU affinity of the process to be migrated, that the correspondence between the process to be migrated and the migration target CPU is suitable for: determining the corresponding relation according to the CPU affinity of the processes to be migrated so as to minimize the quantity difference of the processes to be migrated among all the migration target CPUs after migration; or

Determining the corresponding relation according to the CPU affinity of the process to be migrated so as to enable the load difference of the process to be migrated among all migration target CPUs to be minimum after migration; or

Determining the corresponding relation according to the CPU affinity of the processes to be migrated, so that the sequence of the processes to be migrated on the running queue of the same migration target CPU after migration is consistent with the sequence of the processes to be migrated on the running queue of the CPU of the migrated process; or

Determining the corresponding relation according to the CPU affinity of the process to be migrated, so that the closer the CPU of the migrated process is, the greater the number of the processes to be migrated corresponding to the migrated target CPU is; or

And determining the corresponding relation according to the CPU affinity of the process to be migrated, so that the smaller the load, the larger the number of the processes to be migrated corresponding to the migration target CPU.

In the embodiment of the present invention, load balancing is implemented by once migration during a migration process, which may specifically be implemented by the following functions:

the method comprises the steps of obtaining a main interface of a rapid balancing scheme by utilizing int fast _ load _ balance (struct rq this _ rq), calling in a tick periodic processing function of each CPU, mainly judging whether a current CPU reaches a migratable load and task number threshold, directly obtaining related load information of the CPU because the load can be periodically updated, and then migrating a part of processes from the current CPU to a selected part of CPUs.

Whether rapid balancing can be performed on the current CPU is judged by using Bool can _ fast _ balance (struct rq is _ rq), whether the rapid balancing can be performed on the current CPU is judged according to the load and process quantity information of the current CPU, and if the load and the process quantity information exceed a set migration threshold value, a subsequent migration flow is performed when a condition is reached.

And searching a specified number of idle or low-load CPUs in the system by using Void find _ end _ CPU (int × connected _ CPU, int task _ nr), sequentially polling CPUs in the NODE from near to far according to the NODE distance through for _ reach _ NODE, adding the CPUs into the input parameters connected _ CPU if the load is lower than a set threshold value, and determining the number of migration CPUs to be searched according to the number of tasks to be migrated task _ nr.

stream is a typical tool for testing memory performance, and mainly performs operations such as copying and calculation of a plurality of arrays, when a multi-thread test is started, a test program can continuously create threads with the same number as that of CPUs for calculation, and at this time, the speed of load balancing can seriously affect the test performance. Part of the stream codes are as follows:

1. array definition

static double a[N+OFFSET]，

b[N+OFFSET]，

c[N+OFFSET]；

2. Array initialization

/*Get initial valuefor system clock.*/

#pragma omp parallel for

for(j＝0；j＜N；j++){

a[j]＝1.0；

b[j]＝2.0；

c[j]＝0.0；

}

3. Array multithreading copy operation using omp library

The program is mainly used for carrying out multithreading CPU operation, a main process can instantly create a large number of threads to carry out large array operation together, and on a server multi-NODE architecture such as a 128-core, a large number of threads exist on the NODE of the main process and need to be migrated to other NODE depending on the following load balance, so that the large array operation speed of the program can be influenced by the load balance speed.

Copy test data using different protocols are as follows:

	original plan	This scheme
			Score of	60000M/S	100000M/S

The above table shows that the scheme can significantly improve the performance of multi-thread operation.

Embodiments also provide a computing device, referring to fig. 7, comprising a memory 1120, a processor 1110 and a computer program stored in said memory 1120 and executable by said processor 1110, the computer program being stored in a space 1130 for program code in the memory 1120, the computer program, when executed by the processor 1110, implementing the method steps 1131 for performing any of the methods according to the invention.

The embodiment of the application also provides a computer readable storage medium. Referring to fig. 8, the computer readable storage medium comprises a storage unit for program code provided with a program 1131' for performing the steps of the method according to the invention, which program is executed by a processor.

The various techniques described herein may be implemented in connection with hardware or software or, alternatively, with a combination of both. Thus, the methods and apparatus of the present invention, or certain aspects or portions thereof, may take the form of program code (i.e., instructions) embodied in tangible media, such as removable hard drives, U.S. disks, floppy disks, CD-ROMs, or any other machine-readable storage medium, wherein, when the program is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the invention.

In the case of program code execution on programmable computers, the computing device will generally include a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device. Wherein the memory is configured to store program code; the processor is configured to execute the scheduling method for load balancing of the present invention according to instructions in the program code stored in the memory.

By way of example, and not limitation, readable media may comprise readable storage media and communication media. Readable storage media store information such as computer readable instructions, data structures, program modules or other data. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. Combinations of any of the above are also included within the scope of readable media.

In the description provided herein, algorithms and displays are not inherently related to any particular computer, virtual system, or other apparatus. Various general purpose systems may also be used with examples of this invention. The required structure for constructing such a system will be apparent from the description above. Moreover, the present invention is not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present invention as described herein, and any descriptions of specific languages are provided above to disclose preferred embodiments of the invention.

In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim.

Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in a device as described in this embodiment or alternatively may be located in one or more devices different from the devices in this example. The modules in the foregoing examples may be combined into one module or may be further divided into multiple sub-modules.

Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments.

Furthermore, some of the described embodiments are described herein as a method or combination of method elements that can be performed by a processor of a computer system or by other means of performing the described functions. A processor having the necessary instructions for carrying out the method or method elements thus forms a means for carrying out the method or method elements. Further, the elements of the apparatus embodiments described herein are examples of the following apparatus: the apparatus is used to implement the functions performed by the elements for the purpose of carrying out the invention.

As used herein, unless otherwise specified the use of the ordinal adjectives "first", "second", "third", etc., to describe a common object, merely indicate that different instances of like objects are being referred to, and are not intended to imply that the objects so described must be in a given sequence, either temporally, spatially, in ranking, or in any other manner.

While the invention has been described with respect to a limited number of embodiments, those skilled in the art, having benefit of this description, will appreciate that other embodiments can be devised which do not depart from the scope of the invention as described herein. Furthermore, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter.

Claims

1. A scheduling method for load balancing, comprising the steps of:

determining a migrating process CPU and one or more migration target CPUs in CPUs of all nodes of the multi-core system at least based on the load of the CPUs;

traversing all processes on the CPU of the migrated process, and determining the process to be migrated at least according to the CPU affinity of each process;

determining the corresponding relation between the process to be migrated and the migration target CPU according to the CPU affinity of the process to be migrated; and

and migrating the process to be migrated to the corresponding migration target CPU according to the corresponding relation.

2. The scheduling method of claim 1, wherein said determining a migrating process CPU and one or more migration target CPUs based at least on CPU load size comprises:

selecting at least part of all the nodes as a first candidate node, and determining the CPU with the maximum load or the CPU with the load larger than or equal to a preset emigration threshold value as an emigration process CPU in CPUs corresponding to the first candidate node;

and selecting at least part of all the nodes as second candidate nodes, and determining N CPUs with the minimum load or N CPUs with the load smaller than or equal to a preset immigration threshold value as migration target CPUs in CPUs corresponding to the second candidate nodes, wherein N is a positive integer larger than or equal to 1.

3. The scheduling method of claim 2, wherein the first candidate node is a most loaded node among the entire nodes; the second candidate node is a node which is less than or equal to a distance threshold value from the node where the migrating process CPU is located in all the nodes, or one or more nodes with the minimum load.

4. The scheduling method of claim 2 or 3, wherein prior to said step of determining a migrated process CPU and one or more migrated target CPUs based at least on CPU load size, further comprising:

periodically detecting the load condition of each CPU in all the nodes; or

And detecting the load condition of each CPU in all the nodes when the state of the process changes.

5. The scheduling method of claim 1, wherein traversing all processes on the migrating process CPU, determining a process to be migrated based at least on the CPU affinity of each process comprises:

for each process, determining whether the process allows migration according to the CPU affinity of the process, and determining the migration allowed range of the process under the condition that the process allows migration;

when the process is allowed to be migrated and at least one of the one or more migration target CPUs is within the allowed migration range of the process, determining the process as a process to be migrated.

6. The scheduling method according to claim 1 or 5, wherein the step of determining the correspondence between the process to be migrated and the migration target CPU according to the CPU affinity of the process to be migrated includes:

determining the corresponding relation according to the CPU affinity of the processes to be migrated so as to minimize the quantity difference of the processes to be migrated among all the migration target CPUs after migration; or

7. A scheduling apparatus for load balancing, comprising:

the searching module is suitable for determining a migrating process CPU and one or more migrating target CPUs in CPUs of all nodes of the multi-core system at least based on the load of the CPUs;

the traversal module is suitable for traversing all processes on the CPU of the migrated process and determining the process to be migrated at least according to the CPU affinity of each process;

the matching module is suitable for determining the corresponding relation between the process to be migrated and the migration target CPU according to the CPU affinity of the process to be migrated; and

and the migration module is suitable for migrating the process to be migrated to the corresponding migration target CPU according to the corresponding relation.

8. The scheduling apparatus of claim 7, wherein the lookup module determines, based at least on the CPU load size, the migrating process CPUs and the one or more migration target CPUs are adapted to:

9. A computing device, comprising:

at least one processor and a memory storing program instructions;

the program instructions, when read and executed by the processor, cause the computing device to perform the scheduling method for load balancing of any one of claims 1-6.

10. A readable storage medium storing program instructions which, when read and executed by a computing device, cause the computing device to perform the scheduling method for load balancing according to any one of claims 1 to 6.