WO2021022964A1

WO2021022964A1 - Task processing method, device, and computer-readable storage medium based on multi-core system

Info

Publication number: WO2021022964A1
Application number: PCT/CN2020/100834
Authority: WO
Inventors: 张凯
Original assignee: 中兴通讯股份有限公司
Priority date: 2019-08-02
Filing date: 2020-07-08
Publication date: 2021-02-11
Also published as: CN112306669A

Abstract

Provided are a task processing method, device, and computer-readable storage medium based on a multi-core system, said method comprising: dividing a CPU core into a service core set domain and a control core set domain; binding a service processing task to the service core in the service core set domain, said service core processing task being processed by said service core.

Description

Task processing method, device and computer readable storage medium based on multi-core system

Cross references to related applications

This application is based on a Chinese patent application with an application number of 201910714824.8 and an application date of August 2, 2019, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby incorporated into this application by reference.

Technical field

The present invention relates to the field of computer technology, and in particular to a task processing method, device and computer-readable storage medium based on a multi-core system.

Background technique

With the development of multi-core CPU systems, especially in the embedded field, in addition to basic functions such as system management and control, the system has an increasing demand for simultaneous completion of high real-time, large data volume business processing tasks. For such complex systems with different processing tasks, it is particularly important how multi-core CPUs coordinate with each other to prevent system jitter and interference to improve business processing performance.

Summary of the invention

The present invention aims to solve one of the technical problems in the related art at least to a certain extent. To this end, the present invention provides a task processing method, device and computer-readable storage medium based on a multi-core system.

The task processing method based on the multi-core system according to the embodiment of the present invention includes: dividing a CPU core into a service core collection domain and a control core collection domain; binding service processing tasks to the service cores in the service core collection domain, The business processing task is processed by the business core.

The task processing device based on the multi-core system according to the embodiment of the present invention includes: a dividing module for dividing a CPU core into a service core collection domain and a control core collection domain; a task processing module for binding service processing tasks to all On the business core in the business core collection domain, the business core processes the business processing tasks.

According to the electronic device of the embodiment of the present invention, the electronic device includes: a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and when the computer program is executed by the processor Implement the steps of the task processing method based on the multi-core system as described above.

According to the computer storage medium of the embodiment of the present invention, a computer program is stored on the computer storage medium, and when the computer program is executed by a processor, the steps of the task processing method based on the multi-core system as described above are realized.

Description of the drawings

Figure 1 is a schematic diagram of the comparison between the existing symmetric pair processing system and the asymmetric multiprocessing system;

2 is a schematic structural diagram of a task processing device based on a multi-core system according to an embodiment of the present invention;

3 is a flowchart of a task processing method based on a multi-core system according to an embodiment of the present invention;

4 is a flowchart of a method for dividing a service core set domain and a control core set domain according to an embodiment of the present invention;

Fig. 5 is a schematic diagram of a logic state transition of a CPU core according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of a system call proxy using a shared memory layout according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of a system call agent flow diagram of a service core and a control core according to an embodiment of the present invention;

8 is a schematic diagram of the flow of an integrated system for management and forwarding of communication equipment according to an embodiment of the present invention;

Fig. 9 is a schematic structural diagram of a task processing device based on a multi-core system according to an embodiment of the present invention.

Detailed ways

In order to further explain the technical means and effects of the present invention to achieve the predetermined purpose, the present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments.

In traditional methods, for processing different tasks, there are two solutions: Asymmetric Multi-Processing (AMP) and Symmetrical Multi-Processing (SMP). Their characteristics are shown in Figure 1.

Among them, in the AMP solution, each CPU core handles different tasks, or even runs a system separately, and the CPU cores will not affect each other. However, this kind of system has high complexity and difficult business program development and transplantation. Rarely applied in.

The SMP method is widely used, and its characteristic is that each CPU core shares processing and runs a single copy of the operating system. Tasks are fairly scheduled and load balanced among multiple CPU cores.

In a symmetric multi-processing system, in order to complete business processing tasks with high real-time performance and large data volume, the method adopted in related technologies is to reduce system jitter through system optimization to meet processing performance. The basic working methods and principles are described below:

First, shut down unnecessary system service processes, further, migrate ordinary tasks running in the business core to other CPU cores, and further, migrate all interrupts to other non-business cores, and further, bind business processing tasks Scheduled to the business CPU core for execution. Through these operating procedures, the time to execute the business processing program on the business CPU core is as unaffected as possible.

The main disadvantages of the above methods are: because of the inherent symmetrical processing characteristics of the SMP solution, it cannot completely solve the system jitter problem, especially in the application scenario where the CPU service core is dynamically isolated after the CPU core is started for service binding. The optimization method cannot be completely isolated. The business processing tasks are not only vulnerable to the task processes, timers, and interrupt lower-half programs that have been bound to the CPU core before the CPU core where they are located, but also unnecessary system clocks. The extra overhead caused by interrupt scheduling (traditional dynamic clock TICKLESS can only be reduced), and system calls are blocked due to competition, business processing tasks cannot occupy 100% of the CPU cores running business processing, resulting in business processing tasks that cannot meet business processing performance , And even easily due to scheduling strategy and scheduling priority issues, reverse blocking and affecting the normal operation of management and control functions.

As shown in FIG. 3, a task processing method based on a multi-core system according to an embodiment of the present invention includes:

S101: Divide the CPU core into a service core collection domain and a control core collection domain;

As a result, different task processing can be performed on the CPU cores of different collection domains, thereby improving service processing efficiency. For example, as shown in FIG. 2, in a symmetric pair processing system, the control core in the control core set domain may be a CPU core used for management control; the service core may be a CPU core used for service processing. By placing the control core and the business core in different scheduling sets, the control core and the business core can be isolated from each other without affecting each other, and an asymmetric multiprocessing system in business logic can be realized.

S102: Bind the business processing task to the business core in the business core collection domain, and the business core processes the business processing task.

According to the task processing method based on the multi-core system of the embodiment of the present invention, by dividing the CPU and the service core collection domain and the control core collection domain, the service processing tasks are bound to the service cores, and the service cores process the service processing tasks. In a symmetric multi-processing system, it can dynamically and smoothly switch to the asymmetric multi-processing state in business logic, so that business processing tasks completely monopolize the operation of the business core, and will not cause bottlenecks due to system jitter, which will affect business performance and indicators. It can make full use of the single-core CPU processing capacity of the business core to improve business processing performance.

According to some embodiments of the present invention, as shown in FIG. 4, dividing the CPU core into a service core collection domain and a control core collection domain includes:

S201: Make all CPU cores online;

S202: Select some CPU cores from all CPU cores as control cores to form a control core set domain;

S203: Use the remaining CPU cores as service cores, and process the service cores to go offline and then go online to form a service core set domain.

It should be noted that traditionally, the logic state of the CPU core has two types: CPU_ONLINE and CPU_OFFLINE. As shown in Figure 5, the present invention proposes a new CPU core logic state CPU_ONLINE_EXCLUSIVE. Using CPU hot plug technology, the service core can switch from the traditional state to the third logical state CPU_ONLINE_EXCLUSIVE by going offline and then online, as shown in the figure As shown in 5, after the introduction of this new state, the operating system divides the business core into a completely different scheduling set domain from the control core, independently scheduling, avoiding mutual influence, and achieving asymmetric processing effects in business logic.

When dividing the service core collection domain and the control core collection domain, firstly, all CPU cores can be online, and then, by selecting some CPU cores as service cores, it is formed by the processing method of offline and then online. Business core collection domain.

In some embodiments of the present invention, the method may further include:

When all CPU cores are online, tasks are distributed to all CPU cores for processing.

When the business core is offline, the tasks in the business core are migrated to the control core.

According to some embodiments of the present invention, the method further includes: before the service core goes online again, turning on the clock interrupt of the service core, and sending an online notification to the operating system.

It should be noted that when all programs or services running on the business core are migrated to the control core, the operating environment of the business core has been cleaned up. Before the business core goes online again, the clock interrupt of the corresponding business core can be turned on and the CPU is sent Prepare to notify the operating system kernel of the business launch.

In some embodiments of the present invention, the method further includes: after binding the service processing task to the service core in the service core set domain, turning off the clock interruption of the service core. Therefore, by turning off the clock interrupt for scheduling on the service core, it can avoid interrupting the service processing task, thereby further improving the service processing performance.

According to some embodiments of the present invention, the method may further include:

Run the system call agent task on the control core;

By sharing memory, the business core makes system calls based on the running results of the system call agent task running on the control core.

It should be noted that when the clock interruption on the service core is turned off, the service processing task directly enters the system call, which will be blocked by resource competition such as semaphores and spin locks. Therefore, system calls should be avoided as much as possible for business processing tasks, but certain scenarios are still unavoidable, such as obtaining time, reading and writing files, etc. In order to solve this problem, as shown in Figure 6 and Figure 7, the present invention proposes a method of system call proxy. The system call proxy task is run on the control core in advance, and the system call is not directly called on the business core, but through The shared memory method informs the agent that the task is completed indirectly, avoiding the system performance loss caused by system blockage and context switching of the business core.

Among them, by sharing the memory queue, the business core makes system calls according to the system call agent tasks, including:

Allocate shared memory, which includes system call descriptors. As shown in Figure 6, you can pre-allocate two shared memory that can be accessed by the control core and the business core. One of them is used to represent the set of system call descriptors. Each descriptor contains fields such as status, system call number, parameters, and results. , Each description character is equal in length, continuous in the address space, organized in a ring.

Write the system call parameter information when the business core makes a system call to the descriptor, and modify the status of the descriptor to request. When the service core makes a system call, it first obtains the position of the write pointer in the description ring. After legal judgment, if the parameter requires a large block of memory, first apply for a block of memory from the parameter memory pool, and fill in the system call number and the parameter or parameter memory address. In the descriptor, further, the status of the descriptor is changed to requesting, the loop waits for the status of the descriptor to be changed to the completion of execution, the result is retrieved, and the parameter memory pool is released.

The control core searches for the descriptor whose status is in the request, and executes the system call based on the system call parameter information in the descriptor. As shown in Figure 7, the control core starts the system call proxy task and sets it to a higher priority. This task continuously polls the status of the descriptor at the position of the read pointer in the descriptor ring until the query finds that the status is changed to the request. , First modify the status to executing, and then take out the system call number and parameters to execute the system call. After the analysis and execution, if the parameter needs a large block of memory to save the result, apply from the parameter memory pool and copy the result to the parameter memory. Further, the modification status is execution complete. The read pointer is incremented by 1, and the next descriptor is continued.

As shown in Figure 4 and Figure 5, the specific steps of dividing the CPU core into a control core and a service core include:

Step 1: After the CPU is initialized, the CPU normally enters the CPU_ONLINE logic state. In this state, the CPU checks all tasks in parallel and symmetric processing, fair scheduling, and load balancing;

Step 2: On the basis of the above steps, send the service core ready offline notification to the operating system core, and migrate all programs or services running in the service core to the control core, including management control tasks, timers, work queues, interrupts, and Interrupt the lower half of the program. When the migration operation is completed, the CPU logic state of the service core changes to the CPU_OFFLINE state. In this state, the service core is powered off and offline.

Step 3: On the basis of the above steps, the operating environment of the business core has been cleaned up. Further, turn on the clock interrupt of the corresponding business core, send the CPU ready to go online notification to the operating system kernel, wake up the business core and send the CPU of the business core The logical state state transition CPU_ONLINE_EXCLUSIVE, in this state, the business core is in an independent scheduling set domain, the process scheduler starts execution in a completely different way from the control core, the run queue is empty, no task is performed, and no work is performed Queues, timers, etc. interrupt the lower half of the program.

Step 4: On the basis of the above steps, the clock interrupt of the service core has been turned on, and its scheduler starts to run. At this time, the service processing task is started and bound to the corresponding service core to run.

Through the above steps, the service core is in the CPU_ONLINE_EXCLUSIVE logic state set, and the control core is in the CPU_ONLINE logic state set. The two enter different scheduling set domains. The control core runs ordinary management control, work queues, timers, interrupts, and the second half of interrupts. Department and other tasks, the business core runs business processing tasks separately, and different tasks will not be load balanced between the control core and the business core: the business core will not actively pull up the tasks of the control core task queue, and the control core scheduler cannot The task is put into the task scheduling queue of the business core.

Step 5. Optionally, on the basis of step 4, the business processing task is already running. In order to further improve the business processing performance, choose to turn off the clock interrupt for scheduling on the business core to avoid interrupting the business processing task.

Step 6: Optionally, on the basis of step 4, run the system call proxy task on the control core in advance. The business processing task running on the business core does not directly call the system call, but informs through the shared memory queue The agent task is completed indirectly to avoid system performance loss caused by system blockage and context switching.

The task processing device based on the multi-core system according to the embodiment of the present invention includes: a division module and a task processing module.

Specifically, the dividing module may be used to divide the CPU core into a service core set domain and a control core set domain.

The task processing module is used to bind the business processing tasks to the business cores in the business core collection domain, and the business cores process the business processing tasks.

According to the task processing device based on the multi-core system of the embodiment of the present invention, in the symmetric multi-processing system, it can dynamically and smoothly switch to the asymmetric multi-processing state in business logic, so that the business processing task completely monopolizes the operation of the business core. The bottleneck caused by system jitter affects business performance and indicators.

According to some embodiments of the present invention, the division module is specifically used for:

Make all CPU cores online;

Select some CPU cores from all CPU cores as control cores to form a control core set domain;

The remaining CPU cores are used as business cores, and the business cores are processed offline and then online to form a business core set domain.

It should be noted that using CPU hot-plug technology, the service core can switch from the traditional state to the third logical state CPU_ONLINE_EXCLUSIVE by offline and then online, as shown in Figure 5. After introducing this new state, the operating system will The core is divided into a completely different scheduling set domain from the control core, independent scheduling, avoiding mutual influence, and achieving asymmetric processing effects in business logic.

In some embodiments of the present invention, the task processing module is also used to:

When all CPU cores are online, assign tasks to all CPU cores for processing;

After the business core is offline, the tasks in the business core are migrated to the control core.

According to some embodiments of the present invention, the device further includes: a control module. The control module is used to turn on the clock interrupt of the service core before the service core goes online again, and send an online notification to the operating system.

In some embodiments of the present invention, the control module is further configured to: after binding the service processing task to the service core in the service core set domain, turn off the clock interruption of the service core. Therefore, by turning off the clock interrupt for scheduling on the service core, it can avoid interrupting the service processing task, thereby further improving the service processing performance.

According to some embodiments of the present invention, the device may further include: a system call proxy module, which is used to run a system call proxy task on the control core;

Among them, the system call proxy module can be specifically used for:

Write the system call parameter information when the business core makes a system call to the descriptor, and modify the descriptor status to request. When the service core makes a system call, it first obtains the position of the write pointer in the description ring. After legal judgment, if the parameter requires a large block of memory, first apply for a block of memory from the parameter memory pool, and fill in the system call number and the parameter or parameter memory address. In the descriptor, further, the status of the descriptor is changed to requesting, the loop waits for the status of the descriptor to be changed to the completion of execution, the result is retrieved, and the parameter memory pool is released.

Step 3: On the basis of the above steps, the operating environment of the business core has been cleaned up. Further, turn on the clock interrupt of the corresponding business core, send the CPU ready to go online notification to the operating system kernel, wake up the business core and send the CPU of the business core The logical state transitions to CPU_ONLINE_EXCLUSIVE. In this state, the business core is in an independent scheduling set domain, and the process scheduler starts execution in a completely different way from the control core. The run queue is empty, and no task or work is performed. Queues, timers, etc. interrupt the lower half of the program.

In the following, a specific embodiment is used to describe in detail the implementation of introducing a new CPU core logic state to perform asymmetric multi-processing scheduling in business logic according to an embodiment of the present invention:

Step 1: Add a new set of CPU_ONLINE_EXCLUSIVE. This set uses a CPU bitmap to indicate the CPU core in this logical state. The BIT position represents the index of the CPU core. 1 means that the CPU core is in the logical state, and 0 means not here. Logical state. The initial state of the collection is empty.

Step 2: The control core sends a offline notification to perform offline processing on one or more business cores. After the business core enters the CPU_OFFLINE logic state, the operating system removes the business core from the CPU_ONLINE set, which will be represented in the CPU bitmap representing the set The BIT of the business core is set from 1 to 0.

Step 3: The control core sends a service online notification. Before executing the wake-up action, add one or more service cores to the CPU_ONLINE_EXCLUSIVE set, that is, set the BIT of the service core in the CPU bitmap representing the set from 0 to 1.

Step 4: When the service core is awakened, it checks the CPU bitmap representing the CPU_ONLINE_EXCLUSIVE collection, and finds that its BIT is 1, then it no longer joins the CPU_ONLINE collection.

Step 5: After the service core is awakened, the clock interrupt triggers task scheduling. Each time the scheduler runs, the CPU bitmap representing the set of CPU_ONLINE_EXCLUSIVE is checked, and after finding that its BIT is 1, enter a different scheduling process.

Step 6: When the control core starts the business processing task and binds it to the business core, the judging legal condition is no longer just CPU_ONLINE, but the union of CPU_ONLINE and CPU_ONLINE_EXCLUSIVE.

Step 7: After the service core is scheduled to run the service processing program, turn off the clock interrupt.

The following describes in detail the implementation of the system call proxy according to the embodiment of the present invention with a specific embodiment:

Step 1: Pre-allocate shared memory that can be accessed by two control cores and business cores: as shown in Figure 6, one of them is used to represent the set of system call descriptors, and each descriptor contains status, system call number, parameters, and results For fields, each descriptor has the same length, is continuous in the address space, and is organized in a ring. The other block is used as a parameter memory pool to store parameters larger than 8 bytes in system calls (such as file buf).

Step 2: As shown in Figure 7, the control core starts the system call proxy task and sets it to a higher priority. This task continuously polls the status of the descriptor at the position of the read pointer in the descriptor ring until the query status changes to requesting Further, first modify the status as executing, and then take out the system call number and parameters to execute the system call. After the parsing and execution, if the parameter needs a large memory to save the result, apply from the parameter memory pool and copy the result to the parameter In the memory, further, the modification status is execution complete. The read pointer is incremented by 1, and the next descriptor is continued.

Step 3: When the service core makes a system call, it first obtains the position of the write pointer in the descriptor ring. After the legal judgment, if the parameter requires a large block of memory, first apply for a block of memory from the parameter memory pool, and set the system call number and parameters or parameters The memory address is filled in the descriptor, and further, the descriptor status is changed to requesting, the loop waits for the descriptor status to be changed to the completion of execution, the result is retrieved, and the parameter memory pool is released.

In the following, a specific embodiment is used to describe in detail the implementation of the multi-core system-based task processing method in the integrated system of communication equipment management and forwarding board according to the embodiment of the present invention:

Step 1: As shown in Figure 8, the system is powered on. After the CPU completes normal initialization, the status is CPU_ONLINE. All CPU cores complete the system version loading together, and the process and service are initialized and powered on. Different from the isolation of the CPU core at startup, the parallel execution of the CPU core greatly improves the power-on speed.

Step 2: Choose to divide the control core used for management and the service core used for data message forwarding, further, shut down the operating system to interrupt the load balancing management task, and further, send the CPU ready offline notification to the operating system kernel.

Step 3: On the basis of step 2, the operating system will run on the management process of the business core after receiving the above notification, the normal business process, the kernel timer and work queue, interrupt and interrupt the lower half of the program, migrate to the control core, After completing the migration operation, modify the CPU logic state of the service core to the CPU_OFFLINE state.

Step 4: On the basis of step 3, turn on the service core clock interrupt and modify the CPU logic state to CPU_ONLINE_EXCLUSIVE.

Step 5: On the basis of Step 4, start the packet sending and receiving task and bind it to the corresponding business core.

Step 6: Optionally, on the basis of step 5, turn off the clock interruption on the service core to reduce the time consumption caused by unnecessary system scheduling. At this time, the data forwarding task completely takes up the processing time of the service core and makes full use of the CPU core performance.

Step 7: Optionally, on the basis of Step 5, run the system call agent task on the control core.

The following describes in detail the implementation of the multifunctional server system according to the embodiment of the present invention with a specific embodiment:

With the development of SDN and virtualization, the use of server architecture software to customize network control and computing nodes, deployment of OVS, OpenStack and other applications are becoming more and more extensive. The implementation of the present invention in the application system is as follows:

Step 1: Type in the relevant kernel patch of the present invention on the server operating system.

Step 2: According to the number of server CPU cores and whether it is sufficient for business use, if the number of CPU cores is sufficient, the business CPU cores will be isolated during the system startup phase. Preferably, the startup parameter activates the patch. It can also be used before the business software runs. Make full use of each CPU core.

Step 3: The server is powered on, the operating system is loaded and the related application services are started. Optionally, the CPU core and memory resources are specified to start the corresponding virtual machine.

Step 4: Optionally, on the basis of step 2, select the CPU core in the host or virtual machine as the computing or soft forwarding interactive service. The same steps 3 to 7 as in the above embodiment, the service core is dynamically Isolate and bind to the business core for execution.

Step 5: Optionally, repeat the above steps to realize the recovery of business cores and the reuse of control cores.

According to the electronic device of the embodiment of the present invention, as shown in FIG. 9, the electronic device includes: a memory 901, a processor 902, and a computer program stored in the memory 901 and running on the processor 902, and the computer program is executed by the processor 902 When implementing the steps of the above-mentioned method.

According to the electronic device of the embodiment of the present invention, by executing the task processing method based on the multi-core system proposed by the present invention, in a symmetric multi-processing system, it can dynamically and smoothly switch to the asymmetric multi-processing state in business logic, so that business processing Tasks completely exclusively run the business core, and will not cause bottlenecks due to system jitter, which will affect business performance and indicators.

According to the computer storage medium of the embodiment of the present invention, by executing the task processing method based on the multi-core system proposed by the present invention, in a symmetric multi-processing system, it can dynamically and smoothly switch to the asymmetric multi-processing state in business logic, so that the business The processing tasks are completely exclusive to the operation of the business core, thus making full use of the single-core CPU processing capacity, and will not cause bottlenecks due to system jitter, affecting business performance and indicators.

Through the description of the specific embodiments, it should be possible to have a more in-depth and specific understanding of the technical means and effects adopted by the present invention to achieve the intended purpose. However, the attached drawings are only for reference and explanation, and are not used to describe the present invention. limit.

Claims

A task processing method based on a multi-core system, including:

Divide the CPU core into a service core collection domain and a control core collection domain;

The service processing task is bound to the service core in the service core collection domain, and the service core processes the service processing task.
The task processing method based on a multi-core system according to claim 1, wherein the dividing the CPU cores into a service core set domain and a control core set domain comprises:

Make all CPU cores online;

Select some CPU cores from all CPU cores as control cores to form the control core set domain;

The remaining CPU cores are used as service cores, and the service cores are processed offline and then online to form the service core set domain.
The task processing method based on a multi-core system according to claim 2, further comprising:

When all CPU cores are online, assign tasks to all CPU cores for processing;

After the business core is offline, the tasks in the business core are migrated to the control core.
The task processing method based on a multi-core system according to claim 2, further comprising:

Before the service core goes online again, the clock interrupt of the service core is turned on, and an online notification is sent to the operating system.
The task processing method based on the multi-core system according to claim 4, further comprising:

After the service processing task is bound to the service core in the service core set domain, the clock interruption of the service core is turned off.
The task processing method based on a multi-core system according to claim 1, further comprising:

Run a system call agent task on the control core;

By means of shared memory, the service core is made to perform system calls according to the running results of the system call agent task running on the control core.
7. The task processing method based on a multi-core system according to claim 6, wherein the method of sharing a memory queue to enable the service core to make a system call according to the system call agent task comprises:

Allocating the shared memory, where the shared memory includes a system call descriptor;

Write system call parameter information when the service core makes a system call into the descriptor, and modify the descriptor status to request;

The control core searches for the descriptor whose status is in the request, and executes the system call based on the system call parameter information in the descriptor.
A task processing device based on a multi-core system, including:

The division module is used to divide the CPU core into a service core collection domain and a control core collection domain;

The task processing module is used to bind business processing tasks to the business cores in the business core set domain, and the business cores process the business processing tasks.
An electronic device, comprising: a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the computer program is executed by the processor to implement 7. The steps of any one of the methods.
A computer storage medium storing a computer program, wherein when the computer program is executed by a processor, the steps of the task processing method based on a multi-core system according to any one of claims 1 to 7 are realized.