CN114416325A - Batch task computing system based on intelligent analysis - Google Patents
Batch task computing system based on intelligent analysis Download PDFInfo
- Publication number
- CN114416325A CN114416325A CN202210340398.8A CN202210340398A CN114416325A CN 114416325 A CN114416325 A CN 114416325A CN 202210340398 A CN202210340398 A CN 202210340398A CN 114416325 A CN114416325 A CN 114416325A
- Authority
- CN
- China
- Prior art keywords
- task
- module
- batch
- running
- temporary
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5044—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering hardware capabilities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/48—Indexing scheme relating to G06F9/48
- G06F2209/484—Precedence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5018—Thread allocation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5021—Priority
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a batch task computing system based on intelligent analysis, which aims to solve the technical problems that in the prior art, the proper number of threads cannot be created according to the memory occupation condition, the running speed of a program cannot be maximized, the intelligent analysis function is not provided, and tasks running simultaneously cannot be properly assigned for processing according to the system running condition, so that the system is easy to jam and the normal running of batch tasks is influenced. The system comprises a task acquisition module, a storage module, a task sequencing module, a task allocation module, a task calculation module and an intelligent analysis module; the task obtaining module obtains configuration information of batch tasks to obtain a batch task list. The system can set the threads with the correct number to maximize the running speed of the program by utilizing the task computing module, and can rearrange the batch tasks according to the running time, so that the system has the function of intelligent analysis and ensures that the batch tasks normally run in the optimal and fastest mode.
Description
Technical Field
The invention belongs to the field of computing systems, and particularly relates to a batch task computing system based on intelligent analysis.
Background
With the continuous development of computer technology, the function and the requirement of batch processing tasks are seen everywhere in numerous internet projects, and great convenience and efficiency improvement are brought to daily work through the batch processing tasks.
At present, the invention patent with patent number CN 201910598614.7 discloses a batch task arrangement method, which includes: acquiring resource use information for processing batch tasks; generating the optimal concurrency of the batch tasks according to the resource use information and a preset concurrency evaluation model; and arranging the batch tasks according to the optimal concurrency, the historical processing information of the batch tasks and a preset arrangement evaluation model. The method adopts scene analysis to further realize scientific job task arrangement, but the computing system cannot establish a proper number of threads according to the memory occupation condition, cannot maximize the running speed of a program, does not have the function of intelligent analysis, cannot properly allocate tasks running simultaneously according to the running condition of the system to process, is easy to cause system blockage, and influences the normal running of batch tasks.
Therefore, in order to solve the above-mentioned problems that the running speed of the program cannot be maximized and the function of intelligent analysis is not available, it is necessary to improve the usage scenario of the computing system.
Disclosure of Invention
(1) Technical problem to be solved
Aiming at the defects of the prior art, the invention aims to provide a batch task computing system based on intelligent analysis, which aims to solve the technical problems that in the prior art, the number of threads with proper number cannot be created according to the memory occupation condition, the running speed of a program cannot be maximized, the intelligent analysis function is unavailable, tasks running simultaneously cannot be properly assigned according to the running condition of the system, the system is easy to jam, and the normal running of batch tasks is influenced.
(2) Technical scheme
In order to solve the technical problems, the invention provides a batch task computing system based on intelligent analysis, which comprises a task acquisition module, a storage module, a task sequencing module, a task allocation module, a task computing module and an intelligent analysis module; wherein the content of the first and second substances,
the task obtaining module obtains configuration information of batch tasks to obtain a batch task list;
the storage module comprises a cache unit and a timing unit, temporarily stores the configuration information acquired by the task acquisition module in the cache unit, and then periodically clears cache data in the cache unit through the timing unit so as to release a cache space in the cache unit;
the task ordering module orders the tasks of the task acquisition module and generates a task ordering table, a GBDT ordering model is pre-installed in the task ordering module, and the algorithm steps of the GBDT ordering model are as follows: input (a)
，
) T, L, wherein
In order to be a sample of the sample,
for sorting labels, initialization
，
Calculating the response as a standard value
，i=1,2,…，N，
For the loss function, then the t-th tree is learned,
=arg
finding the step length
= arg
，
Is the complex conjugate of the model parameters,
in order to be the parameters of the model,
in order to predict the value of the target,
after the step length meets the condition formula, the model is updated:
，
to classify the output value of the learner, output
；
The task allocation module is internally pre-loaded with a scheduling algorithm, the scheduling algorithm divides the task sorting table according to the task sorting table of the task sorting module, generates sub-table 1, sub-table 2, … and sub-table Z respectively according to the sequence, and calculates the running time of the first task in the sub-table 1
Adding the first task to the first position in the running table 1, and calculating the running time of the first task in the sub-table 1
By using
And
make a comparison if
＜
Inserting the second task into the front end of the first task, if so
≥
If the second task is placed at the rear end of the first task, the adjustment of all task positions in the sub-table 1 is completed in the same way, the operation table 1 is generated, the operation tables 2, … and the operation table Z are generated in the same way, and the tasks in the operation table 1, the operation tables 2, … and the operation table Z are sequentially added to be concentrated to one position and a new batch task list is generated;
the task computing module creation
The number of the threads is one,
the running speed of the program is maximized by setting the correct number of threads;
an analysis rule is pre-installed in the intelligent analysis module, a new batch task list generated by the task allocation module is used for establishing a temporary task running queue, a first task in the new batch task list is added into the temporary task running queue, a second task is added into the temporary task running queue, the CPU occupancy rates of the first task and the second task are analyzed, if the CPU occupancy rate is more than or equal to 50%, the second task is kicked out of the temporary task running queue, only the first task is in the temporary task running queue and a task running queue is generated, if the CPU occupancy rate is less than 50%, the second task is reserved, at the moment, the temporary task running queue comprises the first task and the second task, then a third task is added into the temporary task running queue, the CPU occupancy rates of the first task, the second task and the third task are analyzed, and if the CPU occupancy rate is more than or equal to 50%, and kicking the third task out of the temporary task running queue, wherein the temporary task running queue comprises the first task and the second task and generates a task running queue, if the CPU occupancy rate is less than 50%, the third task is reserved, the temporary task running queue comprises the first task, the second task and the third task, and so on, and the final generated task running queue is generated.
When the system of the technical scheme is used, the task acquisition module acquires configuration information of tasks in batches to obtain a batch task list, the task sorting module sorts the tasks of the task acquisition module and generates a task sorting table, and the GBDT sorting model comprises the following algorithm steps: input (a)
，
) T, L, initialization
Calculating the response
I =1,2, …, N, then learns the t-th tree,
=arg
finding the step length
= arg
And after the condition formula is satisfied, updating the model:
output of
(ii) a The scheduling algorithm divides the task sorting table according to the task sorting table of the task sorting module, generates sub-table 1, sub-table 2, … and sub-table Z respectively according to the sequence, and calculates the running time of the first task in the sub-table 1
Adding the first task to the first position in the running table 1, and calculating the running time of the first task in the sub-table 1
By using
And
make a comparison if
＜
Inserting the second task into the front end of the first task, if so
≥
If the second task is placed at the rear end of the first task, the adjustment of all task positions in the sub-table 1 is completed in the same way, the operation table 1 is generated, the operation tables 2, … and the operation table Z are generated in the same way, and the tasks in the operation table 1, the operation tables 2, … and the operation table Z are sequentially added to be concentrated to one position and a new batch task list is generated; task computing module creation
The number of the threads is one,
the running speed of the program is maximized by setting the correct number of threads; the intelligent analysis module is internally pre-loaded with analysis rules, the new batch task list generated by the task allocation module establishes a temporary task running queue, and the first batch task list is added into the temporary task running queueOne task, then adding a second task into the temporary task running queue, analyzing the CPU occupancy rates of the first task and the second task, if the CPU occupancy rate is more than or equal to 50%, kicking the second task out of the temporary task running queue, only the first task in the temporary task running queue and generating a task running queue, if the CPU occupancy rate is less than 50%, reserving the second task, if the temporary task running queue comprises the first task and the second task, then adding a third task into the temporary task running queue, analyzing the CPU occupancy rates of the first task, the second task and the third task, if the CPU occupancy rate is more than or equal to 50%, kicking the third task out of the temporary task running queue, if the temporary task running queue comprises the first task and the second task and generating a task running queue, if the CPU occupancy rate is less than 50%, reserving the third task, at this time, the temporary task running queue includes a first task, a second task and a third task, and so on, to generate a final generated task running queue, and the storage module temporarily stores the configuration information acquired by the task acquisition module in the cache unit, and then periodically clears the cache data in the cache unit through the timing unit, thereby releasing the cache space in the cache unit.
Preferably, the configuration information of the task obtaining module includes a task identifier, a task type, and a task parameter.
Preferably, the
Represents L to F is
The negative bias, F is the initial value of the model, L is the mean square error,
=arg
representing the learning process of the t-th tree to which is fitted
Given a negative bias, T is the number of CART.
Preferably, the condition formula in the task sequencing module is
。
Preferably, the task computing module
Is the optimum number of threads.
Preferably, the number of tasks in sub-table 1, sub-table 2, … and sub-table Z in the task allocation module is 100.
Preferably, the specific process of the timing unit is as follows: setting the working period of the timing unit to be G minutes, screening the finished tasks in the cache unit after the G minutes are reached, and then clearing cache data corresponding to the finished tasks.
(3) Advantageous effects
Compared with the prior art, the invention has the beneficial effects that: the system of the invention can set the correct number of threads to maximize the running speed of the program by utilizing the task computing module, can sequence the batch tasks by utilizing the GBDT sequencing model pre-installed in the task sequencing module, can rearrange the batch tasks according to the running time by utilizing the task allocation module, and has the intelligent analysis function by utilizing the intelligent analysis module, so that the number of the tasks running each time can be conveniently allocated according to the running condition of the system, the utilization rate of the system is maximally improved, the blockage of the system can not be caused, and the batch tasks can be ensured to normally run in the optimal and fastest mode.
Drawings
FIG. 1 is a schematic diagram of an overall frame structure of one embodiment of the system of the present invention;
FIG. 2 is a flowchart of the operation of a scheduling algorithm in one embodiment of the system of the present invention;
FIG. 3 is a flowchart of the GBDT ranking model in an embodiment of the system of the present invention.
Detailed Description
Example 1
The specific embodiment is a batch task computing system based on intelligent analysis, the schematic diagram of the overall framework structure of the system is shown in fig. 1, the work flow diagram of a scheduling algorithm is shown in fig. 2, the work flow diagram of a GBDT sorting model is shown in fig. 3, and the system comprises a task acquisition module, a storage module, a task sorting module, a task allocation module, a task computing module and an intelligent analysis module;
the task obtaining module obtains configuration information of batch tasks to obtain a batch task list;
the storage module comprises a cache unit and a timing unit, temporarily stores the configuration information acquired by the task acquisition module in the cache unit, and then regularly clears cache data in the cache unit through the timing unit so as to release a cache space in the cache unit;
the task ordering module orders the tasks of the task acquisition module and generates a task ordering table, a GBDT ordering model is pre-installed in the task ordering module, and the algorithm steps of the GBDT ordering model are as follows: input (a)
，
) T, L, wherein
In order to be a sample of the sample,
for sorting labels, initialization
，
Calculating the response as a standard value
，i=1,2,…，N，
For the loss function, then the t-th tree is learned,
=arg
finding the step length
= arg
，
Is the complex conjugate of the model parameters,
in order to be the parameters of the model,
in order to predict the value of the target,
after the step length meets the condition formula, the model is updated:
，
to classify the output value of the learner, output
；
The task allocation module is internally pre-loaded with a scheduling algorithm and a scheduling algorithm rootAccording to the task sorting table of the task sorting module, the task sorting table is divided, sub-table 1, sub-table 2, sub-table … and sub-table Z are respectively generated according to the sequence, and the running time of the first task in the sub-table 1 is calculated
Adding the first task to the first position in the running table 1, and calculating the running time of the first task in the sub-table 1
By using
And
make a comparison if
＜
Inserting the second task into the front end of the first task, if so
≥
If the second task is placed at the rear end of the first task, the adjustment of all task positions in the sub-table 1 is completed in the same way, the operation table 1 is generated, the operation tables 2, … and the operation table Z are generated in the same way, and the tasks in the operation table 1, the operation tables 2, … and the operation table Z are sequentially added to be concentrated to one position and a new batch task list is generated;
task computing module creation
The number of the threads is one,
the running speed of the program is maximized by setting the correct number of threads;
an analysis rule is pre-installed in the intelligent analysis module, a new batch task list generated by the task allocation module is established, a temporary task running queue is established, a first task in the new batch task list is added into the temporary task running queue, a second task is added into the temporary task running queue, the CPU occupancy rates of the first task and the second task are analyzed, if the CPU occupancy rate is larger than or equal to 50%, the second task is kicked out of the temporary task running queue, only the first task is in the temporary task running queue at the moment and the task running queue is generated, if the CPU occupancy rate is smaller than 50%, the second task is reserved, at the moment, the temporary task running queue comprises the first task and the second task, then a third task is added into the temporary task running queue, the CPU occupancy rates of the first task, the second task and the third task are analyzed, and if the CPU occupancy rate is larger than or equal to 50%, and kicking the third task out of the temporary task running queue, wherein the temporary task running queue comprises the first task and the second task and generates a task running queue, if the CPU occupancy rate is less than 50%, the third task is reserved, the temporary task running queue comprises the first task, the second task and the third task, and so on, and the final generated task running queue is generated.
Wherein the configuration information of the task acquisition module comprises task identification, task type and task parameter,
represents L to F is
The negative bias, F is the initial value of the model, L is the mean square error,
=arg
representing the learning process of the t-th tree, the t-th tree simulationIs prepared by
Given a negative bias, T is the number of CART.
Meanwhile, the condition formula in the task sequencing module is
In task computing modules
Is the optimum number of threads.
In addition, the number of tasks in sub-table 1, sub-table 2, … and sub-table Z in the task allocation module is 100, and the specific flow of the timing unit is as follows: setting the working period of the timing unit to be G minutes, screening the finished tasks in the cache unit after the G minutes are reached, and then clearing cache data corresponding to the finished tasks.
When the system of the technical scheme is used, the task obtaining module obtains configuration information of batch tasks, the configuration information comprises task identifiers, task types and task parameters, a batch task list is obtained, the task sorting module sorts the tasks of the task obtaining module and generates a task sorting table, and the GBDT sorting model comprises the following algorithm steps: the task ordering module orders the tasks of the task acquisition module and generates a task ordering table, a GBDT ordering model is pre-installed in the task ordering module, and the algorithm steps of the GBDT ordering model are as follows: input (a)
，
) T, L, wherein
In order to be a sample of the sample,
for sorting labels, initialization
，
Calculating the response as a standard value
，i=1,2,…，N，
For the loss function, then the t-th tree is learned,
=arg
finding the step length
= arg
，
Is the complex conjugate of the model parameters,
in order to be the parameters of the model,
in order to predict the value of the target,
after the step length meets the condition formula, the model is updated:
，
to classify the output value of the learner, output
，
Represents L to F is
The negative bias, F is the initial value of the model, L is the mean square error,
=arg
representing the learning process of the t-th tree to which is fitted
Given a negative bias, T is the number of CARTs; the scheduling algorithm divides the task sorting table according to the task sorting table of the task sorting module, generates sub-table 1, sub-table 2, … and sub-table Z respectively according to the sequence, the number of tasks in sub-table 1, sub-table 2, … and sub-table Z in the task allocation module is 100, and calculates the running time of the first task in sub-table 1
Adding the first task to the first position in the running table 1, and calculating the running time of the first task in the sub-table 1
By using
And
make a comparison if
＜
Inserting the second task into the front end of the first task, if so
≥
If the second task is placed at the rear end of the first task, the adjustment of all task positions in the sub-table 1 is completed in the same way, the operation table 1 is generated, the operation tables 2, … and the operation table Z are generated in the same way, and the tasks in the operation table 1, the operation tables 2, … and the operation table Z are sequentially added to be concentrated to one position and a new batch task list is generated; task computing module creation
The number of the threads is one,
the running speed of the program is maximized by setting the correct number of threads,
the optimal number of threads; an analysis rule is pre-installed in the intelligent analysis module, a new batch task list generated by the task allocation module is used for establishing a temporary task running queue, a first task in the new batch task list is added into the temporary task running queue, a second task is added into the temporary task running queue, the CPU occupancy rates of the first task and the second task are analyzed, if the CPU occupancy rate is larger than or equal to 50%, the second task is kicked out of the temporary task running queue, only the first task is in the temporary task running queue at the moment and generates the temporary task running queue, if the CPU occupancy rate is smaller than 50%, the second task is reserved, at the moment, the first task and the second task are included in the temporary task running queue, a third task is added into the temporary task running queue, and the analysis rule is added into the temporary task running queue, so that a third task is analyzedIf the CPU occupancy rate is more than or equal to 50%, kicking the third task out of the temporary task running queue, wherein the temporary task running queue comprises the first task and the second task and generates a task running queue, if the CPU occupancy rate is less than 50%, reserving the third task, wherein the temporary task running queue comprises the first task, the second task and the third task, and so on, generating a final generated task running queue, meanwhile, temporarily storing the configuration information acquired by the task acquisition module in the cache unit by the storage module, and then, regularly clearing cache data in the cache unit through the timing unit so as to release the cache space in the cache unit, wherein the specific flow of the timing unit is as follows: setting the working period of the timing unit to be G minutes, screening the finished tasks in the cache unit after the G minutes are reached, and then clearing cache data corresponding to the finished tasks.
Claims (7)
1. The system comprises a task acquisition module, a storage module, a task sequencing module, a task allocation module, a task calculation module and an intelligent analysis module; it is characterized in that the preparation method is characterized in that,
the task obtaining module obtains configuration information of batch tasks to obtain a batch task list;
the storage module comprises a cache unit and a timing unit, temporarily stores the configuration information acquired by the task acquisition module in the cache unit, and then periodically clears cache data in the cache unit through the timing unit so as to release a cache space in the cache unit;
the task ordering module orders the tasks of the task acquisition module and generates a task ordering table, a GBDT ordering model is pre-installed in the task ordering module, and the algorithm steps of the GBDT ordering model are as follows: input (a)
，
) T, L, wherein
In order to be a sample of the sample,
for sorting labels, initialization
，
Calculating the response as a standard value
，i=1,2,…，N，
For the loss function, then the t-th tree is learned,
=arg
finding the step length
= arg
，
Is the complex conjugate of the model parameters,
in order to be the parameters of the model,
in order to predict the value of the target,
after the step length meets the condition formula, the model is updated:
，
to classify the output value of the learner, output
；
The task allocation module is internally pre-loaded with a scheduling algorithm, the scheduling algorithm divides the task sorting table according to the task sorting table of the task sorting module, generates sub-table 1, sub-table 2, … and sub-table Z respectively according to the sequence, and calculates the running time of the first task in the sub-table 1
Adding the first task to the first position in the running table 1, and calculating the running time of the first task in the sub-table 1
By using
And
make a comparison if
＜
Inserting the second task into the front end of the first task, if so
≥
If the second task is placed at the rear end of the first task, the adjustment of all task positions in the sub-table 1 is completed in the same way, the operation table 1 is generated, the operation tables 2, … and the operation table Z are generated in the same way, and the tasks in the operation table 1, the operation tables 2, … and the operation table Z are sequentially added to be concentrated to one position and a new batch task list is generated;
the task computing module creation
The number of the threads is one,
the running speed of the program is maximized by setting the correct number of threads;
an analysis rule is pre-installed in the intelligent analysis module, a new batch task list generated by the task allocation module is used for establishing a temporary task running queue, a first task in the new batch task list is added into the temporary task running queue, a second task is added into the temporary task running queue, the CPU occupancy rates of the first task and the second task are analyzed, if the CPU occupancy rate is more than or equal to 50%, the second task is kicked out of the temporary task running queue, only the first task is in the temporary task running queue and a task running queue is generated, if the CPU occupancy rate is less than 50%, the second task is reserved, at the moment, the temporary task running queue comprises the first task and the second task, then a third task is added into the temporary task running queue, the CPU occupancy rates of the first task, the second task and the third task are analyzed, and if the CPU occupancy rate is more than or equal to 50%, and kicking the third task out of the temporary task running queue, wherein the temporary task running queue comprises the first task and the second task and generates a task running queue, if the CPU occupancy rate is less than 50%, the third task is reserved, the temporary task running queue comprises the first task, the second task and the third task, and so on, and the final generated task running queue is generated.
2. The batch task computing system based on intelligent analysis of claim 1, wherein the configuration information of the task obtaining module includes task identification, task type, and task parameters.
3. The batch task computing system based on intelligent analysis of claim 1, wherein the batch task computing system is based on a batch task computing system based on intelligent analysis of the batch task computing system
Represents L to F is
The negative bias, F is the initial value of the model, L is the mean square error,
=arg
representing the learning process of the t-th tree to which is fitted
Given a negative bias, T is the number of CART.
4. The batch task computing system based on intelligent analysis of claim 1, wherein the conditional formula in the task ranking module is
。
5. The batch task computing system based on intelligent analysis of claim 1, wherein the task computing module is provided with
Is the optimum number of threads.
6. The batch task computing system based on intelligent analysis of claim 1, wherein the number of tasks in sub-table 1, sub-table 2, …, sub-table Z in the task allocation module is 100.
7. The batch task computing system based on intelligent analysis according to claim 1, wherein the specific flow of the timing unit is as follows: setting the working period of the timing unit to be G minutes, screening the finished tasks in the cache unit after the G minutes are reached, and then clearing cache data corresponding to the finished tasks.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210340398.8A CN114416325B (en) | 2022-04-02 | 2022-04-02 | Batch task computing system based on intelligent analysis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210340398.8A CN114416325B (en) | 2022-04-02 | 2022-04-02 | Batch task computing system based on intelligent analysis |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114416325A true CN114416325A (en) | 2022-04-29 |
| CN114416325B CN114416325B (en) | 2022-08-26 |
Family
ID=81264052
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210340398.8A Active CN114416325B (en) | 2022-04-02 | 2022-04-02 | Batch task computing system based on intelligent analysis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114416325B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117806261A (en) * | 2024-02-29 | 2024-04-02 | 佛山慧谷科技股份有限公司 | Stone polishing queue management system and method |
| CN117806261B (en) * | 2024-02-29 | 2024-05-28 | 佛山慧谷科技股份有限公司 | Stone polishing queue management system and method |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080195241A1 (en) * | 2007-02-13 | 2008-08-14 | Tech Semiconductor Singapore Pte Ltd | Method and system for wafer lot order |
| US20130290976A1 (en) * | 2012-04-30 | 2013-10-31 | Ludmila Cherkasova | Scheduling mapreduce job sets |
| US20150058858A1 (en) * | 2013-08-21 | 2015-02-26 | Hasso-Platt ner-Institut fur Softwaresystemtechnik GmbH | Dynamic task prioritization for in-memory databases |
| CN110119307A (en) * | 2018-02-05 | 2019-08-13 | 上海交通大学 | Processing method, device, storage medium and the electronic device of data processing request |
| US20200026569A1 (en) * | 2017-05-04 | 2020-01-23 | Salesforce.Com, Inc. | Systems, methods, and apparatuses for implementing a scheduler and workload manager with cyclical service level target (slt) optimization |
| CN111078396A (en) * | 2019-11-22 | 2020-04-28 | 厦门安胜网络科技有限公司 | Distributed data access method and system based on multitask instances |
| CN113238861A (en) * | 2021-05-08 | 2021-08-10 | 北京天空卫士网络安全技术有限公司 | Task execution method and device |
-
2022
- 2022-04-02 CN CN202210340398.8A patent/CN114416325B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080195241A1 (en) * | 2007-02-13 | 2008-08-14 | Tech Semiconductor Singapore Pte Ltd | Method and system for wafer lot order |
| US20130290976A1 (en) * | 2012-04-30 | 2013-10-31 | Ludmila Cherkasova | Scheduling mapreduce job sets |
| US20150058858A1 (en) * | 2013-08-21 | 2015-02-26 | Hasso-Platt ner-Institut fur Softwaresystemtechnik GmbH | Dynamic task prioritization for in-memory databases |
| US20200026569A1 (en) * | 2017-05-04 | 2020-01-23 | Salesforce.Com, Inc. | Systems, methods, and apparatuses for implementing a scheduler and workload manager with cyclical service level target (slt) optimization |
| CN110119307A (en) * | 2018-02-05 | 2019-08-13 | 上海交通大学 | Processing method, device, storage medium and the electronic device of data processing request |
| CN111078396A (en) * | 2019-11-22 | 2020-04-28 | 厦门安胜网络科技有限公司 | Distributed data access method and system based on multitask instances |
| CN113238861A (en) * | 2021-05-08 | 2021-08-10 | 北京天空卫士网络安全技术有限公司 | Task execution method and device |
Non-Patent Citations (1)
| Title |
|---|
| 中国通信学会学术工作委员会编: "《第十届中国通信学会学术年会论文集》", 31 December 2015, pages: 295 - 296 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117806261A (en) * | 2024-02-29 | 2024-04-02 | 佛山慧谷科技股份有限公司 | Stone polishing queue management system and method |
| CN117806261B (en) * | 2024-02-29 | 2024-05-28 | 佛山慧谷科技股份有限公司 | Stone polishing queue management system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114416325B (en) | 2022-08-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104317658B (en) | A kind of loaded self-adaptive method for scheduling task based on MapReduce | |
| CN109933306B (en) | Self-adaptive hybrid cloud computing framework generation method based on operation type recognition | |
| CN105956021B (en) | A kind of automation task suitable for distributed machines study parallel method and its system | |
| CN113157413B (en) | Deep learning task resource optimization configuration method and system based on service quality requirement | |
| CN110019486A (en) | Collecting method, device, equipment and storage medium | |
| CN107305501B (en) | Multi-stream data processing method and system | |
| CN107908536B (en) | Performance evaluation method and system for GPU application in CPU-GPU heterogeneous environment | |
| CN106874112B (en) | Workflow backfilling method combined with load balancing | |
| WO2023124947A1 (en) | Task processing method and apparatus, and related device | |
| CN105677763A (en) | Image quality evaluating system based on Hadoop | |
| WO2023093375A1 (en) | Computing resource acquisition method and apparatus, electronic device, and storage medium | |
| CN110990121A (en) | Kubernetes scheduling strategy based on application portrait | |
| CN109445386A (en) | A kind of most short production time dispatching method of the cloud manufacturing operation based on ONBA | |
| CN106294381A (en) | The method and system that big data calculate | |
| CN113010283A (en) | Online service method and device, electronic equipment and readable storage medium | |
| CN114416325B (en) | Batch task computing system based on intelligent analysis | |
| CN113010296B (en) | Formalized model based task analysis and resource allocation method and system | |
| CN112966486A (en) | Intelligent engineering quantity list generation method and device, terminal and storage medium | |
| CN104657422A (en) | Classification decision tree-based intelligent content distribution classification method | |
| CN111190704A (en) | Task classification processing method based on big data processing framework | |
| CN109885384A (en) | Task concurrency optimization method, apparatus, computer equipment and storage medium | |
| CN112561130B (en) | Material resource purchasing demand balance optimization system and method | |
| CN114625654A (en) | Test method and related equipment thereof | |
| CN103258009A (en) | Method and system for acquiring and analyzing method performance data | |
| CN112070162A (en) | Multi-class processing task training sample construction method, device and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |