CN116700929A - Task batch processing method and system based on artificial intelligence - Google Patents
Task batch processing method and system based on artificial intelligence Download PDFInfo
- Publication number
- CN116700929A CN116700929A CN202310722194.5A CN202310722194A CN116700929A CN 116700929 A CN116700929 A CN 116700929A CN 202310722194 A CN202310722194 A CN 202310722194A CN 116700929 A CN116700929 A CN 116700929A
- Authority
- CN
- China
- Prior art keywords
- task
- allocation
- tasks
- constraint
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 27
- 238000013473 artificial intelligence Methods 0.000 title claims abstract description 10
- 238000012545 processing Methods 0.000 claims abstract description 151
- 238000000034 method Methods 0.000 claims abstract description 22
- 238000013468 resource allocation Methods 0.000 claims abstract description 17
- 230000005540 biological transmission Effects 0.000 claims description 34
- 238000004364 calculation method Methods 0.000 claims description 6
- 230000001174 ascending effect Effects 0.000 claims description 4
- 230000010365 information processing Effects 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 10
- 230000003068 static effect Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to the field of information processing, and discloses a task batch processing method and system based on artificial intelligence, wherein the method comprises the following steps: identifying request characteristics of the task processing requests, and grading the tasks of the task processing requests to obtain graded tasks; inquiring a processing node of the classified task, and performing task pre-allocation on the classified task based on the processing node to obtain a pre-allocation task; computing the computing resource of the pre-allocation task, inquiring the available resource of the resource allocation node, and identifying the serial port of the pre-allocation task; taking computing resources, available resources and allocation serial ports as constraint conditions of pre-allocation tasks; task planning is carried out on the pre-allocation task according to constraint conditions, a preliminary allocation task is obtained, constraint values of the preliminary allocation task are calculated, and an optimal strategy of the preliminary planning task is created according to the constraint values; and scheduling the task processing request according to the optimal strategy to obtain a task scheduling strategy, and processing the task of the task processing request according to the task scheduling strategy. The invention can improve the efficiency of batch task processing.
Description
Technical Field
The invention relates to the field of information processing, in particular to a task batch processing method and system based on artificial intelligence.
Background
In daily operation and maintenance of an enterprise, update and maintenance work on enterprise information and work reports are required, such as creation of commodities, name change, description change, distribution area setting, and the like. As enterprise data increases, so too does the effort, which is important for how large numbers of tasks are handled in batches.
Currently, batch processing of tasks is generally based on a multithreading method, and by constructing a thread pool, different tasks are placed in the thread pool to perform processing resource allocation and then task processing is performed. However, this approach does not take into account the problem of resource reallocation when demand changes occur to batch tasks, resulting in inefficiency in batch processing of tasks.
Disclosure of Invention
In order to solve the technical problems, the invention provides a task batch processing method and a task batch processing system, which can improve the task batch processing efficiency.
In a first aspect, the present invention provides a method and a system for batch processing of tasks, including:
acquiring a task processing request, identifying request characteristics of the task processing request, and classifying the task processing request based on the request characteristics to obtain classified tasks;
inquiring a processing node of each task in the hierarchical tasks, and performing task pre-allocation on the hierarchical tasks based on the processing nodes to obtain pre-allocation tasks;
calculating the calculation resources of the pre-allocation task, inquiring available resources of the resource allocation node corresponding to the task processing request, and identifying a task allocation serial port of the pre-allocation task;
taking the computing resources, the available resources and the allocation serial ports as task constraint conditions of the pre-allocation task;
performing task planning on the pre-allocation task according to the task constraint conditions to obtain a preliminary allocation task, calculating a constraint value of the preliminary allocation task, and creating an optimal task strategy of the preliminary planning task according to the constraint value;
and carrying out task scheduling on the task processing request according to the optimal task strategy to obtain a task scheduling strategy, and carrying out task processing on the task processing request according to the task scheduling strategy.
In a possible implementation manner of the first aspect, the identifying a request feature of the task processing request includes:
decoding the task processing request to obtain a decoding task;
inquiring a request identifier of the decoding task, and identifying an annotation symbol in the request identifier;
and analyzing annotation semantics of the annotation symbol, and identifying request features of the task processing request according to the annotation semantics.
In a possible implementation manner of the first aspect, the task grading the task processing request based on the request feature, to obtain a graded task, includes:
according to the request characteristics, calculating task weights of the hierarchical tasks by using the following formula:
wherein G represents task weight of hierarchical tasks, n represents task number of hierarchical tasks, and x 1 Representing a first hierarchical task, x 2 Representing a second hierarchical task, x n Represents the nth hierarchical task, f represents a weighted average, x i Representing an ith request feature;
classifying the task weight according to a preset weight interval to obtain a classified weight interval;
and grading the tasks in the task processing request according to the grades corresponding to the grading weight intervals to obtain graded tasks.
In a possible implementation manner of the first aspect, the calculating the computing resource of the pre-allocation task includes:
calculating the computing resource duty ratio of the pre-allocation task by using the following formula:
wherein T represents the computing resource duty ratio of the pre-allocation tasks, n represents the number of the pre-allocation tasks, S r The resource transmission rate of the resource allocation node for allocating resources for the r-th task in the pre-allocation tasks is represented, and r (t) represents the task cache resources of the r-th task in the pre-allocation tasks;
and obtaining the computing resources of the pre-allocation task according to the computing resource duty ratio.
In a possible implementation manner of the first aspect, the identifying a task allocation serial port of the pre-allocation task includes:
inquiring a transmission protocol of the pre-allocation task;
identifying the transmission ip of the pre-allocation task according to the transmission protocol;
and searching a task transmission channel of the pre-allocation task according to the transmission ip, and inquiring a transmission serial port corresponding to the task transmission channel.
In one possible implementation manner of the first aspect, the task planning, according to the task constraint condition, the pre-allocation task to obtain a preliminary allocation task includes:
constructing constraint parameters of the task constraint conditions and task parameters of the pre-allocation task;
setting the constraint parameter as a limiting condition, and calculating the adaptability of the task parameter to the limiting condition by using the following formula:
wherein S represents fitness, y i Representing computational resource constraints in a defined condition, y j Representing available resource limits in a defined condition, y k Represents the distribution serial port limitation in the limiting condition, exp represents the global search function, m represents the parameter quantity of the task parameters, T C Representing task parametersC parameter of (a);
based on the adaptability, carrying out parameter search on the task parameters to obtain target parameters;
and carrying out parameter receipt on the pre-allocation task according to the target parameter to obtain a preliminary allocation task.
In a possible implementation manner of the first aspect, the calculating a constraint value of the preliminary allocation task includes:
calculating constraint values of the preliminary allocation tasks by using the following formula:
wherein Y represents constraint value, f (Y) represents constraint cost function, delta represents constraint relation among constraint conditions, m represents task number of preliminary allocation tasks, and L q Representing the q-th task of the preliminary assigned tasks.
In a possible implementation manner of the first aspect, the creating an optimal task strategy for the preliminary planning task according to the constraint value includes:
creating a task list of the preliminary planning task;
arranging the constraint values in the task list according to ascending order to obtain a constraint value sequence;
inquiring the preliminary planning task corresponding to the constraint value according to the constraint sequence, and arranging the preliminary planning task according to the constraint value arrangement rule to obtain an arrangement task;
generating task parameters of the arrangement tasks, and inputting the task parameters into a pre-constructed component to obtain a task component;
and constructing a task execution strategy of the preliminary planning task according to the arrangement sequence of the task components to obtain an optimal task strategy.
In one possible implementation manner of the first aspect, the task scheduling, according to the optimal task policy, is performed on the task processing request to obtain a task scheduling policy, including:
identifying parallel tasks in the optimal task strategy;
adding a parallel character for the task processing request according to the parallel task, and setting an execution interval for the task processing request according to a task time sequence in the optimal task strategy;
and setting a trigger event of the task processing request based on the parallel character and the execution interval, and executing task scheduling of the task processing request based on the trigger event.
In a second aspect, the present invention provides a method and a system for batch processing of tasks, where the system includes:
the task classification module is used for acquiring task processing requests, identifying request characteristics of the task processing requests, and classifying the task processing requests based on the request characteristics to obtain classified tasks;
the task allocation module is used for inquiring the processing node of each task in the hierarchical tasks, and performing task pre-allocation on the hierarchical tasks based on the processing nodes to obtain pre-allocation tasks;
the constraint query module is used for calculating the calculation resources of the pre-allocation task, querying available resources of the resource allocation node corresponding to the task processing request and identifying a task allocation serial port of the pre-allocation task;
the constraint configuration module is used for taking the computing resources, the available resources and the allocation serial ports as task constraint conditions of the pre-allocation task;
the strategy planning module is used for carrying out task planning on the pre-allocation task according to the task constraint condition to obtain a preliminary allocation task, calculating a constraint value of the preliminary allocation task, and creating an optimal task strategy of the preliminary planning task according to the constraint value;
and the task execution module is used for carrying out task scheduling on the task processing request according to the optimal task strategy to obtain a task scheduling strategy, and carrying out task processing on the task processing request according to the task scheduling strategy.
Compared with the prior art, the technical principle and beneficial effect of this scheme lie in:
according to the scheme, firstly, the specificity of all tasks can be known by identifying the request characteristics of the task processing requests, so that the tasks are better distributed, and the task processing requests are subjected to task classification based on the request characteristics, so that a large number of task requests can be classified according to the classification tasks, and the execution priority of each task can be clearly known; secondly, the embodiment of the invention can know the execution time of each task by inquiring the processing node of each task in the hierarchical tasks, thereby better distributing the processing sequence of each task, wherein the processing node refers to the deadline of task processing; computing resources of the pre-allocation tasks can know computing resources required by each task to be processed, so that reasonable computing resource allocation is performed on the pre-allocation tasks; further, the embodiment of the invention can know the serial port through which each task needs to pass when being processed by identifying the task distribution serial port of the pre-distribution task, so that the tasks can be used as similar tasks or processed simultaneously, the distribution amount of the computer tasks is reduced, the computing resources, the available resources and the distribution serial port are reduced, the constraint condition of the task which is used as the pre-distribution task can list the constraint condition which needs to be considered when the task is distributed, further, when the task is distributed, the reasonable task distribution can be carried out according to the constraint condition, task planning can be carried out on the pre-distribution task according to the constraint condition of the task, a task processing method which meets the condition constraint of an actual application scene can be obtained by primarily distributing the tasks, further, the task processing scheme which is better in the pre-distribution task can be distributed, the task scheduling request can be scheduled according to the optimal task scheduling policy, the task execution method and the task scheduling time which are practically feasible can be arranged for the task processing request are obtained, and a reliable static scheduling scheme which refers to the task execution policy is provided for the task processing request, wherein the task scheduling policy comprises the task scheduling policy and the priority scheduling policy. Therefore, the task batch processing method and system based on the artificial intelligence can improve the task batch processing efficiency.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the embodiments of the invention or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, and it will be obvious to a person skilled in the art that other drawings can be obtained from these drawings without inventive effort.
FIG. 1 is a schematic flow chart of a task batch processing method and system according to an embodiment of the present invention;
fig. 2 is a schematic block diagram of a task batch processing method and system according to an embodiment of the present invention.
Detailed Description
It should be understood that the detailed description is presented by way of example only and is not intended to limit the invention.
The embodiment of the invention provides a task batch processing method and a task batch processing system, wherein an execution main body of the task batch processing method and the task batch processing system comprises at least one of a server, a terminal and the like which can be configured to execute the method provided by the embodiment of the invention. In other words, the task batch processing method and system can be executed by software or hardware installed in a terminal device or a server device, and the software can be a blockchain platform. The service end includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like. The server may be an independent server, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.
Referring to fig. 1, a flow chart of a task batch processing method and system according to an embodiment of the invention is shown. The task batch processing method and system based on artificial intelligence depicted in fig. 1 comprises the following steps S1-S6:
s1, acquiring a task processing request, identifying the request characteristics of the task processing request, and classifying the task processing request based on the request characteristics to obtain classified tasks.
According to the embodiment of the invention, the specificity of all tasks can be known through the request characteristics for identifying the task processing request, so that the tasks can be better distributed. Wherein, the request feature refers to the special requirements of the specific result or target of the request such as when the request must be completed or the task execution of a specific unit.
As one embodiment of the present invention, the identifying the request feature of the task processing request includes: decoding the task processing request to obtain a decoding task, inquiring a request identifier of the decoding task, identifying an annotation symbol in the request identifier, analyzing annotation semantics of the annotation symbol, and identifying request characteristics of the task processing request according to the annotation semantics.
The decoding refers to a technology of converting data into a computer recognizable signal, the request identifier refers to an indication symbol added for a request task and can be represented by a binary code or a digital symbol, the annotating symbol refers to an expanded template for a class/method, each class/method annotates different parameters for the class/method according to rules in the annotating class, such as automatically adding commodity consumption amount when commodity order is queried, and the annotating semantic refers to meaning represented by the annotating symbol.
Optionally, the task processing request is decoded, the decoding task is obtained and implemented through a decoding function generated by a c language, a request identifier of the decoding task is searched for a symbol representing query of the decoding task, an annotation symbol in the request identifier is identified through an annotation tool of a programming language java, and annotation semantics of the annotation symbol are interpreted through translation of the annotation symbol code.
Furthermore, in the embodiment of the invention, the task processing requests are subjected to task classification based on the request characteristics, so that a large number of task requests can be classified in level by obtaining the classified tasks, and the execution priority of each task can be clearly known.
As one embodiment of the present invention, the task grading the task processing request based on the request feature, to obtain a graded task, includes:
according to the request characteristics, calculating task weights of the hierarchical tasks by using the following formula:
wherein G represents task weight of hierarchical tasks, n represents task number of hierarchical tasks, and x 1 Representing a first hierarchical task, x 2 Representing a second hierarchical task, x n Represents the nth hierarchical task, f represents a weighted average, x i Representing an ith request feature;
classifying the task weight according to a preset weight interval to obtain a classified weight interval;
and grading the tasks in the task processing request according to the grades corresponding to the grading weight intervals to obtain graded tasks.
The preset weight interval may be set to 1 level, 2 level, 3 level, etc., for example, the early seven-point conference is 1 level, the customer of the receiving company is 2 level, the higher the value of the account of the inquiring company is 3 level, the higher the weight level is, and the preset weight interval may also be set according to the actual application scene.
S2, inquiring processing nodes of each task in the hierarchical tasks, and performing task pre-allocation on the hierarchical tasks based on the processing nodes to obtain pre-allocation tasks.
According to the method and the system for processing the task, the processing node of each task in the hierarchical task can be queried to know the execution time of each task, and further the processing sequence of each task is better distributed, wherein the processing node refers to a certain time period of task processing, such as a meeting notice sent by a company to each employee, and the meeting notice needs to be completed within the time node from one afternoon to three afternoon.
Optionally, the processing node querying each task in the hierarchical tasks queries by identifying a time tag in a demand tag of each task in the hierarchical tasks.
Further, in the embodiment of the present invention, task pre-allocation is performed on the hierarchical task based on the processing node, so that the pre-allocation task is obtained, and a preliminary executable target task is created for the hierarchical task. Wherein the pre-allocation refers to an initial allocation scheme and not necessarily a final scheme.
Optionally, the task pre-allocation is performed on the hierarchical task based on the processing node, so that the pre-allocation task performs task pre-allocation on the hierarchical task according to the node sequence by inquiring the node sequence of the processing node.
S3, calculating the calculation resources of the pre-allocation task, inquiring available resources of the resource allocation node corresponding to the task processing request, and identifying a task allocation serial port of the pre-allocation task.
According to the embodiment of the invention, the computing resources required by each task to be processed can be known through the computing resources for computing the pre-allocation tasks, so that the pre-allocation tasks are reasonably distributed. The computing resources refer to processing resources allocated by a processor when the computer executes tasks, for example, when a user inquires a commodity order, the computer needs to allocate resources to inquire the commodity, and the larger the task requirement is, the more computing resources are needed.
As one embodiment of the present invention, the computing resources for computing the pre-allocation task includes:
calculating the computing resource duty ratio of the pre-allocation task by using the following formula:
wherein T represents the computing resource duty ratio of the pre-allocation tasks, n represents the number of the pre-allocation tasks, S r And (3) representing the resource transmission rate of the resource allocation node for the resource allocation of the r-th task in the pre-allocation tasks, r (t) representing the task cache resource of the r-th task in the pre-allocation tasks, and obtaining the computing resource of the pre-allocation tasks according to the computing resource duty ratio.
The computing resources of the pre-allocation task are obtained according to the computing resource duty ratio by querying the total computing resources of the pre-allocation task and multiplying the computing resource duty ratio.
Furthermore, the embodiment of the invention can know the available resources of the current computer by inquiring the available resources of the resource allocation node corresponding to the task processing request, and further reasonably allocate and process the task according to the available resources.
Optionally, the querying the available resources of the resource allocation node corresponding to the task processing request is implemented by accessing a resource space of the resource allocation node and querying the resource availability attribute in the resource space.
The embodiment of the invention can know the serial port which each task needs to pass through when processing by identifying the task distribution serial port of the pre-distribution task, thereby being capable of processing the tasks as the same kind of tasks or simultaneously and reducing the distribution amount of the computer task. The serial port refers to an interface for data transmission.
As a personal embodiment of the present invention, the task allocation serial port for identifying the pre-allocation task includes: inquiring a transmission protocol of the pre-allocation task, identifying a transmission ip of the pre-allocation task according to the transmission protocol, retrieving a task transmission channel of the pre-allocation task according to the transmission ip, and inquiring a task allocation serial port corresponding to the task transmission channel.
Wherein the transmission protocol refers to a system standard that allows two or more terminals in a transmission system to propagate information between them in any physical medium, and also refers to a common language of computer communication or network devices, the transmission ip refers to an address of data transmission, and the transmission channel refers to a line provided for data transmission.
Optionally, the transmission protocol is obtained by querying the pre-allocation task protocol packet, the transmission ip identifying the pre-allocation task according to the transmission protocol is obtained by identifying an ip packet in the protocol packet, and the task allocation serial port retrieving the pre-allocation task according to the transmission ip is retrieved by a retrieval script.
S4, taking the computing resources, the available resources and the distribution serial ports as task constraint conditions of the pre-distribution task.
According to the embodiment of the invention, the computing resources, the available resources and the allocation serial port are used as the task constraint conditions of the pre-allocation task, so that the constraint conditions which are required to be considered in task allocation of the pre-allocation task can be listed, and further, during task allocation, reasonable task allocation can be performed according to the constraint conditions.
Where the constraint means that the objective function is often maximized (or minimized) under certain constraints, they contain variables that are used to represent the decision scheme, thereby imposing a limit on the decision scheme.
S5, performing task planning on the pre-allocation task according to the task constraint conditions to obtain a preliminary allocation task, calculating a constraint value of the preliminary allocation task, and creating an optimal task strategy of the preliminary planning task according to the constraint value.
According to the task processing method, task planning is conducted on the pre-allocation task according to the task constraint conditions, so that a task processing method which meets the condition limitation of the actual application scene can be obtained through preliminary allocation of the task, and further a better task processing scheme in the pre-allocation task can be analyzed.
As an embodiment of the present invention, the task planning for the pre-allocation task according to the task constraint condition, to obtain a preliminary allocation task, includes: constructing constraint parameters of the task constraint conditions and task parameters of the pre-allocation task, inputting the constraint parameters and the task parameters into a pre-constructed task planning model, setting the constraint parameters as limiting conditions, and constructing the constraint parameters of the task constraint conditions and the task parameters of the pre-allocation task based on the limiting conditions;
setting the constraint parameter as a limiting condition, and calculating the adaptability of the task parameter to the limiting condition by using the following formula:
wherein S represents fitness, y i Representing computational resource constraints in a defined condition, y j Representing available resource limits in a defined condition, y k Represents the distribution serial port limitation in the limiting condition, exp represents the global search function, m represents the parameter quantity of the task parameters, T C And expressing the c-th parameter in the task parameters, carrying out parameter search on the task parameters based on the adaptability to obtain target parameters, and carrying out parameter receipt on the pre-allocation task according to the target parameters to obtain a preliminary allocation task.
The parameter searching refers to searching parameters meeting constraint conditions by using the global searching function, the parameter receipt refers to feeding the searched parameters meeting constraint conditions back to the system, and the system further identifies corresponding pre-allocation tasks according to the parameters.
Furthermore, in the embodiment of the present invention, the feasibility of limiting the execution scheme of each task due to various reasons in the actual application scenario can be known by calculating the constraint value of the preliminary task allocation. The constraint value is a specific value obtained through data calculation, and represents condition constraint, when the constraint value is larger than 0.5, the limit for executing a task with higher constraint is higher, when the constraint value is larger than or equal to 1, the condition is represented to completely limit the task not to be executed, and when the constraint value is not larger than 0.5, the limit for executing the task is represented to be lower, and the constraint value can be set according to actual application scenes.
As one embodiment of the present invention, the calculating the constraint value of the preliminary allocation task includes:
calculating constraint values of the preliminary allocation tasks by using the following formula:
wherein Y represents constraint value, f (Y) represents constraint cost, delta represents various constraint relations, m represents task number of preliminary allocation tasks, and L q Representing the q-th task of the preliminary assigned tasks.
Furthermore, according to the embodiment of the invention, the most reasonable or more reasonable processing method can be created for the task to be processed through the optimal task strategy for creating the preliminary planning task according to the constraint value, so that the task processing efficiency is improved.
As an embodiment of the present invention, the creating the optimal task strategy for the preliminary planning task according to the constraint value includes: creating a task list of the preliminary planning task, arranging the constraint values in the task list according to an ascending order to obtain a constraint value sequence, inquiring the preliminary planning task corresponding to the constraint values according to the constraint sequence, arranging the preliminary planning task according to the constraint value arrangement rule to obtain an arrangement task, generating task parameters of the arrangement task, inputting the task parameters into a pre-constructed component to obtain a task component, and constructing a task execution strategy of the preliminary planning task according to the arrangement order of the task component to obtain an optimal task strategy.
Wherein, the task list refers to a table for displaying data, and the component refers to an original for packaging the data.
Optionally, the task list of the preliminary planning task is created through excel, the constraint values are arranged in the task list according to ascending order, the obtained constraint value sequence is arranged according to a preset rule through a script generated by JAVA language according to the constraint values, and task parameters of the arranged task are compiled through binary codes.
And S6, performing task scheduling on the task processing request according to the optimal task strategy to obtain a task scheduling strategy, and performing task processing on the task processing request according to the task scheduling strategy.
According to the embodiment of the invention, the task processing request is subjected to task scheduling according to the optimal task strategy, so that a task scheduling strategy can be obtained to schedule a specific feasible task execution method and time for the task processing request, and a reliable execution scheme is provided for the task processing request, wherein the scheduling strategy refers to the priority of task execution, and comprises static task scheduling and dynamic task scheduling.
As an embodiment of the present invention, the task scheduling the task processing request according to the optimal task policy to obtain a task scheduling policy includes: and identifying parallel tasks in the optimal task strategy, adding parallel symbols for the task processing requests according to the parallel tasks, setting execution intervals for the task processing requests according to task time sequences in the optimal task strategy, setting trigger events of the task processing requests based on the parallel symbols and the execution intervals, and executing task scheduling of the task processing requests based on the trigger events to obtain a task scheduling strategy.
The parallel tasks refer to tasks which are set to be executed at the same time by the system, the parallel symbols refer to tools capable of receiving data simultaneously during data transmission, the execution interval refers to time intervals of task execution, such as 30s, 50s, 60s and the like, and the triggering event refers to that a certain condition system is reached to automatically execute a certain task.
Optionally, the parallel task is identified by querying a task tag of the optimal task policy, when executing nodes in the task tag are the same, the parallel task is represented, the parallel character is added through a script generated by java language, the executing interval is set through a java language construction time script, and the triggering event is set through an annotation character added in a task processing tag of the task processing request by Python language.
Furthermore, in the embodiment of the present invention, the task processing of the task processing request may be implemented by performing the task processing of the task processing request according to the task scheduling policy.
According to the scheme, the specificity of all tasks can be known by identifying the request characteristics of the task processing requests, so that the tasks are better distributed, and the task processing requests are subjected to task classification based on the request characteristics, so that a large number of task requests can be classified according to the classification tasks, and the execution priority of each task can be clearly known; secondly, the embodiment of the invention can know the execution time of each task by inquiring the processing node of each task in the hierarchical tasks, thereby better distributing the processing sequence of each task, wherein the processing node refers to the deadline of task processing; computing resources of the pre-allocation tasks can know computing resources required by each task to be processed, so that reasonable computing resource allocation is performed on the pre-allocation tasks; further, the embodiment of the invention can know the serial port through which each task needs to pass when being processed by identifying the task distribution serial port of the pre-distribution task, so that the tasks can be used as similar tasks or processed simultaneously, the distribution amount of the computer tasks is reduced, the computing resources, the available resources and the distribution serial port are reduced, the constraint condition of the task which is used as the pre-distribution task can list the constraint condition which needs to be considered when the task is distributed, further, when the task is distributed, the reasonable task distribution can be carried out according to the constraint condition, task planning can be carried out on the pre-distribution task according to the constraint condition of the task, a task processing method which meets the condition constraint of an actual application scene can be obtained by primarily distributing the tasks, further, the task processing scheme which is better in the pre-distribution task can be distributed, the task scheduling request can be scheduled according to the optimal task scheduling policy, the task execution method and the task scheduling time which are practically feasible can be arranged for the task processing request are obtained, and a reliable static scheduling scheme which refers to the task execution policy is provided for the task processing request, wherein the task scheduling policy comprises the task scheduling policy and the priority scheduling policy. Therefore, the task batch processing method and the task batch processing system provided by the embodiment of the invention can improve the task batch processing efficiency.
FIG. 2 is a functional block diagram of a task batch processing method and system according to the present invention.
The task batch processing method and system 200 of the invention can be installed in electronic equipment. The task batch processing method and system can include a task ranking module 201, a task allocation module 202, a constraint query module 203, a constraint configuration module 204, a policy planning module 205, and a task execution module 206, depending on the functions implemented.
The module of the invention, which may also be referred to as a unit, refers to a series of computer program segments, which are stored in the memory of the electronic device, capable of being executed by the processor of the electronic device and of performing a fixed function.
In the embodiment of the present invention, the functions of each module/unit are as follows:
the task classification module 201 is configured to obtain a task processing request, identify a request feature of the task processing request, and perform task classification on the task processing request based on the request feature to obtain a classified task;
the task allocation module 202 is configured to query a processing node of each task in the hierarchical tasks, and perform task pre-allocation on the hierarchical tasks based on the processing node to obtain pre-allocation tasks;
the constraint query module 203 is configured to calculate a computing resource of the pre-allocation task, query an available resource of a resource allocation node corresponding to the task processing request, and identify a task allocation serial port of the pre-allocation task;
the constraint configuration module 204 is configured to use the computing resources, the available resources, and the allocation serial port as task constraint conditions of the pre-allocation task;
the policy planning module 205 is configured to perform task planning on the pre-allocation task according to the task constraint condition, obtain a preliminary allocation task, calculate a constraint value of the preliminary allocation task, and create an optimal task policy of the preliminary planning task according to the constraint value;
the task execution module 206 is configured to perform task scheduling on the task processing request according to the optimal task policy, obtain a task scheduling policy, and perform task processing on the task processing request according to the task scheduling policy.
In detail, the modules in the task batch processing method and system 200 in the embodiment of the present invention use the same technical means as the task batch processing method and system described in fig. 1, and can produce the same technical effects, which are not described herein.
The foregoing is only a specific embodiment of the invention to enable those skilled in the art to understand or practice the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. The task batch processing method and system based on artificial intelligence are characterized by comprising the following steps:
acquiring a task processing request, identifying request characteristics of the task processing request, and classifying the task processing request based on the request characteristics to obtain classified tasks;
inquiring a processing node of each task in the hierarchical tasks, and performing task pre-allocation on the hierarchical tasks based on the processing nodes to obtain pre-allocation tasks;
calculating the calculation resources of the pre-allocation task, inquiring available resources of the resource allocation node corresponding to the task processing request, and identifying a task allocation serial port of the pre-allocation task;
taking the computing resources, the available resources and the allocation serial ports as task constraint conditions of the pre-allocation task;
performing task planning on the pre-allocation task according to the task constraint conditions to obtain a preliminary allocation task, calculating a constraint value of the preliminary allocation task, and creating an optimal task strategy of the preliminary planning task according to the constraint value;
and carrying out task scheduling on the task processing request according to the optimal task strategy to obtain a task scheduling strategy, and carrying out task processing on the task processing request according to the task scheduling strategy.
2. The method of claim 1, wherein the identifying the request feature of the task processing request comprises:
decoding the task processing request to obtain a decoding task;
inquiring a request identifier of the decoding task, and identifying an annotation symbol in the request identifier;
and analyzing annotation semantics of the annotation symbol, and identifying request features of the task processing request according to the annotation semantics.
3. The method of claim 1, wherein the task ranking the task processing requests based on the request characteristics to obtain ranked tasks comprises:
according to the request characteristics, calculating task weights of the hierarchical tasks by using the following formula:
wherein the method comprises the steps ofG represents task weight of hierarchical tasks, n represents task number of hierarchical tasks, x 1 Representing a first hierarchical task, x 2 Representing a second hierarchical task, x n Represents the nth hierarchical task, f represents a weighted average, x i Representing an ith request feature;
classifying the task weight according to a preset weight interval to obtain a classified weight interval;
and grading the tasks in the task processing request according to the grades corresponding to the grading weight intervals to obtain graded tasks.
4. The method of claim 1, wherein the computing resources of the pre-allocation task comprises:
calculating the computing resource duty ratio of the pre-allocation task by using the following formula:
wherein T represents the computing resource duty ratio of the pre-allocation tasks, n represents the number of the pre-allocation tasks, S r The resource transmission rate of the resource allocation node for allocating resources for the r-th task in the pre-allocation tasks is represented, and r (t) represents the task cache resources of the r-th task in the pre-allocation tasks;
and obtaining the computing resources of the pre-allocation task according to the computing resource duty ratio.
5. The method of claim 1, wherein the identifying the task allocation serial port of the pre-allocation task comprises:
inquiring a transmission protocol of the pre-allocation task;
identifying the transmission ip of the pre-allocation task according to the transmission protocol;
and searching a task transmission channel of the pre-allocation task according to the transmission ip, and inquiring a task allocation serial port corresponding to the task transmission channel.
6. The method according to claim 1, wherein performing task planning on the pre-allocation task according to the task constraint condition to obtain a preliminary allocation task comprises:
constructing constraint parameters of the task constraint conditions and task parameters of the pre-allocation task;
setting the constraint parameter as a limiting condition, and calculating the adaptability of the task parameter to the limiting condition by using the following formula:
wherein S represents fitness, y i Representing computational resource constraints in a defined condition, y j Representing available resource limits in a defined condition, y k Represents the distribution serial port limitation in the limiting condition, exp represents the global search function, m represents the parameter quantity of the task parameters, T C Representing a c-th parameter of the task parameters;
based on the adaptability, carrying out parameter search on the task parameters to obtain target parameters;
and carrying out parameter receipt on the pre-allocation task according to the target parameter to obtain a preliminary allocation task.
7. The method of claim 1, wherein said calculating constraint values for said preliminary allocation tasks comprises:
calculating constraint values of the preliminary allocation tasks by using the following formula:
wherein Y represents constraint value, f (Y) represents constraint cost function, delta represents constraint relation among constraint conditions, m represents task number of preliminary allocation tasks, and L q Representing preliminary allocation tasksThe q-th task in (a).
8. The method of claim 1, wherein the creating an optimal task strategy for the preliminary planning task based on the constraint values comprises:
creating a task list of the preliminary planning task, and arranging the constraint values in the task list according to an ascending order to obtain a constraint value sequence;
inquiring the preliminary planning task corresponding to the constraint value according to the constraint sequence, and arranging the preliminary planning task according to the constraint value arrangement rule to obtain an arrangement task;
generating task parameters of the arrangement tasks, and inputting the task parameters into a pre-constructed component to obtain a task component;
and constructing a task execution strategy of the preliminary planning task according to the arrangement sequence of the task components to obtain an optimal task strategy.
9. The method of claim 1, wherein performing task scheduling on the task processing request according to the optimal task policy to obtain a task scheduling policy, comprises:
identifying parallel tasks in the optimal task strategy;
adding a parallel character to the task processing request according to the parallel task, and setting an execution interval for the task processing request according to a task time sequence in the optimal task strategy;
and setting a trigger event of the task processing request based on the parallel character and the execution interval, and carrying out task scheduling on the task processing request based on the trigger event to obtain a task scheduling strategy.
10. An artificial intelligence based task batch processing method and system, characterized in that it is used for executing the artificial intelligence based task batch processing method according to any one of claims 1-9, the system comprises:
the task classification module is used for acquiring task processing requests, identifying request characteristics of the task processing requests, and classifying the task processing requests based on the request characteristics to obtain classified tasks;
the task allocation module is used for inquiring the processing node of each task in the hierarchical tasks, and performing task pre-allocation on the hierarchical tasks based on the processing nodes to obtain pre-allocation tasks;
the constraint query module is used for calculating the calculation resources of the pre-allocation task, querying available resources of the resource allocation node corresponding to the task processing request and identifying a task allocation serial port of the pre-allocation task;
the constraint configuration module is used for taking the computing resources, the available resources and the allocation serial ports as task constraint conditions of the pre-allocation task;
the strategy planning module is used for carrying out task planning on the pre-allocation task according to the task constraint condition to obtain a preliminary allocation task, calculating a constraint value of the preliminary allocation task, and creating an optimal task strategy of the preliminary planning task according to the constraint value;
and the task execution module is used for carrying out task scheduling on the task processing request according to the optimal task strategy to obtain a task scheduling strategy, and carrying out task processing on the task processing request according to the task scheduling strategy.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310722194.5A CN116700929A (en) | 2023-06-16 | 2023-06-16 | Task batch processing method and system based on artificial intelligence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310722194.5A CN116700929A (en) | 2023-06-16 | 2023-06-16 | Task batch processing method and system based on artificial intelligence |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116700929A true CN116700929A (en) | 2023-09-05 |
Family
ID=87825409
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310722194.5A Withdrawn CN116700929A (en) | 2023-06-16 | 2023-06-16 | Task batch processing method and system based on artificial intelligence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116700929A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117170832A (en) * | 2023-06-21 | 2023-12-05 | 海南歪酷网络科技有限公司 | Task batch processing method and system based on artificial intelligence |
-
2023
- 2023-06-16 CN CN202310722194.5A patent/CN116700929A/en not_active Withdrawn
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117170832A (en) * | 2023-06-21 | 2023-12-05 | 海南歪酷网络科技有限公司 | Task batch processing method and system based on artificial intelligence |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6637620B2 (en) | Agent allocation method, apparatus, server and recording medium | |
| US20190310977A1 (en) | Bucket data distribution for exporting data to worker nodes | |
| US6510457B1 (en) | Data analysis method and apparatus for data mining | |
| CN107590188B (en) | Crawler crawling method and management system for automatic vertical subdivision field | |
| CN104679595B (en) | A kind of application oriented IaaS layers of dynamic resource allocation method | |
| CN111160658B (en) | Collaborative manufacturing resource optimization method, system and platform | |
| CN109299157A (en) | A kind of data export method and device of distributed big single table | |
| CN104021125A (en) | Search engine sorting method and system and search engine | |
| CN116700929A (en) | Task batch processing method and system based on artificial intelligence | |
| CN111125518A (en) | System and method for recommending household appliance information | |
| CN105786941B (en) | Information mining method and device | |
| CN109885729B (en) | Method, device and system for displaying data | |
| US8224933B2 (en) | Method and apparatus for case-based service composition | |
| CN113806446A (en) | Rapid retrieval method for mass data of big data | |
| US20060224431A1 (en) | Data processing method, system and computer program | |
| Cheung et al. | Dynamic resource selection for service composition in the grid | |
| Cheung et al. | Towards autonomous service composition in a grid environment | |
| CN110188258B (en) | Method and device for acquiring external data by using crawler | |
| CN111913784B (en) | Task scheduling method and device, network element and storage medium | |
| CN117170832A (en) | Task batch processing method and system based on artificial intelligence | |
| CN114443293A (en) | Deployment system and method for big data platform | |
| CN113283742A (en) | Task allocation method and device | |
| CN107070987A (en) | Data capture method and system for distributed objects storage system | |
| Zhu et al. | Optimal bandwidth allocation for web crawler systems | |
| CN111740871A (en) | Data acquisition method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20230905 |