CN116126496B - Resource scheduling method and device for image data processing system - Google Patents
Resource scheduling method and device for image data processing system Download PDFInfo
- Publication number
- CN116126496B CN116126496B CN202211733630.0A CN202211733630A CN116126496B CN 116126496 B CN116126496 B CN 116126496B CN 202211733630 A CN202211733630 A CN 202211733630A CN 116126496 B CN116126496 B CN 116126496B
- Authority
- CN
- China
- Prior art keywords
- task
- event
- processed
- image data
- processing system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims abstract description 163
- 238000000034 method Methods 0.000 title claims abstract description 54
- 230000006870 function Effects 0.000 claims description 55
- 238000005457 optimization Methods 0.000 claims description 33
- 238000004422 calculation algorithm Methods 0.000 claims description 13
- 230000007246 mechanism Effects 0.000 claims description 13
- 239000000725 suspension Substances 0.000 claims description 12
- 239000011159 matrix material Substances 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 claims 4
- 239000010931 gold Substances 0.000 claims 4
- 229910052737 gold Inorganic materials 0.000 claims 4
- 238000009825 accumulation Methods 0.000 abstract description 11
- 230000007774 longterm Effects 0.000 abstract description 9
- 238000010586 diagram Methods 0.000 description 19
- 238000004590 computer program Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 8
- 238000011156 evaluation Methods 0.000 description 8
- 238000000605 extraction Methods 0.000 description 5
- 230000004044 response Effects 0.000 description 5
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 239000002699 waste material Substances 0.000 description 3
- 238000012952 Resampling Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000002068 genetic effect Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000002245 particle Substances 0.000 description 2
- 101000822695 Clostridium perfringens (strain 13 / Type A) Small, acid-soluble spore protein C1 Proteins 0.000 description 1
- 101000655262 Clostridium perfringens (strain 13 / Type A) Small, acid-soluble spore protein C2 Proteins 0.000 description 1
- 101000655256 Paraclostridium bifermentans Small, acid-soluble spore protein alpha Proteins 0.000 description 1
- 101000655264 Paraclostridium bifermentans Small, acid-soluble spore protein beta Proteins 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/48—Indexing scheme relating to G06F9/48
- G06F2209/484—Precedence
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Accessory Devices And Overall Control Thereof (AREA)
Abstract
The application discloses a resource scheduling method, device and equipment of an image data processing system and a storage medium. Wherein the method comprises the following steps: acquiring a plurality of task attributes of a plurality of events to be processed; priority scoring is carried out on the preset task characteristic dimension based on a plurality of task attributes, and a task characteristic scoring result of each event to be processed is obtained; acquiring an optimal resource scheduling strategy based on a task characteristic scoring result of each event to be processed; and scheduling the resources of the image data processing system based on the optimal resource scheduling strategy. By the technical scheme, resource scheduling management can be performed according to the task attribute of the event to be processed, the task accumulation rate when short-term tasks are more is reduced, and the utilization rate of long-term computing resources is improved.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a resource scheduling method and apparatus for an image data processing system.
Background
As the variety, source, and amount of data of images continue to increase, the amount of data and tasks that an image data processing system needs to process also continues to increase. To increase the processing efficiency as much as possible, reasonable scheduling of the resources of the system is required. The resource scheduling method of the existing image data processing system has the following defects: when the short-term task amount is large, as the computing resources cannot be expanded infinitely, the situation that the processing tasks cannot be executed in time due to insufficient computing resources exists, a large amount of processing tasks are accumulated, and particularly the completion timeliness of emergency tasks is affected; strategies to increase computing resources are employed to address the problem of multitasking in the short term, but over long periods of time, if the amount of tasks is relatively insufficient, it may result in lower computing resource utilization; for emergency tasks, only a high-priority scheduling strategy is designed, namely, the resource requirement of the emergency tasks is guaranteed by adopting a method for stopping executing the existing tasks, so that not only is the waste of computing resources caused, but also the stability of processing the conventional tasks can be affected.
Disclosure of Invention
The application provides a resource scheduling method, device and equipment of an image data processing system and a storage medium. The resource scheduling management can be performed according to the task attribute of the event to be processed, so that the task accumulation rate of a short-term task with more tasks is reduced, and the utilization rate of long-term computing resources is improved.
In a first aspect, an embodiment of the present application provides a resource scheduling method of an image data processing system, including: acquiring a plurality of task attributes of a plurality of events to be processed; priority scoring is carried out on the preset task characteristic dimension based on the task attributes, and a task characteristic scoring result of each event to be processed is obtained; acquiring an optimal resource scheduling strategy based on the task characteristic scoring result of each event to be processed; and scheduling the resources of the image data processing system based on the optimal resource scheduling strategy.
In the technical scheme, the task characteristic scoring result of the event to be processed can be obtained according to the task attribute of the event to be processed, so that an optimal resource scheduling strategy is obtained according to the task characteristic scoring result of each event to be processed, so that the resources of the image data processing system are scheduled, the task accumulation rate when short-term tasks are more is reduced, and the utilization rate of long-term computing resources is improved.
In one implementation, the task attributes include at least one of an image type, an image resolution, and a processing task type.
In one implementation, the task characteristic dimension includes at least one of resource occupancy, processing duration, and urgency.
In an optional implementation manner, the task attributes of each event to be processed include multiple types, the scoring of priorities in a preset task characteristic dimension based on the multiple task attributes, and obtaining a task characteristic scoring result of each event to be processed includes: acquiring a plurality of target task priorities matched with the task attribute of each event to be processed from a plurality of preset task priorities of the task characteristic dimension; acquiring a plurality of task characteristic scores corresponding to the target task priorities of each event to be processed; and selecting the largest task characteristic score from the plurality of task characteristic scores of each event to be processed as the task characteristic score result of each event to be processed in the task characteristic dimension.
In the technical scheme, one of a plurality of task characteristic scores of the event to be processed in one task dimension can be selected as a task characteristic score result of the event to be processed, so that an optimal resource scheduling strategy is obtained according to the task characteristic score result of each event to be processed, resources of an image data processing system are scheduled, the task accumulation rate when short-term tasks are more is reduced, and the utilization rate of long-term computing resources is improved.
In one implementation, the obtaining the optimal resource scheduling policy based on the task characteristic scoring result of each of the events to be processed includes: acquiring an optimized task characteristic dimension objective function of the task characteristic dimension based on the task characteristic scoring result of each event to be processed; acquiring a computing resource optimization objective function of the image data processing system based on the optimization task characteristic dimension objective function; and solving the computing resource optimization objective function to obtain the optimal resource scheduling strategy.
According to the technical scheme, task characteristic scoring results of the event to be processed on different task characteristic dimensions can be obtained, an optimized task characteristic dimension objective function is obtained, and based on the optimized task characteristic dimension objective function, a computing resource optimization objective function of the image data processing system is obtained, and then an optimal resource scheduling strategy is obtained, so that resources of the image data processing system are scheduled, the task accumulation rate when short-term tasks are more is reduced, and the utilization rate of long-term computing resources is improved.
In one implementation, the event to be processed is a first event, and the method further includes: performing independent modularized division on the first event to obtain a plurality of independent modules connected in series; acquiring a plurality of data interface relations among the plurality of independent modules; responsive to receiving a suspension execution instruction, storing output data and data interface information of an operating module of the plurality of serially connected independent modules based on the plurality of data interface relationships; responsive to receiving a continue execution instruction, continuing to execute the first event based on the output data and the data interface information; wherein the first event satisfies the following condition: the resource occupancy is greater than or equal to a first threshold and the processing time period is greater than or equal to a second threshold.
In one implementation, the method further comprises: and responding to the event to be processed as a special type event, acquiring a preset grading value of the event to be processed, and taking the preset grading value as the grading result of the task characteristic of the event to be processed.
In a second aspect, an embodiment of the present application provides a resource scheduling apparatus of an image data processing system, including: the acquisition module is used for acquiring a plurality of task attributes of a plurality of events to be processed; the first processing module is used for carrying out priority grading on the preset task characteristic dimension based on the task attributes and obtaining a task characteristic grading result of each event to be processed; the second processing module is used for acquiring an optimal resource scheduling strategy based on the task characteristic scoring result of each event to be processed; and the scheduling module is used for scheduling the resources of the image data processing system based on the optimal resource scheduling strategy.
In one implementation, the task attributes include at least one of an image type, an image resolution, and a processing task type.
In one implementation, the task characteristic dimension includes at least one of resource occupancy, processing duration, and urgency.
In an alternative implementation manner, the task attribute of each event to be processed includes a plurality of types, and the first processing module is specifically configured to: acquiring a plurality of target task priorities matched with the task attribute of each event to be processed from a plurality of preset task priorities of the task characteristic dimension; acquiring a plurality of task characteristic scores corresponding to the target task priorities of each event to be processed; and selecting the largest task characteristic score from the plurality of task characteristic scores of each event to be processed as the task characteristic score result of each event to be processed in the task characteristic dimension.
In one implementation, the second processing module is specifically configured to: acquiring an optimized task characteristic dimension objective function of the task characteristic dimension based on the task characteristic scoring result of each event to be processed; acquiring a computing resource optimization objective function of the image data processing system based on the optimization task characteristic dimension objective function; and solving the computing resource optimization objective function to obtain the optimal resource scheduling strategy.
In one implementation, the event to be processed is a first event, and the apparatus further includes: the third processing module is used for independently and modularly dividing the first event to obtain a plurality of independent modules which are connected in series; acquiring a plurality of data interface relations among the plurality of independent modules; responsive to receiving a suspension execution instruction, storing output data and data interface information of an operating module of the plurality of serially connected independent modules based on the plurality of data interface relationships; responsive to receiving a continue execution instruction, continuing to execute the first event based on the output data and the data interface information; wherein the first event satisfies the following condition: the resource occupancy is greater than or equal to a first threshold and the processing time period is greater than or equal to a second threshold.
In the technical scheme, the first event meeting the preset condition can be divided into a plurality of independent modules, so that when a suspending execution instruction is received, the first event is suspended and processed based on the plurality of task modules, the rapid execution of the emergency task is ensured, and when a continuous execution instruction is received, the first event is continuously executed, and the waste of system resources caused by re-executing the first event is avoided.
In one implementation, the apparatus further comprises: the fourth processing module is used for responding to the event to be processed as a special type event, obtaining a preset grading value of the event to be processed, and taking the preset grading value as the grading result of the task characteristic of the event to be processed.
In a third aspect, an embodiment of the present application provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of scheduling resources of an image data processing system according to the first aspect.
In a fourth aspect, an embodiment of the present application provides a computer readable storage medium storing instructions that, when executed, cause a method as described in the first aspect to be implemented.
In a fifth aspect, an embodiment of the present application provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the resource scheduling method of an image data processing system as described in the first aspect.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the application or to delineate the scope of the application. Other features of the present application will become apparent from the description that follows.
Drawings
The drawings are included to provide a better understanding of the present application and are not to be construed as limiting the application. Wherein:
FIG. 1 is a schematic diagram of a resource scheduling method of an image data processing system according to an embodiment of the present application;
FIG. 2 is a schematic diagram of another method for scheduling resources of an image data processing system according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a resource scheduling method of yet another image data processing system according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a resource scheduling method of yet another image data processing system according to an embodiment of the present application;
FIG. 5 is a flow chart of a resource scheduling scheme for an image data processing system provided by an embodiment of the present application;
FIG. 6 is a schematic diagram of a conventional high memory long-lived event suspension mechanism according to an embodiment of the present application;
FIG. 7 is a schematic diagram of a resource scheduling apparatus of an image data processing system according to an embodiment of the present application;
FIG. 8 is a schematic diagram of a resource scheduling apparatus of another image data processing system according to an embodiment of the present application;
FIG. 9 is a schematic diagram of a resource scheduling apparatus of yet another image data processing system provided by an embodiment of the present application;
FIG. 10 is a schematic block diagram of an example electronic device that may be used to implement embodiments of the present application.
Detailed Description
Exemplary embodiments of the present application will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present application are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Wherein, in the description of the present application, "/" means or is meant unless otherwise indicated, for example, a/B may represent a or B; "and/or" herein is merely an association relationship describing an association object, and means that three relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist together, and B exists alone. The first, second, etc. numbers referred to in the present application are merely for convenience of description and are not intended to limit the scope of the embodiments of the present application, nor represent the sequence.
Referring to fig. 1, fig. 1 is a schematic diagram of a resource scheduling method of an image data processing system according to an embodiment of the present application, as shown in fig. 1, the method may include, but is not limited to, the following steps.
Step S101: a plurality of task attributes for a plurality of events to be processed are obtained.
For example, the attribute of each of the plurality of events to be processed is analyzed to obtain the task attribute of each event to be processed, so as to obtain a plurality of task attributes of the plurality of events to be processed.
In one implementation, the task attributes include at least one of an image type, an image resolution, and a processing task type.
Step S102: and carrying out priority grading on the preset task characteristic dimension based on the plurality of task attributes, and obtaining a task characteristic grading result of each event to be processed.
In one implementation, the task property dimensions include at least one of resource occupancy, processing duration, and urgency.
For example, the task characteristic dimension may be divided into a plurality of priority levels in advance, where different priority levels respectively correspond to different task attributes, and a corresponding evaluation score is set for each priority level, and the specific dividing method may be as follows:
Where Q z is the resource occupancy, Q s is the processing duration, and Q j is the urgency. As described above, the task characteristic dimensions such as the resource occupancy, the processing time length, and the urgency may be divided into five levels of extremely high, medium, low, and extremely low, the evaluation score corresponding to the extremely high level is 2, the evaluation score corresponding to the high level is 1, the evaluation score corresponding to the medium level is 0, the evaluation score corresponding to the low level is-1, and the evaluation score corresponding to the extremely low level is-2. Therefore, the priority level corresponding to the task attribute can be obtained according to the task attribute of the event to be processed, and the evaluation score corresponding to the priority level corresponding to the task attribute is used as a task characteristic scoring result.
It should be noted that, since the number of specific cases included in a task attribute may be smaller than the number of priority levels, a corresponding level may be set for each specific case of a task attribute.
As an example, taking the image resolution task attribute as an example, the task attribute may include three specific cases of low resolution regular data, medium resolution regular data and high resolution regular data, for the resource occupancy task characteristic dimension, the corresponding priority level may be set to be lower for the low resolution regular data, to be medium for the medium resolution regular data, and to be extremely high for the high resolution regular data.
It can be understood that when the task dimension is one, the task characteristic scoring result of each event to be processed is one; when the task dimension is multiple, the task characteristic scoring result of each event to be processed is also multiple.
As an example, taking an example that the task characteristic dimension includes the resource occupancy rate, assuming that the task attribute of an event to be processed is medium in the level corresponding to the task dimension of the resource occupancy rate, the task characteristic scoring result Q z =0 of the event to be processed in the task dimension of the resource occupancy rate.
As another example, taking three task characteristic dimensions including a resource occupancy rate, a processing duration and an urgency as an example, assuming that a task attribute of an event to be processed is medium at a level corresponding to the task dimension of the resource occupancy rate, higher at a level corresponding to the task dimension of the processing duration, and lower at a level corresponding to the task dimension of the urgency, a task characteristic scoring result Q z＝0,Qs＝1,Qj = -1 of the event to be processed is obtained.
Step S103: and acquiring an optimal resource scheduling strategy based on the task characteristic scoring result of each event to be processed.
For example, based on the task characteristic scoring result of each event to be processed, taking the accumulation condition of the task to be processed and the use condition of the computing resource at the current moment as constraint conditions, and acquiring an optimal resource scheduling strategy by adopting a preset optimization algorithm (such as an ant colony algorithm, a particle swarm algorithm, a genetic algorithm and the like).
Step S104: and scheduling the resources of the image data processing system based on the optimal resource scheduling strategy.
For example, the allocation of computing resources of the image data processing system and the event to be processed is performed based on an optimal resource scheduling policy.
By implementing the embodiment of the application, the task characteristic scoring result of the event to be processed can be obtained according to the task attribute of the event to be processed, so that the optimal resource scheduling strategy is obtained according to the task characteristic scoring result of each event to be processed, so as to schedule the resources of the image data processing system, reduce the task accumulation rate when the short-term tasks are more, and improve the utilization rate of long-term computing resources.
In an alternative implementation manner, the task attribute of each event to be processed includes multiple task scores corresponding to the task attribute of each event to be processed, and the largest task score is selected from the task feature scores as the task feature score result of the event to be processed. As an example, referring to fig. 2, fig. 2 is a schematic diagram of another resource scheduling method of an image data processing system according to an embodiment of the present application, and as shown in fig. 2, the method may include, but is not limited to, the following steps.
Step S201: a plurality of task attributes for a plurality of events to be processed are obtained.
In the embodiment of the present application, step S201 may be implemented in any manner of each embodiment of the present application, which is not limited to this embodiment, and is not repeated.
Step S202: and acquiring a plurality of target task priorities matched with the task attribute of each event to be processed from a plurality of preset task priorities of the task characteristic dimension.
Example 1, taking the example that the task characteristic dimensions include the resource occupancy rate, the processing duration and the urgency, the task attributes include the image type, the image resolution and the processing task type, the image type includes the gray type, the three-band true color and the four-band false color, the image resolution includes the low resolution regular data, the medium resolution regular data and the high resolution regular data, the processing task type includes the data primary processing, the data base processing, the resampling fine processing and the information extraction, and each specific task attribute is set with a corresponding level and evaluation score in each task characteristic dimension as shown in table 1:
TABLE 1 task Property and task Property dimension partitioning Table
| Image type | Resource occupancy rate Q z1 | Processing duration Q s1 | Urgency Q j1 |
| Gray scale image | Lower/-1 | Lower/-1 | Higher/1 |
| Three-band true color | Medium/0 | Medium/0 | Medium/0 |
| Four-band pseudo-color | Higher/1 | Higher/1 | Lower/-1 |
| Image resolution | Resource occupancy rate Q z2 | Processing duration Q s2 | Urgency Q j2 |
| Low resolution conventional data | Lower/-1 | Lower/-1 | Lower/-1 |
| Medium resolution conventional data | Medium/0 | Medium/0 | Higher/1 |
| High resolution conventional data | Extremely high/2 | Higher/1 | Lower/-1 |
| Processing task type | Resource occupancy rate Q z3 | Processing duration Q s3 | Urgency Q j3 |
| Data primary processing and data base processing | Lower/-1 | Higher/1 | Medium/0 |
| Resampling refinement | Higher/1 | Extremely high/2 | Medium/0 |
| Information extraction | Medium/0 | Lower/-1 | Extremely high/2 |
Assuming that the task attribute of an event to be processed is three-band true color, high-resolution conventional data and information extraction, the task priority corresponding to the image type task attribute of the event to be processed is medium in the resource occupancy rate task characteristic dimension, the task priority corresponding to the image resolution task attribute is extremely high, and the task priority corresponding to the processing task type is medium; the task priority corresponding to the image type task attribute of the event to be processed is medium in the processing time length task characteristic dimension, the task priority corresponding to the image resolution task attribute is higher, and the task priority corresponding to the processing task type is lower; the task priority corresponding to the image type task attribute of the event to be processed is medium in the urgent task characteristic dimension, the task priority corresponding to the image resolution task attribute is low, and the task priority corresponding to the processing task type is extremely high.
Step S203: and obtaining a plurality of task characteristic scores corresponding to the target task priorities of each event to be processed.
Example 2, as described in example 1, assuming that the task attribute of one event to be processed is three-band true color, high-resolution conventional data and information extraction, the task characteristic scores of the resource occupancy task characteristic dimension of the event to be processed are Q z1＝0、Qz2 =2 and Q z3 =0, the task characteristic scores of the processing duration task dimension of the event to be processed are Q s1＝0、Qs2 =1 and Q s3 = -1, and the task characteristic scores of the urgent task dimension of the event to be processed are Q s1＝0、Qs2 = -1 and Q s3 =2.
Step S204: and selecting the largest one from the multiple task characteristic scores of each event to be processed as a task characteristic score result of each event to be processed in a task characteristic dimension.
Example 3, as described in examples 1 and 2, assuming that the task attribute of an event to be processed is three-band true color, high-resolution conventional data and information extraction, the task characteristic scoring result of the event to be processed in the resource occupancy task characteristic dimension is:
Wherein, To take the maximum value. The task characteristic scoring result of the event to be processed in the processing time length task dimension is as follows:
The task characteristic scoring result of the event to be processed in the urgent task dimension is as follows:
Step S205: and acquiring an optimal resource scheduling strategy based on the task characteristic scoring result of each event to be processed.
In the embodiment of the present application, step S205 may be implemented in any one of the embodiments of the present application, which is not limited to this embodiment, and is not repeated.
Step S206: and scheduling the resources of the image data processing system based on the optimal resource scheduling strategy.
In the embodiment of the present application, step S206 may be implemented in any one of the embodiments of the present application, which is not limited to this embodiment, and is not repeated.
By implementing the embodiment of the application, one of a plurality of task characteristic scores of the event to be processed in one task dimension can be selected as the task characteristic score result of the event to be processed, so that the optimal resource scheduling strategy is obtained according to the task characteristic score result of each event to be processed, the resources of the image data processing system are scheduled, the task accumulation rate when short-term tasks are more is reduced, and the utilization rate of long-term computing resources is improved.
In one implementation, the optimal task characteristic dimension objective function may be obtained according to a task characteristic scoring result of the event to be processed, and the computing resource optimization objective function of the image data processing system may be obtained according to the optimal task characteristic dimension objective function, so as to obtain the optimal resource scheduling policy according to the computing resource optimization objective function of the image data processing system. As an example, referring to fig. 3, fig. 3 is a schematic diagram of a resource scheduling method of another image data processing system according to an embodiment of the present application, as shown in fig. 3, the method may include, but is not limited to, the following steps.
Step S301: a plurality of task attributes for a plurality of events to be processed are obtained.
In the embodiment of the present application, step S301 may be implemented by any one of the embodiments of the present application, which is not limited to this embodiment, and is not described in detail.
Step S302: and carrying out priority grading on the preset task characteristic dimension based on the plurality of task attributes, and obtaining a task characteristic grading result of each event to be processed.
In the embodiment of the present application, step S302 may be implemented in any manner of each embodiment of the present application, which is not limited to this embodiment, and is not repeated.
Step S303: and acquiring an optimized task characteristic dimension objective function of the task characteristic dimension based on the task characteristic scoring result of each event to be processed.
Example 4, assuming that there are m events to be processed and n processing resources of the device in the image data processing system, denoted as r= { R 1,r2,...,rm }, the image data processing system computing resource set d= { D 1,d2,...,dn},Mrd is a mapping relationship between the events to be processed and the computing resources. The execution time matrix for distributing m events to be processed to n devices through a resource scheduling algorithm is as follows:
ETMmn＝ETD(riMrd,dk)
Wherein ETM mn is an execution time matrix, i is greater than or equal to 1 and less than or equal to m, k is greater than or equal to 1 and less than or equal to n, the matrix represents a time duration process of distributing an event r i to be processed to the device d k through a task scheduling distribution mechanism, and a processing time duration scoring result Q s can be used for calculation conveniently. The earliest completion time of the pending event r i on the computing device d k by the resource scheduling map M rd can be expressed as:
Finish(riMrd,dk)＝Start(dk)+ETD(riMrd,dk)
Wherein Finish (r iMrd,dk) is the earliest completion time, start (d k) represents the earliest Start time of a task that can be performed on physical device d k, and the total time it takes to allocate the task to be performed on physical device d k can be expressed as:
Where Sum (d k) is the total time it takes to allocate task execution on physical device d k, c ik =1 indicates that pending event r i is executing on device d k, and c ik =0 indicates that pending event r i is not executing on device d k. Thus, the total time for all tasks to execute on individual computing resources through the resource scheduling assignment mechanism can be expressed as:
Total (T) is the total time that all tasks are executed on each computing resource through a resource scheduling allocation mechanism. It will be appreciated that one basic principle of rational scheduling of image data processing systems is that the total execution time is the shortest, i.e. the value of total (T) in the above equation is the smallest, and thus the optimal time efficiency objective function can be expressed as:
Because the image data processing system has the characteristic of multi-task parallelism, in order to reduce task accumulation, events with low resource occupation are preferentially arranged under the condition of meeting the maximum computing resource limitation, and therefore, the optimized resource occupation objective function can be expressed as:
Wherein n' is the number of event tasks processed in parallel, Z max is the total computation resource limit of the image data processing system, and Q zk is the task characteristic scoring result of the kth event to be processed in the resource occupancy rate task characteristic dimension. In connection with an event task urgency scoring system, the optimization urgency objective function may be expressed as:
Gol (J) =maxq jk,Qjk is the task property scoring result of the kth event to be processed in the urgent task dimension.
Step S304: based on the optimized task characteristic dimension objective function, a computing resource optimization objective function of the image data processing system is obtained.
Example 5, according to the optimization time efficiency objective function, the optimization resource occupancy objective function, and the optimization urgency objective function obtained in example 4, the computing resource optimization objective function of the image data processing system may be expressed as:
Goal(Y)＝max(wjGoal(J)+wzGoal(Z)-wtGoal(T))
Wherein w j is task urgency weight, w z is computing resource weight, w t is processing duration weight, and the weight can be adjusted according to business requirements.
Step S305: and solving the computing resource optimization objective function to obtain an optimal resource scheduling strategy.
For example, a preset optimization algorithm (such as an ant colony algorithm, a particle swarm algorithm, or a genetic algorithm) is used to solve the computing resource optimization objective function, and an optimal resource scheduling strategy is obtained based on the result of the solution.
Step S306: and scheduling the resources of the image data processing system based on the optimal resource scheduling strategy.
By implementing the embodiment of the application, task characteristic scoring results of the event to be processed on different task characteristic dimensions can be obtained, the optimized task characteristic dimension objective function is obtained, the computing resource optimization objective function of the image data processing system is obtained based on the optimized task characteristic dimension objective function, and then the optimal resource scheduling strategy is obtained, so that the resources of the image data processing system are scheduled, the task accumulation rate when short-term tasks are more is reduced, and the utilization rate of long-term computing resources is improved.
In one implementation, a suspension mechanism may be set for conventional events with a high occupancy rate of resources and a long processing time required to ensure rapid execution of emergency tasks. As an example, referring to fig. 4, fig. 4 is a schematic diagram of a resource scheduling method of another image data processing system according to an embodiment of the present application, as shown in fig. 4, the method may include, but is not limited to, the following steps.
Step S401: independent modularized division is carried out on the first event, and a plurality of independent modules connected in series are obtained.
Wherein, in an embodiment of the present application, the first event satisfies the following condition: the resource occupancy is greater than or equal to a first threshold and the processing time period is greater than or equal to a second threshold.
For example, a first event having a resource occupancy greater than or equal to a first threshold and a processing time period greater than or equal to a second threshold is independently and modularly divided to divide the first event into a plurality of independent modules that may be executed in series.
Step S402: and acquiring a plurality of data interface relations among a plurality of independent modules.
For example, the data interface relationship between each two adjacent modules in the plurality of independent modules connected in series is analyzed to obtain a plurality of data interface relationships.
Step S403: and in response to receiving the suspending execution instruction, storing output data and data interface information of the running module in the plurality of independent modules in series based on the plurality of data interface relations.
For example, when an emergency occurs in the event to be processed, a suspension execution instruction may be sent to the device executing the first event, after the running module in the multiple independent modules is finished running in response to receiving the suspension execution instruction, output data of the running module is saved, and corresponding data interface information is saved according to a data interface relationship between the running module and an adjacent next module in the multiple data interface relationships.
Step S404: in response to receiving the continue execution instruction, the first event is continued to be executed based on the output data and the data interface information.
For example, after the emergency processing is completed, a continue execution instruction may be sent to the device executing the first event, and in response to receiving the continue execution instruction, the stored output data and the data interface information are read according to the agreed format, and the independent modules that are not executed are continuously executed for the event to be processed, so that the event to be processed is continuously executed.
By implementing the embodiment of the application, the first event meeting the preset condition can be divided into a plurality of independent modules, so that when the suspending execution instruction is received, the first event is suspended and processed based on the plurality of task modules, thereby ensuring the rapid execution of the emergency task, and when the continuing execution instruction is received, the first event is continuously executed, and avoiding the waste of system resources caused by re-executing the first event.
In some embodiments of the present application, the method for scheduling resources of an image data processing system may further include: and responding to the event to be processed as a special type event, acquiring a preset grading value of the event to be processed, and taking the preset grading value as a task characteristic grading result of the event to be processed.
For example, for a to-be-processed event of a special event type that does not meet the conventional partition specification, the specific score value of the to-be-processed event may be designated as a task characteristic scoring result, so as to increase the accuracy of the event scoring and the flexibility of task resource scheduling.
Referring to fig. 5, fig. 5 is a flowchart of a resource scheduling scheme of an image data processing system according to an embodiment of the present application, as shown in fig. 5, the scheme may include the following steps:
Step S501: and carrying out three-dimensional five-level classification on the task attribute of the system processing event according to the dimensions such as the resource occupancy rate, the processing time length, the task urgency and the like.
For example, different specific situations of the same task attribute of an event needing to be processed by the system are classified into five levels from low to high in three dimensions of resource occupancy, processing duration, task urgency and the like.
Step S502: and establishing a unified priority scoring rule, and carrying out unified priority scoring on the event task to be processed to respectively obtain an occupancy rate score Q z, a processing duration score Q s and an urgency score Q j of the event.
For example, according to the levels of the division in step 501, different scores are set for each level to establish a unified priority scoring rule, so that the event to be processed is scored in priority according to the priority scoring rule based on the task attribute of the event to be processed, and the occupancy rate score Q z, the processing duration score Q s and the urgency score Q j of the event are respectively obtained.
Step S503: and according to the task attribute scores of the events, computing resources are automatically distributed to the events to be processed by adopting an optimization strategy, and task scheduling management is carried out.
For example, according to the occupancy rate score Q z, the processing duration score Q s, and the urgency score Q j obtained in step 502, an optimized objective function of the computing resource of the image data processing system is obtained, and a plurality of optimization algorithms are used for task scheduling management for the optimized objective function.
Step S504: and a suspension mechanism is arranged for the conventional high-memory long-time occupied event task, so that the rapid execution of the emergency event task is ensured.
Referring to fig. 6, fig. 6 is a schematic diagram of a conventional high-memory long-time-occupation event suspension mechanism according to an embodiment of the present application. As shown in fig. 6, first, conventional event tasks conforming to the long-time occupation characteristic of high memory are independently and modularly divided, so that the event tasks can be divided into independent modules which are executed in series; secondly, analyzing the data interface relation of the independent modules before and after the serial connection; and finally, adding a module-level suspension flow based on the original flow of the event task, suspending the execution flow by the operation module when the event task receives a suspension instruction (namely initiating an emergency task), storing the output data of the serial independent modules according to a stipulated format, continuously executing the flow by the operation module when the event task receives a continuous execution instruction (namely completing the execution of the emergency task), reading the stored data interface information according to the stipulated format, and continuously suspending the event task.
Referring to fig. 7, fig. 7 is a schematic diagram of a resource scheduling apparatus of an image data processing system according to an embodiment of the present application. As shown in fig. 7, the apparatus 700 includes: an obtaining module 701, configured to obtain a plurality of task attributes of a plurality of events to be processed; the first processing module 702 is configured to perform priority scoring on a preset task characteristic dimension based on a plurality of task attributes, and obtain a task characteristic scoring result of each event to be processed; a second processing module 703, configured to obtain an optimal resource scheduling policy based on a task characteristic scoring result of each event to be processed; a scheduling module 704, configured to schedule resources of the image data processing system based on an optimal resource scheduling policy.
In one implementation, the task attributes include at least one of an image type, an image resolution, and a processing task type.
In one implementation, the task characteristic dimension includes at least one of resource occupancy, processing duration, and urgency.
In an alternative implementation, the task attribute of each event to be processed includes a plurality of types, and the first processing module 702 is specifically configured to: acquiring a plurality of target task priorities matched with the task attribute of each event to be processed from a plurality of preset task priorities of the task characteristic dimension; acquiring a plurality of task characteristic scores corresponding to a plurality of target task priorities of each event to be processed; and selecting the largest one from the multiple task characteristic scores of each event to be processed as a task characteristic score result of each event to be processed in a task characteristic dimension.
In one implementation, the second processing module 703 is specifically configured to: acquiring an optimized task characteristic dimension objective function of the task characteristic dimension based on a task characteristic scoring result of each event to be processed; acquiring a computing resource optimization objective function of the image data processing system based on the optimization task characteristic dimension objective function; and solving the computing resource optimization objective function to obtain an optimal resource scheduling strategy.
In one implementation, the event to be processed is a first event, the apparatus further comprising: and a third processing module. As an example, please refer to fig. 8, fig. 8 is a schematic diagram of a resource scheduling apparatus of another image data processing system according to an embodiment of the present application. As shown in fig. 8, the apparatus 800 includes a third processing module 805 configured to perform independent modular division on the first event to obtain a plurality of independent modules connected in series; acquiring a plurality of data interface relations among a plurality of independent modules; responsive to receiving the suspension execution instruction, storing output data and data interface information of an operating module of the plurality of serially connected independent modules based on the plurality of data interface relationships; responsive to receiving a continue execution instruction, continuing to execute the first event based on the output data and the data interface information; wherein the first event satisfies the following condition: the resource occupancy is greater than or equal to a first threshold and the processing time period is greater than or equal to a second threshold. The modules 801 to 804 in fig. 8 have the same structure and function as the modules 701 to 704 in fig. 7.
In one implementation, the apparatus further comprises: and a fourth processing module. As an example, please refer to fig. 9, fig. 9 is a schematic diagram of a resource scheduling apparatus of another image data processing system according to an embodiment of the present application. As shown in fig. 9, the apparatus 900 includes a fourth processing module 905, configured to obtain a preset score value of the event to be processed, and take the preset score value as a task characteristic scoring result of the event to be processed, in response to the event to be processed being a special event. The modules 901 to 904 in fig. 9 have the same structure and function as the modules 701 to 704 in fig. 7.
By the device provided by the embodiment of the application, the task characteristic scoring result of the event to be processed can be obtained according to the task attribute of the event to be processed, so that the optimal resource scheduling strategy is obtained according to the task characteristic scoring result of each event to be processed, the resources of the image data processing system are scheduled, the task accumulation rate when short-term tasks are more is reduced, and the utilization rate of long-term computing resources is improved.
Based on the embodiments of the present application, the present application also provides a computer readable storage medium, in which computer instructions are used to make a computer execute the resource scheduling method of the image data processing system according to any of the foregoing embodiments provided by the embodiments of the present application.
Referring now to fig. 10, shown in fig. 10 is a schematic block diagram of an example electronic device that may be used to implement an embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the applications described and/or claimed herein.
As shown in fig. 10, the apparatus 1000 includes a computing unit 1001 that can perform various appropriate actions and processes according to a computer program stored in a Read-Only Memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access Memory (Random Access Memory, RAM) 1003. In the RAM 1003, various programs and data required for the operation of the device 1000 can also be stored. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other by a bus 1004. An Input/Output (I/O) interface 1005 is also connected to bus 1004.
Various components in device 1000 are connected to I/O interface 1005, including: an input unit 1006 such as a keyboard, a mouse, and the like; an output unit 1007 such as various types of displays, speakers, and the like; a storage unit 1008 such as a magnetic disk, an optical disk, or the like; and communication unit 1009 such as a network card, modem, wireless communication transceiver, etc. Communication unit 1009 allows device 1000 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunications networks.
The computing unit 1001 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 1001 include, but are not limited to, a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DIGITAL SIGNAL processes, DSPs), and any suitable processors, controllers, microcontrollers, etc. The computing unit 1001 performs the respective methods and processes described above, for example, a resource scheduling method of an image data processing system. For example, in some embodiments, the resource scheduling method of the image data processing system may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 1008. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 1000 via ROM 1002 and/or communication unit 1009. When a computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the resource scheduling method of the image data processing system described above can be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to perform the resource scheduling method of the image data processing system in any other suitable way (e.g. by means of firmware).
Various implementations of the systems and techniques described here above can be implemented in digital electronic circuitry, integrated Circuit System, field programmable gate array (Field Programmable GATE ARRAY, FPGA), application-specific integrated Circuit (ASIC), application-specific standard product (Application SPECIFIC STANDARD PARTS, ASSP), system-On-Chip (SOC), load-programmable logic device (Complex Programmable Logic Device, CPLD), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out methods of the present application may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present application, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-Only Memory (ROM), an erasable programmable read-Only Memory (EPROM) or flash Memory, an optical fiber, a portable compact disc read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a Cathode Ray Tube (CRT) or an LCD (Liquid CRYSTAL DISPLAY) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area networks (Local Area Network, LANs), wide area networks (Wide Area Network, WANs), the internet, and blockchain networks.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of high management difficulty and weak service expansibility in the traditional physical host and Virtual private server (VPS PRIVATE SERVER) service. The server may also be a server of a distributed system or a server that incorporates a blockchain.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially, or in a different order, provided that the desired results of the technical solution of the present application are achieved, and are not limited herein.
The above embodiments do not limit the scope of the present application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of the present application.
Claims (8)
1. A method for scheduling resources of an image data processing system, comprising:
acquiring a plurality of task attributes of a plurality of events to be processed;
priority scoring is carried out on the preset task characteristic dimension based on the task attributes, and a task characteristic scoring result of each event to be processed is obtained;
acquiring an optimal resource scheduling strategy based on the task characteristic scoring result of each event to be processed;
Scheduling resources of the image data processing system based on the optimal resource scheduling policy;
The step of scoring the priority on the preset task characteristic dimension based on the task attributes, and obtaining a task characteristic scoring result of each event to be processed comprises the following steps:
Acquiring a plurality of target task priorities matched with the task attribute of each event to be processed from a plurality of preset task priorities of the task characteristic dimension;
acquiring a plurality of task characteristic scores corresponding to the target task priorities of each event to be processed;
Selecting the largest one from the task characteristic scores of each event to be processed as the task characteristic score result of each event to be processed in the task characteristic dimension;
The obtaining the optimal resource scheduling policy based on the task characteristic scoring result of each event to be processed includes:
acquiring an optimized task characteristic dimension objective function of the task characteristic dimension based on the task characteristic scoring result of each event to be processed; defining processing resources of m to-be-processed events and n devices in the image data processing system, wherein the processing resources are represented as R= { R 1,r2,...,rm }, and the image data processing system calculates a mapping relation between a set of computing resources D= { D 1,d2,...,dn},Mrd and the to-be-processed events and the computing resources, and then allocates the m to-be-processed events to the execution time matrix on the n devices through a resource scheduling algorithm as follows:
ETMmn＝ETD(riMrd,dk)
Wherein ETM mn is an execution time matrix, i is more than or equal to 1 and less than or equal to m, k is more than or equal to 1 and less than or equal to n, the matrix represents a time length process of distributing an event r i to be processed to equipment d k for execution through a task scheduling distribution mechanism, and a processing time length scoring result Q s can be used for calculation; the earliest completion time of the pending event r i on the computing device d k by the resource scheduling map M rd can be expressed as:
Finish(riMrd,dk)＝Start(dk)+ETD(riMrd,dk)
Wherein Finish (r iMrd,dk) is the earliest completion time, start (d k) represents the earliest Start time of a task that can be performed on physical device d k, and the total time it takes to allocate the task to be performed on physical device d k can be expressed as:
wherein Sum (d k) is the total time spent on allocating task execution on the physical device d k, c ik =1 indicates that the event r i to be processed is executed on the device d k, and c ik =0 indicates that the event r i to be processed is not executed on the device d k; thus, the total time for all tasks to execute on individual computing resources through the resource scheduling assignment mechanism can be expressed as:
Total (T) is the total time of all tasks executed on each computing resource through a resource scheduling and distributing mechanism; one basic principle of reasonable scheduling of the image data processing system is that the total execution time is the shortest, that is, the value of total (T) in the above formula is the smallest, and thus, the optimized time efficiency objective function can be expressed as:
To reduce task stacking, the optimized resource occupation objective function can be expressed as:
Wherein n' is the number of event tasks processed in parallel, Z max is the total computation resource limit of the image data processing system, and Q zk is the task characteristic scoring result of the kth event to be processed in the resource occupancy rate task characteristic dimension; in connection with an event task urgency scoring system, the optimization urgency objective function may be expressed as: gol (J) =maxq jk,k∈(1,..,n),Qjk is the task characteristic scoring result of the kth event to be processed in the urgent task dimension;
Acquiring a computing resource optimization objective function of the image data processing system based on the optimization task characteristic dimension objective function: gold (Y) =max (w jGoal(J)+wzGoal(Z)-wt gold (T))
Wherein w j is task urgency weight, w z is computing resource weight, and w t is processing duration weight;
And solving the computing resource optimization objective function to obtain the optimal resource scheduling strategy.
2. The method of claim 1, wherein the task attributes include at least one of an image type, an image resolution, and a processing task type.
3. The method of claim 1, wherein the task property dimension comprises at least one of resource occupancy, processing duration, and urgency.
4. The method of claim 1, wherein the event to be processed is a first event, the method further comprising:
performing independent modularized division on the first event to obtain a plurality of independent modules connected in series;
Acquiring a plurality of data interface relations among the plurality of independent modules;
Responsive to receiving a suspension execution instruction, storing output data and data interface information of an operating module of the plurality of serially connected independent modules based on the plurality of data interface relationships;
Responsive to receiving a continue execution instruction, continuing to execute the first event based on the output data and the data interface information;
Wherein the first event satisfies the following condition: the resource occupancy is greater than or equal to a first threshold and the processing time period is greater than or equal to a second threshold.
5. The method of claim 1, wherein the method further comprises:
And responding to the event to be processed as a special type event, acquiring a preset grading value of the event to be processed, and taking the preset grading value as the grading result of the task characteristic of the event to be processed.
6. A resource scheduling apparatus of an image data processing system, comprising:
The acquisition module is used for acquiring a plurality of task attributes of a plurality of events to be processed;
the first processing module is used for carrying out priority grading on the preset task characteristic dimension based on the task attributes and obtaining a task characteristic grading result of each event to be processed;
the second processing module obtains an optimal resource scheduling strategy based on the task characteristic scoring result of each event to be processed;
the scheduling module is used for scheduling the resources of the image data processing system based on the optimal resource scheduling strategy;
The step of scoring the priority on the preset task characteristic dimension based on the task attributes, and obtaining a task characteristic scoring result of each event to be processed comprises the following steps:
Acquiring a plurality of target task priorities matched with the task attribute of each event to be processed from a plurality of preset task priorities of the task characteristic dimension;
acquiring a plurality of task characteristic scores corresponding to the target task priorities of each event to be processed;
Selecting the largest one from the task characteristic scores of each event to be processed as the task characteristic score result of each event to be processed in the task characteristic dimension;
The obtaining the optimal resource scheduling policy based on the task characteristic scoring result of each event to be processed includes:
acquiring an optimized task characteristic dimension objective function of the task characteristic dimension based on the task characteristic scoring result of each event to be processed; defining processing resources of m to-be-processed events and n devices in the image data processing system, wherein the processing resources are represented as R= { R 1,r2,...,rm }, and the image data processing system calculates a mapping relation between a set of computing resources D= { D 1,d2,...,dn},Mrd and the to-be-processed events and the computing resources, and then allocates the m to-be-processed events to the execution time matrix on the n devices through a resource scheduling algorithm as follows:
ETMmn＝ETD(riMrd,dk)，
Wherein ETM mn is an execution time matrix, i is more than or equal to 1 and less than or equal to m, k is more than or equal to 1 and less than or equal to n, the matrix represents a time length process of distributing an event r i to be processed to equipment d k for execution through a task scheduling distribution mechanism, and a processing time length scoring result Q s can be used for calculation; the earliest completion time of the pending event r i on the computing device d k by the resource scheduling map M rd can be expressed as:
Finish(riMrd,dk)＝Start(dk)+ETD(riMrd,dk)
Wherein Finish (r iMrd,dk) is the earliest completion time, start (d k) represents the earliest Start time of a task that can be performed on physical device d k, and the total time it takes to allocate the task to be performed on physical device d k can be expressed as:
wherein Sum (d k) is the total time spent on allocating task execution on the physical device d k, c ik =1 indicates that the event r i to be processed is executed on the device d k, and c ik =0 indicates that the event r i to be processed is not executed on the device d k; thus, the total time for all tasks to execute on individual computing resources through the resource scheduling assignment mechanism can be expressed as:
Total (T) is the total time of all tasks executed on each computing resource through a resource scheduling and distributing mechanism; one basic principle of reasonable scheduling of the image data processing system is that the total execution time is the shortest, that is, the value of total (T) in the above formula is the smallest, and thus, the optimized time efficiency objective function can be expressed as:
To reduce task stacking, the optimized resource occupation objective function can be expressed as:
wherein n' is the number of event tasks processed in parallel, Z max is the total computation resource limit of the image data processing system, and Q zk is the task characteristic scoring result of the kth event to be processed in the resource occupancy rate task characteristic dimension; in connection with an event task urgency scoring system, the optimization urgency objective function may be expressed as:
gol (J) =maxq jk,k∈(1,..,n),Qjk is the task characteristic scoring result of the kth event to be processed in the urgent task dimension;
Acquiring a computing resource optimization objective function of the image data processing system based on the optimization task characteristic dimension objective function: gold (Y) =max (w jGoal(J)+wzGoal(Z)-wt gold (T))
Wherein w j is task urgency weight, w z is computing resource weight, and w t is processing duration weight;
And solving the computing resource optimization objective function to obtain the optimal resource scheduling strategy.
7. An electronic device, comprising:
At least one processor;
And a memory communicatively coupled to the at least one processor;
Wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the resource scheduling method of the image data processing system of any one of claims 1 to 5.
8. A computer readable storage medium storing instructions which, when executed, cause the method of any one of claims 1 to 5 to be implemented.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211733630.0A CN116126496B (en) | 2022-12-30 | 2022-12-30 | Resource scheduling method and device for image data processing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211733630.0A CN116126496B (en) | 2022-12-30 | 2022-12-30 | Resource scheduling method and device for image data processing system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116126496A CN116126496A (en) | 2023-05-16 |
| CN116126496B true CN116126496B (en) | 2024-07-02 |
Family
ID=86304061
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211733630.0A Active CN116126496B (en) | 2022-12-30 | 2022-12-30 | Resource scheduling method and device for image data processing system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116126496B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104168318A (en) * | 2014-08-18 | 2014-11-26 | 中国联合网络通信集团有限公司 | Resource service system and resource distribution method thereof |
| CN115202847A (en) * | 2022-07-28 | 2022-10-18 | 北京京东振世信息技术有限公司 | Task scheduling method and device |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3367240A1 (en) * | 2017-02-27 | 2018-08-29 | Nokia Solutions and Networks Oy | Allocation method and allocation system for allocating resources to a data-based service |
| CN114327811A (en) * | 2021-11-19 | 2022-04-12 | 浪潮电子信息产业股份有限公司 | Task scheduling method, device and equipment and readable storage medium |
| CN114546646A (en) * | 2022-02-21 | 2022-05-27 | 联想(北京)有限公司 | Processing method and processing apparatus |
-
2022
- 2022-12-30 CN CN202211733630.0A patent/CN116126496B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104168318A (en) * | 2014-08-18 | 2014-11-26 | 中国联合网络通信集团有限公司 | Resource service system and resource distribution method thereof |
| CN115202847A (en) * | 2022-07-28 | 2022-10-18 | 北京京东振世信息技术有限公司 | Task scheduling method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116126496A (en) | 2023-05-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP4083802A1 (en) | Resource scheduling method and apparatus, device, storage medium, and program product | |
| CN106681835B (en) | The method and resource manager of resource allocation | |
| US20110202382A1 (en) | Workforce planning | |
| JP2015146154A (en) | Job scheduling apparatus, job scheduling method and job scheduling program | |
| CN113946431B (en) | Resource scheduling method, system, medium and computing device | |
| JP2017514247A (en) | Framework to optimize project selection and resource allocation within a structured management organization under time, resource and budget constraints | |
| WO2019205370A1 (en) | Electronic device, task distribution method and storage medium | |
| CN115064253B (en) | Medical equipment scheduling method, device, equipment and storage medium | |
| CN111190712A (en) | Task scheduling method, device, equipment and medium | |
| US10423904B2 (en) | Workforce optimization by improved provision of job performance plan | |
| CN112669852B (en) | Memory allocation method and device and electronic equipment | |
| CN113590329A (en) | Resource processing method and device | |
| CN115658311A (en) | Resource scheduling method, device, equipment and medium | |
| CN106201665A (en) | A kind of dynamic queue's method for arranging and equipment | |
| CN109978211B (en) | Method and device for predicting flight arrival and departure rate | |
| CN116126496B (en) | Resource scheduling method and device for image data processing system | |
| CN114153614A (en) | Memory management method and device, electronic equipment and automatic driving vehicle | |
| CN113986497A (en) | Queue scheduling method, device and system based on multi-tenant technology | |
| CN116680588A (en) | Terminal area division setting method, device, equipment and medium based on improved Agent | |
| CN114327897A (en) | Resource allocation method and device and electronic equipment | |
| CN114416349A (en) | Resource allocation method, device, equipment, storage medium and program product | |
| CN113723405A (en) | Method and device for determining area outline and electronic equipment | |
| CN114186841A (en) | High-precision map task allocation method and device, electronic equipment and storage medium | |
| CN110764886B (en) | Batch job cooperative scheduling method and system supporting multi-partition processing | |
| WO2022162733A1 (en) | Allocation cost determination device, allocation cost determination method, and allocation cost determination program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |