CN116974765A - Storage management system of heterogeneous computer - Google Patents
Storage management system of heterogeneous computer Download PDFInfo
- Publication number
- CN116974765A CN116974765A CN202310964088.8A CN202310964088A CN116974765A CN 116974765 A CN116974765 A CN 116974765A CN 202310964088 A CN202310964088 A CN 202310964088A CN 116974765 A CN116974765 A CN 116974765A
- Authority
- CN
- China
- Prior art keywords
- computing
- task
- execution
- calculation
- subtask
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004458 analytical method Methods 0.000 claims abstract description 44
- 238000000034 method Methods 0.000 claims abstract description 34
- 230000008569 process Effects 0.000 claims abstract description 29
- 230000003993 interaction Effects 0.000 claims abstract description 6
- 238000004364 calculation method Methods 0.000 claims description 159
- 238000007726 management method Methods 0.000 claims description 41
- 238000013523 data management Methods 0.000 claims description 17
- 230000005540 biological transmission Effects 0.000 claims description 14
- 230000001133 acceleration Effects 0.000 claims description 12
- 238000012098 association analyses Methods 0.000 claims description 9
- 230000003044 adaptive effect Effects 0.000 claims description 7
- 239000000284 extract Substances 0.000 claims description 6
- 238000003491 array Methods 0.000 claims description 3
- 238000013075 data extraction Methods 0.000 abstract 1
- 230000009471 action Effects 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 238000005538 encapsulation Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the technical field of computer storage management, in particular to a storage management system of a heterogeneous computer, which comprises a subtask balanced scheduling analysis module, wherein the subtask balanced scheduling analysis module carries out self-adaptive balanced processing on each constructed subtask execution chain according to the execution condition of each computing resource in the heterogeneous computer on each type of computing task and the computing task condition in heterogeneous computing requirements, and generates a task scheduling set. In the executing process of the heterogeneous computing demands, the self-adaptive equalization of the computing tasks in the subtasks is realized by combining the self condition of the heterogeneous computer and the data association condition between the computing tasks in the heterogeneous computing demands, and the total time for the heterogeneous computer to complete the heterogeneous computing demands is shortened; meanwhile, the cache data is adopted to improve the data extraction speed, so that the total overhead time of data interaction between computing resources is reduced, and the efficiency of executing heterogeneous computing demands of heterogeneous computers is improved.
Description
Technical Field
The invention relates to the technical field of computer storage management, in particular to a storage management system of a heterogeneous computer.
Background
Heterogeneous computing is a special form of parallel and distributed computing that accomplishes the computational tasks by either a single independent computer that can support both simd and mimd modes, or by a group of independent computers interconnected by a high-speed network. It enables coordinated use of machines of varying performance, architecture to meet different computing demands, and enables code (or code segments) to be executed in a manner that achieves maximum overall performance.
Dividing code segments with the same type into the same subtasks on the basis of extracting the parallelism type of the calculation tasks, then distributing each subtask to the most suitable calculation resource (computer or calculation module) for executing the subtask according to different parallelism types to minimize the total execution time of the calculation tasks, and summarizing the execution results of each calculation resource on the distributed subtasks to a data management terminal in the execution process, and dividing the execution results of each subtask through the data management terminal to generate corresponding heterogeneous calculation demand results;
in the existing heterogeneous computing demand execution process, the operation mode and the storage mode of the heterogeneous computer have larger defects, in each computing task of the parallelism type in each heterogeneous computing demand, the number of computing tasks corresponding to each subtask is variable, and furthermore, the execution time of each subtask may also be different, according to the existing technology, the total execution time of the computing tasks may not reach the minimum value (the execution time corresponding to each subtask may be different, and the situation that part of subtasks are finished and the rest of subtasks need to be processed for a long time occurs); meanwhile, not all the computing tasks in the computing tasks are of a parallelism type, data association (the execution result of the rest computing tasks is needed for the execution of one computing task) may exist among different computing tasks, the distribution mode of the computing tasks of the non-parallelism type affects the total execution time of the computing tasks corresponding to the heterogeneous computing requirements to a certain extent, and the existing mode of summarizing, storing and re-distributing (re-transmitting) the computing results affects the total execution time of the heterogeneous computing requirements to a certain extent, so that the prior art has a larger defect.
Disclosure of Invention
The present invention is directed to a storage management system for heterogeneous computers, so as to solve the above-mentioned problems in the prior art.
In order to solve the technical problems, the invention provides the following technical scheme: the system comprises a heterogeneous computing demand analysis association analysis module, a subtask execution chain acquisition analysis module, a subtask balanced scheduling analysis module, a subtask execution cache management module and a heterogeneous computing demand storage management module;
the heterogeneous computing demand analysis association analysis module extracts computing tasks in the heterogeneous computing demands, numbers the computing tasks and analyzes association priority relations among different computing tasks;
the subtask execution chain acquisition analysis module is used for carrying out type division on the computing tasks and constructing a subtask execution chain according to the same type of the divided computing tasks;
the subtask balanced scheduling analysis module performs self-adaptive balanced processing on each constructed subtask execution chain according to the execution condition of each computing resource in the heterogeneous computer on each type of computing task and the computing task condition in the heterogeneous computing demand, and generates a task scheduling set;
The subtask execution buffer management module manages the execution sequence of the calculation tasks in the execution process of the subtasks, extracts the buffer data of the associated calculation tasks stored in the buffer corresponding to the calculation resources of the subtasks in the execution process of the calculation tasks, and confirms the storage position of the execution result of the calculation tasks;
and the heterogeneous computation demand storage management module judges whether the obtained computation task cache data can generate a corresponding heterogeneous computation demand result, and processes the storage mode of the heterogeneous computation demand result and the cache data in the cache.
Furthermore, the heterogeneous computing demand analysis association analysis module comprises a task sequence number generating unit and an execution priority model analysis unit,
the task sequence number generating unit acquires a calculation task set formed by each calculation task in heterogeneous calculation demands in the data management terminal, and performs task number on each calculation task in the heterogeneous calculation demands to generate a task number sequence, wherein each calculation task corresponds to a unique task number;
the execution priority model analysis unit obtains association priority relation pairs among the computing tasks corresponding to different elements in the computing task set according to the data relation among the computing tasks corresponding to the elements in the computing task set, and builds an execution priority model of the computing task set elements;
When the execution priority model of the computing task set element is constructed, a data parameter set formed by data parameters in a code segment corresponding to each computing task in the computing task set is obtained, and when the data relation between any two computing tasks in the computing task set is analyzed,
when the intersection between the data parameter sets corresponding to the two corresponding computing tasks is not an empty set, further judging the data parameter types corresponding to the elements in the obtained intersection in the two corresponding computing tasks respectively, wherein the data parameter types comprise input data and output data,
if the data parameter types of the elements in the obtained intersection are different from each other in the corresponding two computing tasks, judging that the data relationship exists between the corresponding two computing tasks, wherein the two computing tasks form an association priority relationship pair, and the association priority of the computing task with the data parameter type being output data in the formed association priority relationship pair is greater than that of the computing task with the data parameter type being input data; when zero or one or two associated priority relation pairs are corresponding to the same two computing tasks, and two associated priority relation pairs corresponding to the same two computing tasks,
If the subtasks to which the two computing tasks belong are the same, combining the two computing tasks into a new computing task, updating the computing task set and the task number sequence,
if the subtasks to which the two computing tasks belong are different, the two associated priority relation pairs are simultaneously established;
if the situation that the types of the data parameters corresponding to any element in the two calculation tasks are different does not exist in the obtained intersection, judging that the corresponding two calculation tasks have no data relationship, and the two calculation tasks cannot form an association priority relationship pair;
when the intersection between the data parameter sets corresponding to the two corresponding computing tasks is an empty set, further judging whether the data transmission condition exists in the codes corresponding to the two corresponding computing tasks,
in the data transmission process, after the data receiver receives the data, a new data parameter name may be defined, and the situation may be caused by function encapsulation of a code segment of a computing task in the compiling process;
judging that a data relationship exists between the corresponding two computing tasks when the data transmission conditions exist in the codes corresponding to the corresponding two computing tasks respectively, wherein the two computing tasks form an association priority relationship pair, and the association priority of a data receiver in the data transmission in the formed association priority relationship pair is smaller than that of a data sender;
If the corresponding two computing tasks are not in the data transmission condition in the corresponding codes, judging that the corresponding two computing tasks are not in the data relationship, and the corresponding two computing tasks cannot form an association priority relationship pair;
the execution priority model of the computing task set elements is a set formed by corresponding association priority relation pairs among the elements.
Furthermore, when the subtask execution chain acquisition analysis module performs type classification on the calculation tasks, the parallelism types of different calculation tasks in the calculation task set are obtained, the parallelism types of the calculation tasks are preset in a database,
when the subtask execution chain acquisition and analysis module constructs a subtask execution chain, dividing code segments corresponding to calculation tasks with the same type into the same subtask, generating a subtask initial execution sequence according to the sequence from small task numbers to large task numbers, adjusting the positions of the calculation tasks in the subtask initial execution sequence according to each associated priority relation pair in an execution priority model of calculation task set elements, moving the calculation tasks with high associated priorities in the associated priority relation pair to the front of the calculation tasks with low associated priorities, and taking the finally adjusted execution sequences of the subtasks as the subtask execution chains corresponding to the corresponding subtasks, wherein the code segments corresponding to each calculation task in the subtask execution chain are taken as a chain node.
Further, the sub-task balanced scheduling analysis module comprises a calculation task sequence generating unit and a task scheduling set generating unit,
the calculation task sequence generation unit performs data acquisition on a heterogeneous computer to which the data management terminal belongs to obtain an acceleration ratio set of each calculation resource in the heterogeneous computer relative to a reference serial computer, performs self-adaptive equalization processing on each sub-task execution chain by combining calculation task information in heterogeneous calculation requirements to obtain equalized sub-task execution chains, and takes each equalized sub-task execution chain as a calculation task sequence; the reference serial computer is a computer preset in a database;
the task scheduling set generating unit obtains an array formed by each calculation task in the calculation task sequence corresponding to each subtask, and gathers the arrays corresponding to each subtask to generate a task scheduling set;
the task scheduling set generating unit binds the execution priority model, the task scheduling set and the task number sequence with each sub-task respectively, and transmits the execution priority model, the task scheduling set and the task number sequence to the computing resource corresponding to the corresponding sub-task, wherein each sub-task in the database binds a unique computing resource, and the computing resource comprises a computing module in a single independent computer or one computer in a group of independent computers.
Further, when the calculation task sequence generating unit combines calculation task information in heterogeneous calculation demands to perform self-adaptive equalization processing on each subtask execution chain, acquiring time for a reference serial computer to execute a given task in a history database, acquiring percentage of execution time occupied by the reference serial computer to execute different subtasks in the given task, acquiring acceleration ratio of different calculation resources to execute various subtasks in heterogeneous calculation relative to the reference serial computer, and marking acceleration ratio of the ith calculation resource to execute the jth subtask in heterogeneous calculation relative to the reference serial computer as Aij;
the reference serial computer is a reference object, and is used for measuring the speed of executing the calculation tasks in each subtask by different calculation resources in the heterogeneous computer;
counting the number of calculation tasks in each subtask, and recording the total number of calculation tasks in the ith subtask as Mi; the number of all the associated priority relation pairs of the computing tasks in the ith subtask in the execution priority model of the computing task set element is recorded as Ni;
obtaining a maximum time difference predicted value when each computing resource in the heterogeneous computer executes the bound subtasks, wherein the maximum time difference is equal to a difference value between a predicted maximum time and a predicted minimum time when each computing resource in the heterogeneous computer executes the bound subtasks,
If the obtained maximum time difference predicted value is greater than or equal to a preset value, judging that adaptive equalization is needed to be carried out on a subtask execution chain corresponding to the subtask corresponding to the predicted maximum time, marking the subtask execution chain corresponding to the subtask corresponding to the predicted maximum time as G, wherein the preset value is a preset constant in a database,
if the obtained maximum time difference predicted value is smaller than a preset value, judging that adaptive equalization is not needed to be carried out on a subtask execution chain corresponding to the subtask corresponding to the predicted maximum time;
wherein, the predicted time when the ith computing resource in the heterogeneous computer executes the bound subtask is recorded as Ti,
the Ti= Σ h＝1 Mi ×ti h /wi h +Ni×tb, the ti h Representing reference serial computer execution given arbitrary in a history databaseIn the task, the time length corresponding to each calculation task is averaged when the calculation task is the same as the h calculation task in the ith sub task execution chain; the wi is h The acceleration ratio of the ith computing resource execution in the heterogeneous computer in the historical data to the subtask with the same type of the h computing task parallelism in the ith subtask execution chain relative to the reference serial computer is represented, and tb represents the average overhead time length for single-time data interaction between computing resources in the historical data;
When the parallelism type is divided for the calculation tasks, the calculation tasks of one parallelism type are divided into one subtask, and then one parallelism type corresponds to one subtask, but in the subsequent steps, the self-adaptive equalization of the subtask execution chains exists, the calculation tasks in different subtask execution chains can be moved to other subtask execution chains, and therefore, the parallelism type of the calculation tasks in the subtask execution chains can be non-unique, namely one or more parallelism types are corresponding; ni×tb represents the predicted value of the total duration of machine interaction overhead in heterogeneous computing demand execution.
Acquiring a set formed by all associated priority relation pairs containing the computing tasks in the G in an execution priority model of computing task set elements, marking the set as F, and marking the computing tasks contained in the G in each associated priority relation pair in the F;
selecting any one of the computing tasks in the G to be moved to other subtask execution chains, if the selected computing task is the computing task marked in the obtained F, the selected subtask execution chain to be moved by the computing task can only be the subtask execution chain which belongs to the other computing task in the F and contains the association priority relation of the selected computing task, if the selected computing task is not the computing task marked in the obtained F, the selected subtask execution chain to be moved by the computing task is any one of the other subtask execution chains, and taking each computing task moving result as a subtask execution chain balancing scheme to obtain different subtask execution chain balancing schemes; analyzing the maximum time difference predicted value corresponding to each sub-task execution chain equalizing scheme, taking the sub-task execution chain equalizing scheme with the minimum maximum time difference predicted value as a sub-task execution chain optimal equalizing scheme, substituting the obtained sub-task execution chain optimal equalizing scheme into the calculation task sequence generating unit again for execution until the calculation task sequence generating unit judges that self-adaptive equalization is not needed to be carried out on the sub-task execution chain corresponding to the sub-task corresponding to the predicted maximum time.
Furthermore, the subtask execution cache management module comprises a subtask execution analysis unit and a calculation task cache management unit,
the subtask execution analysis unit judges whether the corresponding calculation task has the associated calculation task or not in the process of executing the subtask,
if no associated computing task exists, the corresponding computing task is directly executed,
if the associated computing task exists, further judging whether the execution result data of the associated computing task of the corresponding computing task exists in the buffer corresponding to the corresponding computing resource,
if the execution result data of the associated computing task of the corresponding computing task exists in the buffer corresponding to the corresponding computing resource, the execution result data of the associated computing task in the buffer is extracted and substituted into the corresponding computing task to execute the corresponding computing task;
if the execution result data of the associated computing task of the corresponding computing task does not exist in the buffer corresponding to the corresponding computing resource, the corresponding computing task in the subtasks is skipped, whether the next computing task in the sequence of computing tasks corresponding to the subtasks can be executed is judged, after one computing task in the sequence of computing tasks corresponding to the corresponding subtasks is completed again, the first non-executed computing task in the sequence of computing tasks corresponding to the corresponding subtasks is skipped again, and whether the subtasks can be executed is judged.
Further, the computing task buffer management unit identifies the task number corresponding to each executed computing task in the process of executing the computing task, transmits and stores the execution result of the corresponding computing task in a buffer in the computing resource corresponding to each computing task associated with the computing task, and if the corresponding computing task does not have the associated computing task, transmits and stores the execution result of the corresponding computing task in the buffer of the data management terminal;
the computing resources respectively corresponding to the computing tasks associated with the computing tasks are obtained by inquiring the subtasks respectively corresponding to the computing task numbers containing the associated priority relation with the corresponding computing tasks in the task scheduling set in the execution priority model.
Further, when the heterogeneous computing demand storage management module judges whether the obtained computing task cache data can generate the corresponding heterogeneous computing demand result, the completion condition of each computing task in the task number sequence in the cache of the data management terminal is monitored to judge whether the cache data can generate the corresponding heterogeneous computing demand result,
when the heterogeneous computing demand storage management module processes the storage mode of the heterogeneous computing demand results and the cache data in the cache, after the corresponding heterogeneous computing demand results are generated, the obtained heterogeneous computing demand results are output and stored in the database, and data related to the heterogeneous computing demands in the cache of each computing resource and the data management terminal are cleared.
Compared with the prior art, the invention has the following beneficial effects: in the executing process of the heterogeneous computing demands, the method and the device of the invention consider the difference of the executing time of each subtask in the computing tasks of each parallelism type in the heterogeneous computing demands, and combine the self situation of the heterogeneous computer and the data association situation among the computing tasks in the heterogeneous computing demands to realize the self-adaptive equalization of the computing tasks in the subtasks and shorten the total time of the heterogeneous computer for completing the heterogeneous computing demands; meanwhile, when the heterogeneous computer executes heterogeneous computing demands, the effective management of the cache data corresponding to each computing resource is realized, the computing task distribution data transmitted through binding in the subtask distribution process is directly stored in a buffer corresponding to the computing resource to which the associated computing task belongs after the execution of the computing task is finished, and after the heterogeneous computing demands are finished, the computing result is stored in a database and the corresponding cache data is cleared.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate the invention and together with the embodiments of the invention, serve to explain the invention. In the drawings:
FIG. 1 is a schematic diagram of a storage management system of a heterogeneous computer according to the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, the present invention provides the following technical solutions: the system comprises a heterogeneous computing demand analysis association analysis module, a subtask execution chain acquisition analysis module, a subtask balanced scheduling analysis module, a subtask execution cache management module and a heterogeneous computing demand storage management module;
the heterogeneous computing demand analysis association analysis module extracts computing tasks in the heterogeneous computing demands, numbers the computing tasks and analyzes association priority relations among different computing tasks;
The heterogeneous computing demand analysis association analysis module comprises a task sequence number generation unit and an execution priority model analysis unit,
the task sequence number generating unit acquires a calculation task set formed by each calculation task in heterogeneous calculation demands in the data management terminal, and performs task number on each calculation task in the heterogeneous calculation demands to generate a task number sequence, wherein each calculation task corresponds to a unique task number;
the execution priority model analysis unit obtains association priority relation pairs among the computing tasks corresponding to different elements in the computing task set according to the data relation among the computing tasks corresponding to the elements in the computing task set, and builds an execution priority model of the computing task set elements;
when the execution priority model of the computing task set element is constructed, a data parameter set formed by data parameters in a code segment corresponding to each computing task in the computing task set is obtained, and when the data relation between any two computing tasks in the computing task set is analyzed,
when the intersection between the data parameter sets corresponding to the two corresponding computing tasks is not an empty set, further judging the data parameter types corresponding to the elements in the obtained intersection in the two corresponding computing tasks respectively, wherein the data parameter types comprise input data and output data,
If the data parameter types of the elements in the obtained intersection are different from each other in the corresponding two computing tasks, judging that the data relationship exists between the corresponding two computing tasks, wherein the two computing tasks form an association priority relationship pair, and the association priority of the computing task with the data parameter type being output data in the formed association priority relationship pair is greater than that of the computing task with the data parameter type being input data; when zero or one or two associated priority relation pairs are corresponding to the same two computing tasks, and two associated priority relation pairs corresponding to the same two computing tasks,
if the subtasks to which the two computing tasks belong are the same, combining the two computing tasks into a new computing task, updating the computing task set and the task number sequence,
if the subtasks to which the two computing tasks belong are different, the two associated priority relation pairs are simultaneously established;
if the situation that the types of the data parameters corresponding to any element in the two calculation tasks are different does not exist in the obtained intersection, judging that the corresponding two calculation tasks have no data relationship, and the two calculation tasks cannot form an association priority relationship pair;
When the intersection between the data parameter sets corresponding to the two corresponding computing tasks is an empty set, further judging whether a data transmission condition exists in the codes corresponding to the two corresponding computing tasks (in the data transmission process, after the data receiver receives the data, a new data parameter name may be defined, which may be caused by function encapsulation of the code segments of the computing tasks in the writing process),
judging that a data relationship exists between the corresponding two computing tasks when the data transmission conditions exist in the codes corresponding to the corresponding two computing tasks respectively, wherein the two computing tasks form an association priority relationship pair, and the association priority of a data receiver in the data transmission in the formed association priority relationship pair is smaller than that of a data sender;
if the corresponding two computing tasks are not in the data transmission condition in the corresponding codes, judging that the corresponding two computing tasks are not in the data relationship, and the corresponding two computing tasks cannot form an association priority relationship pair;
the execution priority model of the computing task set elements is a set formed by corresponding association priority relation pairs among the elements.
The subtask execution chain acquisition analysis module is used for carrying out type division on the computing tasks and constructing a subtask execution chain according to the same type of the divided computing tasks;
when the subtask execution chain acquisition analysis module performs type classification on the calculation tasks, the parallelism types of different calculation tasks in the calculation task set are obtained, wherein the parallelism types of the calculation tasks are preset in a database
The parallelism types of the computing tasks in this embodiment include four types of vectors, multi-instruction multi-data streams, single-instruction multi-data streams and single-instruction single-data streams,
when the subtask execution chain acquisition and analysis module constructs a subtask execution chain, dividing code segments corresponding to calculation tasks with the same type into the same subtask, generating a subtask initial execution sequence according to the sequence from small task numbers to large task numbers, adjusting the positions of the calculation tasks in the subtask initial execution sequence according to each associated priority relation pair in an execution priority model of calculation task set elements, moving the calculation tasks with high associated priorities in the associated priority relation pair to the front of the calculation tasks with low associated priorities, and taking the finally adjusted execution sequences of the subtasks as the subtask execution chains corresponding to the corresponding subtasks, wherein the code segments corresponding to each calculation task in the subtask execution chain are taken as a chain node.
The subtask balanced scheduling analysis module performs self-adaptive balanced processing on each constructed subtask execution chain according to the execution condition of each computing resource in the heterogeneous computer on each type of computing task and the computing task condition in the heterogeneous computing demand, and generates a task scheduling set;
the sub-task balanced scheduling analysis module comprises a calculation task sequence generating unit and a task scheduling set generating unit,
the calculation task sequence generation unit performs data acquisition on a heterogeneous computer to which the data management terminal belongs to obtain an acceleration ratio set of each calculation resource in the heterogeneous computer relative to a reference serial computer, performs self-adaptive equalization processing on each sub-task execution chain by combining calculation task information in heterogeneous calculation requirements to obtain equalized sub-task execution chains, and takes each equalized sub-task execution chain as a calculation task sequence; the reference serial computer is a computer preset in a database;
the task scheduling set generating unit obtains an array formed by each calculation task in the calculation task sequence corresponding to each subtask, and gathers the arrays corresponding to each subtask to generate a task scheduling set;
The task scheduling set generating unit binds the execution priority model, the task scheduling set and the task number sequence with each sub-task respectively, and transmits the execution priority model, the task scheduling set and the task number sequence to the computing resource corresponding to the corresponding sub-task, wherein each sub-task in the database binds a unique computing resource, and the computing resource comprises a computing module in a single independent computer or one computer in a group of independent computers.
When the calculation task sequence generating unit performs self-adaptive equalization processing on each subtask execution chain by combining calculation task information in heterogeneous calculation demands, acquiring time for a reference serial computer to execute a given task in a history database, acquiring percentage of execution time occupied by the reference serial computer to execute different subtasks in the given task, acquiring acceleration ratio of different calculation resources to execute various subtasks in heterogeneous calculation relative to the reference serial computer, and marking acceleration ratio of an ith calculation resource to execute a jth subtask in heterogeneous calculation relative to the reference serial computer as Aij;
in this embodiment, the reference serial computer is a reference object, so as to measure the speed of executing the computing tasks in each subtask by different computing resources in the heterogeneous computer; furthermore, the configuration of the reference serial computers is different, the execution time of a given task is different, and if the preset reference computer is changed, the speed ratio of executing various subtasks by different computing resources in the heterogeneous computation obtained in the follow-up process relative to the reference serial computers is also different.
Counting the number of calculation tasks in each subtask, and recording the total number of calculation tasks in the ith subtask as Mi; the number of all the associated priority relation pairs of the computing tasks in the ith subtask in the execution priority model of the computing task set element is recorded as Ni;
obtaining a maximum time difference predicted value when each computing resource in the heterogeneous computer executes the bound subtasks, wherein the maximum time difference is equal to a difference value between a predicted maximum time and a predicted minimum time when each computing resource in the heterogeneous computer executes the bound subtasks,
if the obtained maximum time difference predicted value is greater than or equal to a preset value, judging that adaptive equalization is needed to be carried out on a subtask execution chain corresponding to the subtask corresponding to the predicted maximum time, marking the subtask execution chain corresponding to the subtask corresponding to the predicted maximum time as G, wherein the preset value is a preset constant in a database,
if the obtained maximum time difference predicted value is smaller than a preset value, judging that adaptive equalization is not needed to be carried out on a subtask execution chain corresponding to the subtask corresponding to the predicted maximum time;
wherein, the predicted time when the ith computing resource in the heterogeneous computer executes the bound subtask is recorded as Ti,
The Ti= Σ h＝1 Mi ×ti h /wi h +Ni×tb, the ti h The method comprises the steps that in a given task executed by a reference serial computer in a historical database, the time length corresponding to each calculation task is averaged when the calculation task is executed by a subtask with the same type of parallelism of an h calculation task in an ith subtask execution chain; the wi is h The acceleration ratio of the ith computing resource execution in the heterogeneous computer in the historical data to the subtask with the same type of the h computing task parallelism in the ith subtask execution chain relative to the reference serial computer is represented, and tb represents the average overhead time length for single-time data interaction between computing resources in the historical data;
in this embodiment, when the computing task is divided into the parallelism types, the computing task of one parallelism type is divided into one subtask, and then one parallelism type corresponds to one subtask, but in the subsequent step, there is adaptive equalization of the subtask execution chains, and the computing tasks in different subtask execution chains are moved to the rest subtask execution chains, so that it is known that the parallelism type to which the computing task in the subtask execution chain belongs may not be unique, that is, corresponds to one or more parallelism types; ni x tb represents the predicted value of the total duration of machine interaction overhead in the heterogeneous computing demand execution process;
Acquiring a set formed by all associated priority relation pairs containing the computing tasks in the G in an execution priority model of computing task set elements, marking the set as F, and marking the computing tasks contained in the G in each associated priority relation pair in the F;
selecting any one of the computing tasks in the G to be moved to other subtask execution chains, if the selected computing task is the computing task marked in the obtained F, the selected subtask execution chain to be moved by the computing task can only be the subtask execution chain which belongs to the other computing task in the F and contains the association priority relation of the selected computing task, if the selected computing task is not the computing task marked in the obtained F, the selected subtask execution chain to be moved by the computing task is any one of the other subtask execution chains, and taking each computing task moving result as a subtask execution chain balancing scheme to obtain different subtask execution chain balancing schemes; analyzing the maximum time difference predicted value corresponding to each sub-task execution chain equalizing scheme, taking the sub-task execution chain equalizing scheme with the minimum maximum time difference predicted value as a sub-task execution chain optimal equalizing scheme, substituting the obtained sub-task execution chain optimal equalizing scheme into the calculation task sequence generating unit again for execution until the calculation task sequence generating unit judges that self-adaptive equalization is not needed to be carried out on the sub-task execution chain corresponding to the sub-task corresponding to the predicted maximum time.
The subtask execution buffer management module manages the execution sequence of the calculation tasks in the execution process of the subtasks, extracts the buffer data of the associated calculation tasks stored in the buffer corresponding to the calculation resources of the subtasks in the execution process of the calculation tasks, and confirms the storage position of the execution result of the calculation tasks;
the subtask execution cache management module comprises a subtask execution analysis unit and a calculation task cache management unit,
the subtask execution analysis unit judges whether the corresponding calculation task has the associated calculation task or not in the process of executing the subtask,
if no associated computing task exists, the corresponding computing task is directly executed,
if the associated computing task exists, further judging whether the execution result data of the associated computing task of the corresponding computing task exists in the buffer corresponding to the corresponding computing resource,
if the execution result data of the associated computing task of the corresponding computing task exists in the buffer corresponding to the corresponding computing resource, the execution result data of the associated computing task in the buffer is extracted and substituted into the corresponding computing task to execute the corresponding computing task;
if the execution result data of the associated computing task of the corresponding computing task does not exist in the buffer corresponding to the corresponding computing resource, the corresponding computing task in the subtasks is skipped, whether the next computing task in the sequence of computing tasks corresponding to the subtasks can be executed is judged, after one computing task in the sequence of computing tasks corresponding to the corresponding subtasks is completed again, the first non-executed computing task in the sequence of computing tasks corresponding to the corresponding subtasks is skipped again, and whether the subtasks can be executed is judged.
The computing task cache management unit identifies the task number corresponding to each executed computing task in the process of executing the computing task, transmits and stores the execution result of the corresponding computing task into a cache in computing resources corresponding to each computing task associated with the computing task, and transmits and stores the execution result of the corresponding computing task into the cache of the data management terminal if the corresponding computing task does not have the associated computing task;
the computing resources respectively corresponding to the computing tasks associated with the computing tasks are obtained by inquiring the subtasks respectively corresponding to the computing task numbers containing the associated priority relation with the corresponding computing tasks in the task scheduling set in the execution priority model.
The heterogeneous computing demand storage management module judges whether the obtained computing task cache data can generate corresponding heterogeneous computing demand results or not, and processes the storage mode of the heterogeneous computing demand results and the cache data in the cache;
when the heterogeneous computing demand storage management module judges whether the obtained computing task cache data can generate the corresponding heterogeneous computing demand result, the completion condition of each computing task in the task number sequence in the cache of the data management terminal is monitored to judge whether the cache data can generate the corresponding heterogeneous computing demand result,
In this embodiment, when there is no incomplete computing task in the task number sequence, or there is an incomplete computing task in the task number sequence but the computing task associated with the incomplete computing task is already completed, and in the association relationship between the two, when the completed computing task needs the execution result of the incomplete computing task to execute, it is determined that the cache data can generate a corresponding heterogeneous computing demand result.
In this embodiment, if the heterogeneous computer is a set of independent computers, the heterogeneous computer includes two computing tasks (U1 and U2 respectively) and two computing resources (computing resource a and computing resource b respectively),
if the computing resource of the subtask corresponding to U1 is A, the computing resource of the subtask corresponding to U2 is B, and the output result after the execution of U1 is the input result when the execution of U2,
after the first computing resource executes the U1, the obtained execution result is directly transmitted to the second computing resource buffer, so that the second computing resource can execute the U2 conveniently.
When the heterogeneous computing demand storage management module processes the storage mode of the heterogeneous computing demand results and the cache data in the cache, after the corresponding heterogeneous computing demand results are generated, the obtained heterogeneous computing demand results are output and stored in the database, and data related to the heterogeneous computing demands in the cache of each computing resource and the data management terminal are cleared.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Finally, it should be noted that: the foregoing description is only a preferred embodiment of the present invention, and the present invention is not limited thereto, but it is to be understood that modifications and equivalents of some of the technical features described in the foregoing embodiments may be made by those skilled in the art, although the present invention has been described in detail with reference to the foregoing embodiments. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (8)
1. The storage management system of the heterogeneous computer is characterized by comprising a heterogeneous computation demand analysis association analysis module, a subtask execution chain acquisition analysis module, a subtask balanced scheduling analysis module, a subtask execution cache management module and a heterogeneous computation demand storage management module;
the heterogeneous computing demand analysis association analysis module extracts computing tasks in the heterogeneous computing demands, numbers the computing tasks and analyzes association priority relations among different computing tasks;
the subtask execution chain acquisition analysis module is used for carrying out type division on the computing tasks and constructing a subtask execution chain according to the same type of the divided computing tasks;
the subtask balanced scheduling analysis module performs self-adaptive balanced processing on each constructed subtask execution chain according to the execution condition of each computing resource in the heterogeneous computer on each type of computing task and the computing task condition in the heterogeneous computing demand, and generates a task scheduling set;
the subtask execution buffer management module manages the execution sequence of the calculation tasks in the execution process of the subtasks, extracts the buffer data of the associated calculation tasks stored in the buffer corresponding to the calculation resources of the subtasks in the execution process of the calculation tasks, and confirms the storage position of the execution result of the calculation tasks;
And the heterogeneous computation demand storage management module judges whether the obtained computation task cache data can generate a corresponding heterogeneous computation demand result, and processes the storage mode of the heterogeneous computation demand result and the cache data in the cache.
2. The storage management system of a heterogeneous computer of claim 1, wherein: the heterogeneous computing demand analysis association analysis module comprises a task sequence number generation unit and an execution priority model analysis unit,
the task sequence number generating unit acquires a calculation task set formed by each calculation task in heterogeneous calculation demands in the data management terminal, and performs task number on each calculation task in the heterogeneous calculation demands to generate a task number sequence, wherein each calculation task corresponds to a unique task number;
the execution priority model analysis unit obtains association priority relation pairs among the computing tasks corresponding to different elements in the computing task set according to the data relation among the computing tasks corresponding to the elements in the computing task set, and builds an execution priority model of the computing task set elements;
when the execution priority model of the computing task set element is constructed, a data parameter set formed by data parameters in a code segment corresponding to each computing task in the computing task set is obtained, and when the data relation between any two computing tasks in the computing task set is analyzed,
When the intersection between the data parameter sets corresponding to the two corresponding computing tasks is not an empty set, further judging the data parameter types corresponding to the elements in the obtained intersection in the two corresponding computing tasks respectively, wherein the data parameter types comprise input data and output data,
if the data parameter types of the elements in the obtained intersection are different from each other in the corresponding two computing tasks, judging that the data relationship exists between the corresponding two computing tasks, wherein the two computing tasks form an association priority relationship pair, and the association priority of the computing task with the data parameter type being output data in the formed association priority relationship pair is greater than that of the computing task with the data parameter type being input data; when zero or one or two associated priority relation pairs are corresponding to the same two computing tasks, and two associated priority relation pairs corresponding to the same two computing tasks,
if the subtasks to which the two computing tasks belong are the same, combining the two computing tasks into a new computing task, updating the computing task set and the task number sequence,
if the subtasks to which the two computing tasks belong are different, the two associated priority relation pairs are simultaneously established;
If the situation that the types of the data parameters corresponding to any element in the two calculation tasks are different does not exist in the obtained intersection, judging that the corresponding two calculation tasks have no data relationship, and the two calculation tasks cannot form an association priority relationship pair;
when the intersection between the data parameter sets corresponding to the two corresponding computing tasks is an empty set, further judging whether the data transmission condition exists in the codes corresponding to the two corresponding computing tasks,
judging that a data relationship exists between the corresponding two computing tasks when the data transmission conditions exist in the codes corresponding to the corresponding two computing tasks respectively, wherein the two computing tasks form an association priority relationship pair, and the association priority of a data receiver in the data transmission in the formed association priority relationship pair is smaller than that of a data sender;
if the corresponding two computing tasks are not in the data transmission condition in the corresponding codes, judging that the corresponding two computing tasks are not in the data relationship, and the corresponding two computing tasks cannot form an association priority relationship pair;
the execution priority model of the computing task set elements is a set formed by corresponding association priority relation pairs among the elements.
3. A storage management system for a heterogeneous computer according to claim 2, wherein: when the subtask execution chain acquisition analysis module performs type division on the calculation tasks, the parallelism types of different calculation tasks in the calculation task set are obtained, the parallelism types of the calculation tasks are preset in a database,
when the subtask execution chain acquisition and analysis module constructs a subtask execution chain, dividing code segments corresponding to calculation tasks with the same type into the same subtask, generating a subtask initial execution sequence according to the sequence from small task numbers to large task numbers, adjusting the positions of the calculation tasks in the subtask initial execution sequence according to each associated priority relation pair in an execution priority model of calculation task set elements, moving the calculation tasks with high associated priorities in the associated priority relation pair to the front of the calculation tasks with low associated priorities, and taking the finally adjusted execution sequences of the subtasks as the subtask execution chains corresponding to the corresponding subtasks, wherein the code segments corresponding to each calculation task in the subtask execution chain are taken as a chain node.
4. The storage management system of a heterogeneous computer of claim 1, wherein: the sub-task balanced scheduling analysis module comprises a calculation task sequence generating unit and a task scheduling set generating unit,
The calculation task sequence generation unit performs data acquisition on a heterogeneous computer to which the data management terminal belongs to obtain an acceleration ratio set of each calculation resource in the heterogeneous computer relative to a reference serial computer, performs self-adaptive equalization processing on each sub-task execution chain by combining calculation task information in heterogeneous calculation requirements to obtain equalized sub-task execution chains, and takes each equalized sub-task execution chain as a calculation task sequence; the reference serial computer is a computer preset in a database;
the task scheduling set generating unit obtains an array formed by each calculation task in the calculation task sequence corresponding to each subtask, and gathers the arrays corresponding to each subtask to generate a task scheduling set;
the task scheduling set generating unit binds the execution priority model, the task scheduling set and the task number sequence with each sub-task respectively, and transmits the execution priority model, the task scheduling set and the task number sequence to the computing resource corresponding to the corresponding sub-task, wherein each sub-task in the database binds a unique computing resource, and the computing resource comprises a computing module in a single independent computer or one computer in a group of independent computers.
5. The storage management system of a heterogeneous computer of claim 4, wherein: when the calculation task sequence generating unit performs self-adaptive equalization processing on each subtask execution chain by combining calculation task information in heterogeneous calculation demands, acquiring time for a reference serial computer to execute a given task in a history database, acquiring percentage of execution time occupied by the reference serial computer to execute different subtasks in the given task, acquiring acceleration ratio of different calculation resources to execute various subtasks in heterogeneous calculation relative to the reference serial computer, and marking acceleration ratio of an ith calculation resource to execute a jth subtask in heterogeneous calculation relative to the reference serial computer as Aij;
counting the number of calculation tasks in each subtask, and recording the total number of calculation tasks in the ith subtask as Mi; the number of all the associated priority relation pairs of the computing tasks in the ith subtask in the execution priority model of the computing task set element is recorded as Ni;
obtaining a maximum time difference predicted value when each computing resource in the heterogeneous computer executes the bound subtasks, wherein the maximum time difference is equal to a difference value between a predicted maximum time and a predicted minimum time when each computing resource in the heterogeneous computer executes the bound subtasks,
If the obtained maximum time difference predicted value is greater than or equal to a preset value, judging that adaptive equalization is needed to be carried out on a subtask execution chain corresponding to the subtask corresponding to the predicted maximum time, marking the subtask execution chain corresponding to the subtask corresponding to the predicted maximum time as G, wherein the preset value is a preset constant in a database,
if the obtained maximum time difference predicted value is smaller than a preset value, judging that adaptive equalization is not needed to be carried out on a subtask execution chain corresponding to the subtask corresponding to the predicted maximum time;
wherein, the predicted time when the ith computing resource in the heterogeneous computer executes the bound subtask is recorded as Ti,
the Ti= Σ h＝1 Mi ×ti h /wi h +Ni×tb, the ti h The method comprises the steps that in a given task executed by a reference serial computer in a historical database, the time length corresponding to each calculation task is averaged when the calculation task is executed by a subtask with the same type of parallelism of an h calculation task in an ith subtask execution chain; the wi is h The acceleration ratio of the ith computing resource execution in the heterogeneous computer in the historical data to the subtask with the same type of the h computing task parallelism in the ith subtask execution chain relative to the reference serial computer is represented, and tb represents the average overhead time length for single-time data interaction between computing resources in the historical data;
Acquiring a set formed by all associated priority relation pairs containing the computing tasks in the G in an execution priority model of computing task set elements, marking the set as F, and marking the computing tasks contained in the G in each associated priority relation pair in the F;
selecting any one of the computing tasks in the G to be moved to other subtask execution chains, if the selected computing task is the computing task marked in the obtained F, the selected subtask execution chain to be moved by the computing task can only be the subtask execution chain which belongs to the other computing task in the F and contains the association priority relation of the selected computing task, if the selected computing task is not the computing task marked in the obtained F, the selected subtask execution chain to be moved by the computing task is any one of the other subtask execution chains, and taking each computing task moving result as a subtask execution chain balancing scheme to obtain different subtask execution chain balancing schemes; analyzing the maximum time difference predicted value corresponding to each sub-task execution chain equalizing scheme, taking the sub-task execution chain equalizing scheme with the minimum maximum time difference predicted value as a sub-task execution chain optimal equalizing scheme, substituting the obtained sub-task execution chain optimal equalizing scheme into the calculation task sequence generating unit again for execution until the calculation task sequence generating unit judges that self-adaptive equalization is not needed to be carried out on the sub-task execution chain corresponding to the sub-task corresponding to the predicted maximum time.
6. The storage management system of a heterogeneous computer of claim 1, wherein: the subtask execution cache management module comprises a subtask execution analysis unit and a calculation task cache management unit,
the subtask execution analysis unit judges whether the corresponding calculation task has the associated calculation task or not in the process of executing the subtask,
if no associated computing task exists, the corresponding computing task is directly executed,
if the associated computing task exists, further judging whether the execution result data of the associated computing task of the corresponding computing task exists in the buffer corresponding to the corresponding computing resource,
if the execution result data of the associated computing task of the corresponding computing task exists in the buffer corresponding to the corresponding computing resource, the execution result data of the associated computing task in the buffer is extracted and substituted into the corresponding computing task to execute the corresponding computing task;
if the execution result data of the associated computing task of the corresponding computing task does not exist in the buffer corresponding to the corresponding computing resource, the corresponding computing task in the subtasks is skipped, whether the next computing task in the sequence of computing tasks corresponding to the subtasks can be executed is judged, after one computing task in the sequence of computing tasks corresponding to the corresponding subtasks is completed again, the first non-executed computing task in the sequence of computing tasks corresponding to the corresponding subtasks is skipped again, and whether the subtasks can be executed is judged.
7. The storage management system of a heterogeneous computer of claim 6, wherein: the computing task cache management unit identifies the task number corresponding to each executed computing task in the process of executing the computing task, transmits and stores the execution result of the corresponding computing task into a cache in computing resources corresponding to each computing task associated with the computing task, and transmits and stores the execution result of the corresponding computing task into the cache of the data management terminal if the corresponding computing task does not have the associated computing task;
the computing resources respectively corresponding to the computing tasks associated with the computing tasks are obtained by inquiring the subtasks respectively corresponding to the computing task numbers containing the associated priority relation with the corresponding computing tasks in the task scheduling set in the execution priority model.
8. The storage management system of a heterogeneous computer of claim 1, wherein: when the heterogeneous computing demand storage management module judges whether the obtained computing task cache data can generate the corresponding heterogeneous computing demand result, the completion condition of each computing task in the task number sequence in the cache of the data management terminal is monitored to judge whether the cache data can generate the corresponding heterogeneous computing demand result,
When the heterogeneous computing demand storage management module processes the storage mode of the heterogeneous computing demand results and the cache data in the cache, after the corresponding heterogeneous computing demand results are generated, the obtained heterogeneous computing demand results are output and stored in the database, and data related to the heterogeneous computing demands in the cache of each computing resource and the data management terminal are cleared.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310964088.8A CN116974765A (en) | 2023-08-02 | 2023-08-02 | Storage management system of heterogeneous computer |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310964088.8A CN116974765A (en) | 2023-08-02 | 2023-08-02 | Storage management system of heterogeneous computer |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116974765A true CN116974765A (en) | 2023-10-31 |
Family
ID=88471111
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310964088.8A Pending CN116974765A (en) | 2023-08-02 | 2023-08-02 | Storage management system of heterogeneous computer |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116974765A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117555695A (en) * | 2024-01-10 | 2024-02-13 | 深圳本贸科技股份有限公司 | Optimization method and system for realizing heterogeneous computation based on parallel computation |
-
2023
- 2023-08-02 CN CN202310964088.8A patent/CN116974765A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117555695A (en) * | 2024-01-10 | 2024-02-13 | 深圳本贸科技股份有限公司 | Optimization method and system for realizing heterogeneous computation based on parallel computation |
| CN117555695B (en) * | 2024-01-10 | 2024-05-14 | 深圳本贸科技股份有限公司 | Optimization method and system for realizing heterogeneous computation based on parallel computation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Shukri et al. | Enhanced multi-verse optimizer for task scheduling in cloud computing environments | |
| US11580441B2 (en) | Model training method and apparatus | |
| Yadwadkar et al. | Multi-task learning for straggler avoiding predictive job scheduling | |
| US8214814B2 (en) | Sharing compiler optimizations in a multi-node system | |
| CN103927229A (en) | Scheduling Mapreduce Jobs In A Cluster Of Dynamically Available Servers | |
| CN112685170A (en) | Dynamic optimization of backup strategies | |
| CN103019855B (en) | Method for forecasting executive time of Map Reduce operation | |
| JP2018515844A (en) | Data processing method and system | |
| CN116974765A (en) | Storage management system of heterogeneous computer | |
| JP6888737B2 (en) | Learning devices, learning methods, and programs | |
| CN110019298A (en) | Data processing method and device | |
| CN114610475A (en) | Training method of intelligent resource arrangement model | |
| CN116127899A (en) | Chip design system, method, electronic device, and storage medium | |
| CN115237592A (en) | Mixed cloud service flow scheduling method based on privacy perception | |
| Shao et al. | Production system performance improvement by assembly line-seru conversion | |
| Baldo et al. | Performance models for master/slave parallel programs | |
| CN112417748A (en) | Method, system, equipment and medium for scheduling automatic driving simulation task | |
| Hu et al. | Optimizing resource allocation for data-parallel jobs via gcn-based prediction | |
| CN114185938B (en) | Project traceability analysis method and system based on digital finance and big data traceability | |
| CN106874215B (en) | Serialized storage optimization method based on Spark operator | |
| CN110377769A (en) | Modeling Platform system, method, server and medium based on graph data structure | |
| Li et al. | Performance modelling and cost effective execution for distributed graph processing on configurable VMs | |
| CN107122849B (en) | Spark R-based product detection total completion time minimization method | |
| US10417228B2 (en) | Apparatus and method for analytical optimization through computational pushdown | |
| CN115391047A (en) | Resource scheduling method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |