CN118656203A - Computing power access capability evaluation system for supercomputing - Google Patents
Computing power access capability evaluation system for supercomputing Download PDFInfo
- Publication number
- CN118656203A CN118656203A CN202410656568.2A CN202410656568A CN118656203A CN 118656203 A CN118656203 A CN 118656203A CN 202410656568 A CN202410656568 A CN 202410656568A CN 118656203 A CN118656203 A CN 118656203A
- Authority
- CN
- China
- Prior art keywords
- computing
- task
- node
- access capability
- computing node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000011156 evaluation Methods 0.000 title claims abstract description 73
- 238000000034 method Methods 0.000 claims abstract description 25
- 238000012544 monitoring process Methods 0.000 claims abstract description 7
- 238000004364 calculation method Methods 0.000 claims description 25
- 238000004590 computer program Methods 0.000 claims description 6
- 238000002372 labelling Methods 0.000 claims 1
- 238000012216 screening Methods 0.000 claims 1
- 238000012545 processing Methods 0.000 abstract description 10
- 238000005516 engineering process Methods 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 8
- 101001121408 Homo sapiens L-amino-acid oxidase Proteins 0.000 description 3
- 102100026388 L-amino-acid oxidase Human genes 0.000 description 3
- 238000013475 authorization Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
- G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
本申请提出一种用于超算的算力接入能力评估系统,涉及算力评估技术领域,其中包括:所述第一评估模块,用于根据系统数据库中的计算任务和计算节点的匹配度信息通过第一评估公式进行第一次评估以筛选出第一待选计算节点;所述监测模块,用于获取各个第一待选计算节点完成计算任务过程中的参数数据;所述第二评估模块,用于根据各个第一待选计算节点完成计算任务过程中的所述参数数据通过第二评估公式进行第二次评估以筛选得到第二待选计算节点;所述第三评估模块,用于根据所述第二待选计算节点选择目标计算节点以完成计算任务。通过三个评估模块评估各个计算节点的计算能力,可以更加准确地为计算任务选择处理的节点,提高任务处理效率。
The present application proposes a computing power access capability evaluation system for supercomputing, which relates to the field of computing power evaluation technology, and includes: the first evaluation module, which is used to perform a first evaluation through a first evaluation formula according to the matching information of the computing task and the computing node in the system database to screen out the first candidate computing node; the monitoring module, which is used to obtain the parameter data of each first candidate computing node in the process of completing the computing task; the second evaluation module, which is used to perform a second evaluation through a second evaluation formula according to the parameter data in the process of completing the computing task of each first candidate computing node to screen out the second candidate computing node; the third evaluation module, which is used to select a target computing node according to the second candidate computing node to complete the computing task. By evaluating the computing power of each computing node through three evaluation modules, it is possible to more accurately select processing nodes for computing tasks and improve task processing efficiency.
Description
技术领域Technical Field
本申请涉及算力评估技术领域,尤其涉及一种用于超算的算力接入能力评估系统。The present application relates to the technical field of computing power evaluation, and in particular to a computing power access capability evaluation system for supercomputing.
背景技术Background Art
超算通常是指超级计算,是指能够执行一般个人电脑无法处理的大量资料与高速运算的电脑。就超级计算机和普通计算机的组成而言,构成组件基本相同,但在性能和规模方面却有差异。超级计算机主要特点包含两个方面:极大的数据存储容量和极快速的数据处理速度,因此它可以在多种领域进行一些人们或者普通计算机无法进行的工作。Supercomputers usually refer to supercomputing, which refers to computers that can perform large amounts of data and high-speed calculations that ordinary personal computers cannot handle. In terms of the composition of supercomputers and ordinary computers, the components are basically the same, but there are differences in performance and scale. The main features of supercomputers include two aspects: huge data storage capacity and extremely fast data processing speed, so it can perform some tasks in many fields that people or ordinary computers cannot perform.
现有超算接入能力评估过程,主要根据计算任务设定相关要求,计算节点则根据自身算力是否达到相关要求来判断是否接入相关任务,若要求过低,则会造成计算节点压力过大,若要求过高,则会造成计算任务迟迟得不到响应,尤其是计算时间要求较为紧张的前提下,如何高效的对各个计算节点的接入能力进行评估是需要解决的问题。The existing supercomputer access capability assessment process mainly sets relevant requirements based on computing tasks, and computing nodes determine whether to access relevant tasks based on whether their own computing power meets the relevant requirements. If the requirements are too low, it will cause excessive pressure on the computing nodes. If the requirements are too high, the computing tasks will not be responded to for a long time. Especially under the premise of tight computing time requirements, how to efficiently evaluate the access capability of each computing node is a problem that needs to be solved.
发明内容Summary of the invention
本申请旨在至少在一定程度上解决相关技术中的技术问题之一。The present application aims to solve one of the technical problems in the related art at least to some extent.
为此,本申请的第一个目的在于提出一种用于超算的算力接入能力评估系统。To this end, the first objective of this application is to propose a computing power access capability assessment system for supercomputing.
本申请的第二个目的在于提出一种电子设备。The second objective of the present application is to provide an electronic device.
本申请的第三个目的在于提出一种计算机可读存储介质。The third object of the present application is to provide a computer-readable storage medium.
本申请的第四个目的在于提出一种计算机程序产品。A fourth object of the present application is to provide a computer program product.
为达上述目的,本申请第一方面实施例提出了一种用于超算的算力接入能力评估系统,包括:To achieve the above-mentioned purpose, the first embodiment of the present application proposes a computing power access capability evaluation system for supercomputing, including:
第一评估模块、监测模块、第二评估模块和第三评估模块;a first assessment module, a monitoring module, a second assessment module, and a third assessment module;
所述第一评估模块,用于根据系统数据库中的计算任务和计算节点的匹配度信息通过第一评估公式进行第一次评估以筛选出第一待选计算节点;其中,所述匹配度信息包括内容匹配度C、内存匹配度M和操作环境匹配度E;The first evaluation module is used to perform a first evaluation based on the matching information of the computing tasks and computing nodes in the system database through a first evaluation formula to screen out the first candidate computing node; wherein the matching information includes a content matching degree C, a memory matching degree M and an operating environment matching degree E;
所述监测模块,用于获取各个第一待选计算节点完成计算任务过程中的参数数据;The monitoring module is used to obtain parameter data of each first candidate computing node during the process of completing the computing task;
所述第二评估模块,用于根据各个第一待选计算节点完成计算任务过程中的所述参数数据通过第二评估公式进行第二次评估以筛选得到第二待选计算节点;The second evaluation module is used to perform a second evaluation through a second evaluation formula according to the parameter data in the process of each first candidate computing node completing the computing task to screen out the second candidate computing node;
所述第三评估模块,用于根据所述第二待选计算节点选择目标计算节点以完成计算任务。The third evaluation module is used to select a target computing node according to the second candidate computing node to complete the computing task.
可选的,所述第一评估模块用于:Optionally, the first evaluation module is used to:
实时获取系统数据库中各个计算节点和当前待进行的计算任务的数据信息,基于获取的数据信息得到内容匹配度C、内存匹配度M和操作环境匹配度E;Acquire data information of each computing node and the current computing task to be performed in the system database in real time, and obtain content matching degree C, memory matching degree M and operating environment matching degree E based on the acquired data information;
将所述内容匹配度C、内存匹配度M和操作环境匹配度E代入第一评估公式计算第一算力接入能力值K:Substitute the content matching degree C, the memory matching degree M and the operating environment matching degree E into the first evaluation formula to calculate the first computing power access capability value K:
式中,α1、α2、α3都是权重系数,默认α1=α2=α3,可以根据待计算任务内容选择权重比例;Wherein, α 1 , α 2 , and α 3 are all weight coefficients. By default, α 1 = α 2 = α 3 . The weight ratio can be selected according to the content of the task to be calculated.
根据公式获得第i个计算节点的第一算力接入能力值Ki;According to the formula Obtain the first computing power access capability value K i of the i-th computing node;
将第一算力接入能力值Ki与第一算力接入能力值阈值K0进行比较,若第一算力接入能力值Ki大于等于第一算力接入能力值阈值K0,则该计算节点满足第一算力接入能力值需求,确定该计算节点为第一待选计算节点,否则,则不满足第一算力接入能力值需求。The first computing power access capability value K i is compared with the first computing power access capability value threshold K 0. If the first computing power access capability value K i is greater than or equal to the first computing power access capability value threshold K 0 , the computing node meets the first computing power access capability value requirement, and the computing node is determined to be the first candidate computing node; otherwise, the first computing power access capability value requirement is not met.
可选的,所述内容匹配度C与待进行计算任务的内容和计算节点处理的计算任务内容相关,若待进行计算任务的内容完全在计算节点处理的计算任务内容范围内,则C=1;若待进行计算任务的内容完全不在计算节点处理的计算任务内容范围内,则X=0;若待进行计算任务的内容与计算节点处理的计算任务内容范围部分重叠,则X=0.5;Optionally, the content matching degree C is related to the content of the computing task to be performed and the content of the computing task processed by the computing node. If the content of the computing task to be performed is completely within the content range of the computing task processed by the computing node, then C=1; if the content of the computing task to be performed is completely outside the content range of the computing task processed by the computing node, then X=0; if the content of the computing task to be performed partially overlaps with the content range of the computing task processed by the computing node, then X=0.5;
所述内存匹配度M与待进行计算任务需求内存和计算节点当前剩余内存相关,若计算任务需求内存远小于计算节点当前剩余内存,则M=1;若若计算任务需求内存大于计算节点当前剩余内存,则M=0;若计算任务需求内存小于计算节点当前剩余内存且计算节点当前剩余内存减去计算任务需求内存低于设定值,则M=0.5;The memory matching degree M is related to the memory required by the computing task to be performed and the current remaining memory of the computing node. If the memory required by the computing task is much smaller than the current remaining memory of the computing node, then M=1; if the memory required by the computing task is larger than the current remaining memory of the computing node, then M=0; if the memory required by the computing task is smaller than the current remaining memory of the computing node and the current remaining memory of the computing node minus the memory required by the computing task is lower than the set value, then M=0.5;
所述操作环境匹配度E与待进行计算任务需求的运行环境和计算节点的操作系统相关,若待进行计算任务需求的运行环境低于计算节点的操作系统版本,则E=1;若待进行计算任务需求的运行环境高于计算节点的操作系统版本,则E=0。The operating environment matching degree E is related to the operating environment required by the computing task to be performed and the operating system of the computing node. If the operating environment required by the computing task to be performed is lower than the operating system version of the computing node, then E=1; if the operating environment required by the computing task to be performed is higher than the operating system version of the computing node, then E=0.
可选的,按第一算力接入能力值Ki的数值从大到小对第一算力接入能力值Ki进行排序,截取排名前n的第一待选计算节点作为第一评估结果。Optionally, the first computing power access capability values K i are sorted from large to small according to their values, and the top n first candidate computing nodes are intercepted as the first evaluation result.
可选的,所述第二匹配模块用于:Optionally, the second matching module is used for:
获取所述第一评估结果中所述第一待选计算节点的运行参数数据;Obtaining operating parameter data of the first candidate computing node in the first evaluation result;
所述运行参数数据包括:计算节点处理的任务数量m、计算节点处理第j个计算任务完成的时长Tj、第j个计算任务需求的计算时长T0j,计算节点处理m个任务的总时长TS、第j个计算任务的完成度Cj、第j个计算任务需求的完成度C0j;The operation parameter data includes: the number of tasks m processed by the computing node, the time T j for the computing node to complete the j-th computing task, the computing time T 0j required for the j-th computing task, the total time T S for the computing node to process m tasks, the completion degree C j of the j-th computing task, and the completion degree C 0j required for the j-th computing task;
将获取的参数代入第二评估公式计算第二算力接入能力值Q:Substitute the obtained parameters into the second evaluation formula to calculate the second computing power access capability value Q:
根据第二评估公式依次获取所述第一待选计算节点的第二算力接入能力值Q,将第二算力接入能力值Q与第二算力接入能力值阈值Q0比较,若第二算力接入能力值Q大于第二算力接入能力值阈值Q0,则当前节点满足第二算力接入能力值需求,将满足第二算力接入能力值需求的第一待选计算节点作为第二待选计算节点,作为第二次评估结果。According to the second evaluation formula, the second computing power access capability value Q of the first candidate computing node is obtained in sequence, and the second computing power access capability value Q is compared with the second computing power access capability value threshold Q 0. If the second computing power access capability value Q is greater than the second computing power access capability value threshold Q 0 , the current node meets the second computing power access capability value requirement, and the first candidate computing node that meets the second computing power access capability value requirement is used as the second candidate computing node as the second evaluation result.
可选的,所述第三评估模块用于:Optionally, the third evaluation module is used to:
获取所述第二待选计算节点并依次标号;Obtain the second candidate computing nodes and label them in sequence;
当前计算任务向最近的第二待选计算节点发出计算请求,若第二待选计算节点是空闲状态则将当前计算任务加入计算列表,进行任务处理,否则将当前计算任务转送给相邻的第二待选计算节点;The current computing task sends a computing request to the nearest second candidate computing node. If the second candidate computing node is idle, the current computing task is added to the computing list for task processing. Otherwise, the current computing task is forwarded to the adjacent second candidate computing node.
当所有的第二待选计算节点都接收到当前计算任务的计算请求,计算请求仍未得到响应,当前计算任务向最近的第二待选计算节点继续发出第二次计算请求;直至所述计算请求被响应加入计算节点的任务列表。When all second candidate computing nodes have received the computing request of the current computing task, and the computing request has not been responded to, the current computing task continues to send a second computing request to the nearest second candidate computing node; until the computing request is responded to and added to the task list of the computing node.
可选的,所述计算请求具有生命周期,在生命周期内没有被响应,则会立即被丢弃,所有计算节点都不会转发当前计算请求。Optionally, the computing request has a life cycle, and if it is not responded to within the life cycle, it will be discarded immediately, and all computing nodes will not forward the current computing request.
为达上述目的,本申请第二方面实施例提出了一种电子设备,包括:处理器,以及与所述处理器通信连接的存储器;To achieve the above-mentioned purpose, a second aspect of the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;
所述存储器存储计算机执行指令;The memory stores computer-executable instructions;
所述处理器执行所述存储器存储的计算机执行指令,以实现如第一方面中任一项所述的系统。The processor executes the computer-executable instructions stored in the memory to implement the system as described in any one of the first aspects.
为达上述目的,本申请第三方面实施例提出了一种计算机可读存储介质,所述计算机可读存储介质中存储有计算机执行指令,所述计算机执行指令被处理器执行时用于实现如第一方面中任一项所述的系统。To achieve the above-mentioned purpose, the third aspect of the present application proposes a computer-readable storage medium, in which computer-readable storage medium is stored computer execution instructions, and when the computer execution instructions are executed by a processor, they are used to implement a system as described in any one of the first aspects.
为达上述目的,本申请第四方面实施例提出了一种计算机程序产品,该计算机程序被处理器执行时实现第一方面中任一项所述的系统。To achieve the above-mentioned purpose, the fourth aspect of the present application provides a computer program product, which, when executed by a processor, implements the system described in any one of the first aspects.
本申请提供的用于超算的算力接入能力评估系统方法、装置、电子设备及存储介质,通过第一评估公式计算第一算力接入能力值,然后将第一算力接入能力值与第一算力接入能力值阈值进行比较,筛选出一组计算节点并作为第一次评估结果输出;然后通过第二评估公式依次获取满足第一次评估结果的所有第一待选计算节点的第二算力接入能力值,进一步筛选出一组第二待选计算节点作为第二次评估结果;获取所有满足第二次评估结果的第二待选计算节点,计算任务向最接近的计算节点发送计算请求;若最近计算节点不空闲,则将计算请求转发给相邻计算节点,直至计算请求被响应加入计算节点的任务列表,获取满足计算要求的最近的计算节点接入计算任务,可以更加准确地为计算任务选择处理的节点,提高任务处理效率。The computing power access capability evaluation system method, device, electronic device and storage medium provided in the present application for supercomputing calculate the first computing power access capability value through a first evaluation formula, and then compare the first computing power access capability value with the first computing power access capability value threshold, screen out a group of computing nodes and output them as the first evaluation result; then obtain the second computing power access capability values of all first candidate computing nodes that meet the first evaluation result in turn through a second evaluation formula, and further screen out a group of second candidate computing nodes as the second evaluation result; obtain all second candidate computing nodes that meet the second evaluation result, and the computing task sends a computing request to the closest computing node; if the nearest computing node is not idle, forward the computing request to the adjacent computing node until the computing request is responded to and added to the task list of the computing node, and obtain the nearest computing node that meets the computing requirements to access the computing task, so as to more accurately select the processing node for the computing task and improve the task processing efficiency.
本申请附加的方面和优点将在下面的描述中部分给出,部分将从下面的描述中变得明显,或通过本申请的实践了解到。Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
本申请上述的和/或附加的方面和优点从下面结合附图对实施例的描述中将变得明显和容易理解,其中:The above and/or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
图1为本申请实施例所提供的一种用于超算的算力接入能力评估系统的结构示意图。FIG1 is a schematic diagram of the structure of a computing power access capability evaluation system for supercomputing provided in an embodiment of the present application.
具体实施方式DETAILED DESCRIPTION
下面详细描述本申请的实施例,所述实施例的示例在附图中示出,其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的,旨在用于解释本申请,而不能理解为对本申请的限制。The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.
现有超算接入能力评估过程,主要根据计算任务设定相关要求,计算节点则根据自身算力是否达到相关要求来判断是否接入相关任务。The existing supercomputer access capability assessment process mainly sets relevant requirements based on computing tasks, and computing nodes determine whether to access relevant tasks based on whether their own computing power meets the relevant requirements.
若计算任务设定相关要求过低,则会造成计算节点压力过大,若计算任务设定相关要求过高,则会造成计算任务迟迟得不到响应,尤其是计算时间要求较为紧张的前提下,如何高效的对各个计算节点的接入能力进行评估是需要解决的问题。If the requirements for computing tasks are set too low, the computing nodes will be overloaded. If the requirements for computing tasks are set too high, the computing tasks will not be responded to for a long time. Especially when the computing time requirements are tight, how to efficiently evaluate the access capabilities of each computing node is a problem that needs to be solved.
针对这一问题,本申请实施例提供了用于超算的算力接入能力评估系统,图1为本申请实施例所提供的一种用于超算的算力接入能力评估系统的流程示意图。如图1所示,该系统包括:To address this problem, an embodiment of the present application provides a system for evaluating the computing power access capability of a supercomputer. FIG1 is a flow chart of a system for evaluating the computing power access capability of a supercomputer provided by an embodiment of the present application. As shown in FIG1 , the system includes:
第一评估模块110、监测模块120、第二评估模块130和第三评估模块140;A first evaluation module 110, a monitoring module 120, a second evaluation module 130 and a third evaluation module 140;
所述第一评估模块,用于根据系统数据库中的计算任务和计算节点的匹配度信息通过第一评估公式进行第一次评估以筛选出第一待选计算节点;其中,所述匹配度信息包括内容匹配度C、内存匹配度M和操作环境匹配度E;The first evaluation module is used to perform a first evaluation based on the matching information of the computing tasks and computing nodes in the system database through a first evaluation formula to screen out the first candidate computing node; wherein the matching information includes a content matching degree C, a memory matching degree M and an operating environment matching degree E;
所述监测模块,用于获取各个第一待选计算节点完成计算任务过程中的参数数据;The monitoring module is used to obtain parameter data of each first candidate computing node during the process of completing the computing task;
所述第二评估模块,用于根据各个第一待选计算节点完成计算任务过程中的所述参数数据通过第二评估公式进行第二次评估以筛选得到第二待选计算节点;The second evaluation module is used to perform a second evaluation through a second evaluation formula according to the parameter data in the process of each first candidate computing node completing the computing task to screen out the second candidate computing node;
所述第三评估模块,用于根据所述第二待选计算节点选择目标计算节点以完成计算任务。The third evaluation module is used to select a target computing node according to the second candidate computing node to complete the computing task.
可选的,所述第一评估模块用于:Optionally, the first evaluation module is used to:
实时获取系统数据库中各个计算节点和当前待进行的计算任务的数据信息,基于获取的数据信息得到内容匹配度C、内存匹配度M和操作环境匹配度E;Acquire data information of each computing node and the current computing task to be performed in the system database in real time, and obtain content matching degree C, memory matching degree M and operating environment matching degree E based on the acquired data information;
将所述内容匹配度C、内存匹配度M和操作环境匹配度E代入第一评估公式计算第一算力接入能力值K:Substitute the content matching degree C, the memory matching degree M and the operating environment matching degree E into the first evaluation formula to calculate the first computing power access capability value K:
式中,α1、α2、Δ3都是权重系数,默认Δ1=Δ2=α3,可以根据待计算任务内容选择权重比例;Wherein, α 1 , α 2 , and Δ 3 are all weight coefficients. By default, Δ 1 = Δ 2 = α 3 . The weight ratio can be selected according to the content of the task to be calculated.
根据公式获得第i个计算节点的第一算力接入能力值Ki;According to the formula Obtain the first computing power access capability value K i of the i-th computing node;
将第一算力接入能力值Ki与第一算力接入能力值阈值K0进行比较,若第一算力接入能力值Ki大于等于第一算力接入能力值阈值K0,则该计算节点满足第一算力接入能力值需求,确定该计算节点为第一待选计算节点,否则,则不满足第一算力接入能力值需求。The first computing power access capability value K i is compared with the first computing power access capability value threshold K 0. If the first computing power access capability value K i is greater than or equal to the first computing power access capability value threshold K 0 , the computing node meets the first computing power access capability value requirement, and the computing node is determined to be the first candidate computing node; otherwise, the first computing power access capability value requirement is not met.
可选的,所述内容匹配度C与待进行计算任务的内容和计算节点处理的计算任务内容相关,若待进行计算任务的内容完全在计算节点处理的计算任务内容范围内,则C=1;若待进行计算任务的内容完全不在计算节点处理的计算任务内容范围内,则X=0;若待进行计算任务的内容与计算节点处理的计算任务内容范围部分重叠,则X=0.5;Optionally, the content matching degree C is related to the content of the computing task to be performed and the content of the computing task processed by the computing node. If the content of the computing task to be performed is completely within the content range of the computing task processed by the computing node, then C=1; if the content of the computing task to be performed is completely outside the content range of the computing task processed by the computing node, then X=0; if the content of the computing task to be performed partially overlaps with the content range of the computing task processed by the computing node, then X=0.5;
所述内存匹配度M与待进行计算任务需求内存和计算节点当前剩余内存相关,若计算任务需求内存远小于计算节点当前剩余内存,则M=1;若若计算任务需求内存大于计算节点当前剩余内存,则M=0;若计算任务需求内存小于计算节点当前剩余内存且计算节点当前剩余内存减去计算任务需求内存低于设定值,则M=0.5;The memory matching degree M is related to the memory required by the computing task to be performed and the current remaining memory of the computing node. If the memory required by the computing task is much smaller than the current remaining memory of the computing node, then M=1; if the memory required by the computing task is larger than the current remaining memory of the computing node, then M=0; if the memory required by the computing task is smaller than the current remaining memory of the computing node and the current remaining memory of the computing node minus the memory required by the computing task is lower than the set value, then M=0.5;
所述操作环境匹配度E与待进行计算任务需求的运行环境和计算节点的操作系统相关,若待进行计算任务需求的运行环境低于计算节点的操作系统版本,则E=1;若待进行计算任务需求的运行环境高于计算节点的操作系统版本,则E=0。The operating environment matching degree E is related to the operating environment required by the computing task to be performed and the operating system of the computing node. If the operating environment required by the computing task to be performed is lower than the operating system version of the computing node, then E=1; if the operating environment required by the computing task to be performed is higher than the operating system version of the computing node, then E=0.
可选的,按第一算力接入能力值Ki的数值从大到小对第一算力接入能力值Ki进行排序,截取排名前n的第一待选计算节点作为第一评估结果。Optionally, the first computing power access capability values K i are sorted from large to small according to their values, and the top n first candidate computing nodes are intercepted as the first evaluation result.
可选的,所述第二匹配模块用于:Optionally, the second matching module is used for:
获取所述第一评估结果中所述第一待选计算节点的运行参数数据;Obtaining operating parameter data of the first candidate computing node in the first evaluation result;
所述运行参数数据包括:计算节点处理的任务数量m、计算节点处理第j个计算任务完成的时长Tj、第j个计算任务需求的计算时长T0j,计算节点处理m个任务的总时长TS、第j个计算任务的完成度Cj、第j个计算任务需求的完成度C0j;The operation parameter data includes: the number of tasks m processed by the computing node, the time T j for the computing node to complete the j-th computing task, the computing time T 0j required for the j-th computing task, the total time T S for the computing node to process m tasks, the completion degree C j of the j-th computing task, and the completion degree C 0j required for the j-th computing task;
将获取的参数代入第二评估公式计算第二算力接入能力值Q:Substitute the obtained parameters into the second evaluation formula to calculate the second computing power access capability value Q:
根据第二评估公式依次获取所述第一待选计算节点的第二算力接入能力值Q,将第二算力接入能力值Q与第二算力接入能力值阈值Q0比较,若第二算力接入能力值Q大于第二算力接入能力值阈值Q0,则当前节点满足第二算力接入能力值需求,将满足第二算力接入能力值需求的第一待选计算节点作为第二待选计算节点,作为第二次评估结果。According to the second evaluation formula, the second computing power access capability value Q of the first candidate computing node is obtained in sequence, and the second computing power access capability value Q is compared with the second computing power access capability value threshold Q 0. If the second computing power access capability value Q is greater than the second computing power access capability value threshold Q 0 , the current node meets the second computing power access capability value requirement, and the first candidate computing node that meets the second computing power access capability value requirement is used as the second candidate computing node as the second evaluation result.
可选的,所述第三评估模块用于:Optionally, the third evaluation module is used to:
获取所述第二待选计算节点并依次标号1、2、3…p;Obtain the second candidate computing nodes and label them 1, 2, 3, ..., p in sequence;
当前计算任务向最近的第二待选计算节点发出计算请求,若第二待选计算节点是空闲状态则将当前计算任务加入计算列表,进行任务处理,否则将当前计算任务转送给相邻的第二待选计算节点;The current computing task sends a computing request to the nearest second candidate computing node. If the second candidate computing node is idle, the current computing task is added to the computing list for task processing. Otherwise, the current computing task is forwarded to the adjacent second candidate computing node.
当所有的第二待选计算节点都接收到当前计算任务的计算请求,计算请求仍未得到响应,当前计算任务向最近的第二待选计算节点继续发出第二次计算请求;直至所述计算请求被响应加入计算节点的任务列表。When all second candidate computing nodes have received the computing request of the current computing task, and the computing request has not been responded to, the current computing task continues to send a second computing request to the nearest second candidate computing node; until the computing request is responded to and added to the task list of the computing node.
可选的,所述计算节点是否空闲判断方法是:获取第二待选计算节点的任务列表,如果列表中没有待处理的任务,那么节点被认为是空闲的。Optionally, the method for determining whether the computing node is idle is: obtaining a task list of the second candidate computing node; if there is no pending task in the list, the node is considered to be idle.
可选的,所述计算请求具有生命周期,在生命周期内没有被响应,则会立即被丢弃,所有计算节点都不会转发当前计算请求。Optionally, the computing request has a life cycle, and if it is not responded to within the life cycle, it will be discarded immediately, and all computing nodes will not forward the current computing request.
为了实现上述实施例,本申请还提出一种电子设备,包括:处理器,以及与所述处理器通信连接的存储器;所述存储器存储计算机执行指令;所述处理器执行所述存储器存储的计算机执行指令,以实现执行前述实施例所提供的系统。In order to implement the above embodiments, the present application also proposes an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the system provided by the above embodiments.
为了实现上述实施例,本申请还提出一种计算机可读存储介质,计算机可读存储介质中存储有计算机执行指令,所述计算机执行指令被处理器执行时用于实现前述实施例所提供的系统。In order to implement the above embodiments, the present application also proposes a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the system provided by the above embodiments.
为了实现上述实施例,本申请还提出一种计算机程序产品,包括计算机程序,该计算机程序被处理器执行时实现前述实施例所提供的系统。In order to implement the above embodiments, the present application also proposes a computer program product, including a computer program, which implements the system provided by the above embodiments when executed by a processor.
本申请中所涉及的用户个人信息的收集、存储、使用、加工、传输、提供和公开等处理,均符合相关法律法规的规定,且不违背公序良俗。The collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in this application are in compliance with relevant laws and regulations and do not violate public order and good morals.
需要说明的是,来自用户的个人信息应当被收集用于合法且合理的用途,并且不在这些合法使用之外共享或出售。此外,应在收到用户知情同意后进行此类采集/共享,包括但不限于在用户使用该功能前,通知用户阅读用户协议/用户通知,并签署包括授权相关用户信息的协议/授权。此外,还需采取任何必要步骤,保卫和保障对此类个人信息数据的访问,并确保有权访问个人信息数据的其他人遵守其隐私政策和流程。It should be noted that personal information from users should be collected for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. In addition, such collection/sharing should be carried out after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement/user notice and sign the agreement/authorization including authorization of relevant user information before the user uses the function. In addition, any necessary steps should be taken to protect and safeguard access to such personal information data and ensure that others who have access to personal information data comply with its privacy policy and procedures.
本申请预期可提供用户选择性阻止使用或访问个人信息数据的实施方案。即本公开预期可提供硬件和/或软件,以防止或阻止对此类个人信息数据的访问。一旦不再需要个人信息数据,通过限制数据收集和删除数据可最小化风险。此外,在适用时,对此类个人信息去除个人标识,以保护用户的隐私。The present application is expected to provide an implementation scheme for users to selectively block the use or access of personal information data. That is, the present disclosure is expected to provide hardware and/or software to prevent or block access to such personal information data. Once the personal information data is no longer needed, the risk can be minimized by limiting data collection and deleting the data. In addition, when applicable, such personal information is de-identified to protect the privacy of the user.
在前述各实施例描述中,参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本申请的至少一个实施例或示例中。在本说明书中,对上述术语的示意性表述不必须针对的是相同的实施例或示例。而且,描述的具体特征、结构、材料或者特点可以在任一个或多个实施例或示例中以合适的方式结合。此外,在不相互矛盾的情况下,本领域的技术人员可以将本说明书中描述的不同实施例或示例以及不同实施例或示例的特征进行结合和组合。In the description of the aforementioned embodiments, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
此外,术语“第一”、“第二”仅用于描述目的,而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此,限定有“第一”、“第二”的特征可以明示或者隐含地包括至少一个该特征。在本申请的描述中,“多个”的含义是至少两个,例如两个,三个等,除非另有明确具体的限定。In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
流程图中或在此以其他方式描述的任何过程或方法描述可以被理解为,表示包括一个或更多个用于实现定制逻辑功能或过程的步骤的可执行指令的代码的模块、片段或部分,并且本申请的优选实施方式的范围包括另外的实现,其中可以不按所示出或讨论的顺序,包括根据所涉及的功能按基本同时的方式或按相反的顺序,来执行功能,这应被本申请的实施例所属技术领域的技术人员所理解。Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.
在流程图中表示或在此以其他方式描述的逻辑和/或步骤,例如,可以被认为是用于实现逻辑功能的可执行指令的定序列表,可以具体实现在任何计算机可读介质中,以供指令执行系统、装置或设备(如基于计算机的系统、包括处理器的系统或其他可以从指令执行系统、装置或设备取指令并执行指令的系统)使用,或结合这些指令执行系统、装置或设备而使用。就本说明书而言,"计算机可读介质"可以是任何可以包含、存储、通信、传播或传输程序以供指令执行系统、装置或设备或结合这些指令执行系统、装置或设备而使用的装置。计算机可读介质的更具体的示例(非穷尽性列表)包括以下:具有一个或多个布线的电连接部(电子装置),便携式计算机盘盒(磁装置),随机存取存储器(RAM),只读存储器(ROM),可擦除可编辑只读存储器(EPROM或闪速存储器),光纤装置,以及便携式光盘只读存储器(CDROM)。另外,计算机可读介质甚至可以是可在其上打印所述程序的纸或其他合适的介质,因为可以例如通过对纸或其他介质进行光学扫描,接着进行编辑、解译或必要时以其他合适方式进行处理来以电子方式获得所述程序,然后将其存储在计算机存储器中。The logic and/or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.
应当理解,本申请的各部分可以用硬件、软件、固件或它们的组合来实现。在上述实施方式中,多个步骤或方法可以用存储在存储器中且由合适的指令执行系统执行的软件或固件来实现。如,如果用硬件来实现和在另一实施方式中一样,可用本领域公知的下列技术中的任一项或他们的组合来实现:具有用于对数据信号实现逻辑功能的逻辑门电路的离散逻辑电路,具有合适的组合逻辑门电路的专用集成电路,可编程门阵列(PGA),现场可编程门阵列(FPGA)等。It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
本技术领域的普通技术人员可以理解实现上述实施例方法携带的全部或部分步骤是可以通过程序来指令相关的硬件完成,所述的程序可以存储于一种计算机可读存储介质中,该程序在执行时,包括方法实施例的步骤之一或其组合。A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.
此外,在本申请各个实施例中的各功能单元可以集成在一个处理模块中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个模块中。上述集成的模块既可以采用硬件的形式实现,也可以采用软件功能模块的形式实现。所述集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时,也可以存储在一个计算机可读取存储介质中。In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
上述提到的存储介质可以是只读存储器,磁盘或光盘等。尽管上面已经示出和描述了本申请的实施例,可以理解的是,上述实施例是示例性的,不能理解为对本申请的限制,本领域的普通技术人员在本申请的范围内可以对上述实施例进行变化、修改、替换和变型。The storage medium mentioned above may be a read-only memory, a disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.
Claims (10)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202410656568.2A CN118656203A (en) | 2024-05-24 | 2024-05-24 | Computing power access capability evaluation system for supercomputing |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202410656568.2A CN118656203A (en) | 2024-05-24 | 2024-05-24 | Computing power access capability evaluation system for supercomputing |
Publications (1)
Publication Number | Publication Date |
---|---|
CN118656203A true CN118656203A (en) | 2024-09-17 |
Family
ID=92699837
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202410656568.2A Pending CN118656203A (en) | 2024-05-24 | 2024-05-24 | Computing power access capability evaluation system for supercomputing |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN118656203A (en) |
-
2024
- 2024-05-24 CN CN202410656568.2A patent/CN118656203A/en active Pending
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US12056583B2 (en) | Target variable distribution-based acceptance of machine learning test data sets | |
CN106055654B (en) | Heterogeneous data integration method and device | |
WO2021243508A1 (en) | Scanning for information according to scan objectives | |
CN113268403B (en) | Time series analysis and forecasting methods, devices, equipment and storage media | |
CN118822660A (en) | Method, device, electronic device and storage medium for collaborative activation of computing resources | |
CN113157671A (en) | Data monitoring method and device | |
CN111859985B (en) | AI customer service model test method and device, electronic equipment and storage medium | |
CN110852384B (en) | Medical image quality detection method, device and storage medium | |
WO2006067026A1 (en) | Method for remembering resource allocation in grids | |
CN108762684B (en) | Hot spot data migration flow control method and device, electronic equipment and storage medium | |
WO2025092288A1 (en) | Model training method and apparatus, text information detection method and apparatus, and device and medium | |
CN118656203A (en) | Computing power access capability evaluation system for supercomputing | |
CN114067964A (en) | Medical image data processing method and device, computer equipment and storage medium | |
CN116827817B (en) | Data link state monitoring method, device, monitoring system and storage medium | |
US8954974B1 (en) | Adaptive lock list searching of waiting threads | |
CN117076579A (en) | Method, device, equipment and storage medium for displaying data blood relationship | |
CN116957354A (en) | A policy evolution path analysis method, device and electronic equipment | |
CN116912771A (en) | Cloud examination room control method, device, equipment and storage medium | |
CN113296951B (en) | Resource allocation scheme determining method and equipment | |
CN111652741B (en) | User preference analysis method, device and readable storage medium | |
CN114443253A (en) | Disk resource scheduling method, device, electronic device, medium and program product | |
CN113918313A (en) | Data processing method, device, equipment and storage medium | |
CN112559331A (en) | Test method and device | |
CN111523681A (en) | Global feature importance representation method and device, electronic equipment and storage medium | |
TWI777481B (en) | Data control method, data processing method, electronic equipment and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |