WO2022012046A1 - 一种深度学习基准测试优化方向的选择方法及装置 - Google Patents

一种深度学习基准测试优化方向的选择方法及装置 Download PDF

Info

Publication number
WO2022012046A1
WO2022012046A1 PCT/CN2021/077359 CN2021077359W WO2022012046A1 WO 2022012046 A1 WO2022012046 A1 WO 2022012046A1 CN 2021077359 W CN2021077359 W CN 2021077359W WO 2022012046 A1 WO2022012046 A1 WO 2022012046A1
Authority
WO
WIPO (PCT)
Prior art keywords
link
optimization
performance
calculation
deep learning
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2021/077359
Other languages
English (en)
French (fr)
Inventor
赵涟水
吴韶华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Wave Intelligent Technology Co Ltd
Original Assignee
Suzhou Wave Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Wave Intelligent Technology Co Ltd filed Critical Suzhou Wave Intelligent Technology Co Ltd
Publication of WO2022012046A1 publication Critical patent/WO2022012046A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Prevention of errors by analysis, debugging or testing of software
    • G06F11/3604—Analysis of software for verifying properties of programs
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"

Definitions

  • the invention relates to the field of deep learning benchmark test optimization, in particular to a method and device for selecting a deep learning benchmark test optimization direction.
  • benchmark testing plays an important role in the iteration of hardware and software. Its test results can not only measure whether the designed hardware or software achieves the expected goals, but also provide developers with a direction for further optimization.
  • the demand for computing speed has promoted the development of computing platforms from CPU to GPU and various ASICs.
  • many related benchmarking frameworks have also emerged.
  • the development trend of these benchmarking frameworks includes covering more comprehensive deep learning models, supporting more deep learning frameworks, and providing more comprehensive optimization directions.
  • scheme (1) only considers the running time of each computing link, and does not consider whether each computing link has the possibility of further optimization, that is, the performance improvement space brought by its optimization. For example, it is determined that the running time of the conv operation is the longest, and the conv needs to be further optimized, but if the conv has reached the maximum performance that the hardware can support, then there is no need to optimize it; if the conv does not reach the maximum performance, then through optimization, Whether the theoretical benefits it brings are worth further optimization.
  • Solution (2) only qualitatively uses it to determine whether the relevant computing links are limited in memory bandwidth or limited in computation, and there is no limitation of solution (1).
  • the present invention provides a method and device for selecting an optimization direction of a deep learning benchmark test, which takes into account not only the running time of each calculation link, but also the possible optimization effect, and comprehensively judges and selects the optimization object.
  • the technical scheme of the present invention is: a method for selecting an optimization direction of a deep learning benchmark test, the deep learning process includes a plurality of calculation links, and the method includes the following steps:
  • the optimization benefit is positively correlated with the running time of the computing link, and is also positively correlated with the degree of improvement that can be obtained after the computing link is optimized;
  • the calculation link whose optimization effect is greater than the preset value is selected as the optimization object.
  • calculation of the optimization benefits of each calculation link specifically includes:
  • the performance parameters include running time, the number of performance factors executed, and the amount of data flowing through the registers;
  • the roof line model is used to determine the maximum performance of each calculation link according to the performance intensity of each calculation link;
  • the optimization benefit of each computing link is calculated.
  • the performance intensity of the calculation link the number of performance factors executed/the amount of data flowing through the register.
  • ⁇ save_i is the optimization benefit of the i-th calculation link
  • T exp_i is the running time of the i-th calculation link
  • T tatal is the running time of the whole process when the deep learning is not optimized
  • P gap_i is the i-th calculation link that can be improved performance
  • P roofline_i is the maximum performance of the i-th computing link
  • the number of performance factors executed refers to the number of instructions executed.
  • the technical solution of the present invention also includes a device for selecting an optimization direction of a deep learning benchmark test, the deep learning process includes a plurality of computing links, and the device includes:
  • Optimization benefit calculation module calculates the optimization benefit of each calculation link; the optimization benefit is related to the running time of the calculation link and the degree of improvement that can be obtained after the calculation link is optimized;
  • Calculation link selection module According to the calculation result, select the calculation link whose optimization effect is greater than the preset value as the optimization object.
  • optimization benefit calculation module includes,
  • Parameter obtaining unit obtains the performance parameters of each computing link; the performance parameters include running time, the number of performance factors executed and the amount of data flowing through the registers;
  • Performance intensity calculation unit Calculate the performance intensity of each calculation link according to the number of performance factors executed by each calculation link and the amount of data flowing through the register;
  • the roof line model is used to determine the maximum performance of each calculation link according to the performance intensity of each calculation link;
  • Optimization benefit calculation execution unit Based on the performance parameters and maximum performance of each calculation link, the optimization benefit of each calculation link is calculated.
  • the performance intensity of the calculation link the number of performance factors executed/the amount of data flowing through the register.
  • T exp_i is the i-th calculation part running time
  • P gap_i i-th calculation part can improve the performance
  • P roofline_i is The maximum performance of the i-th computing link
  • the number of performance factors executed refers to the number of instructions executed.
  • the present invention provides a method and device for selecting an optimization direction for a deep learning benchmark test.
  • the optimization object is selected according to the optimization benefits of each calculation link, wherein the optimization benefits are related to the running time of the calculation link and the degree of improvement that can be obtained after the calculation link is optimized.
  • the optimization benefit is positively correlated with the running time of the calculation link, and also has a positive correlation with the degree of improvement that can be obtained after the calculation link is optimized, so that the final result can reflect that the longer the running time of a calculation link, the more it should be listed.
  • it also reflects the benefits that a certain computing link can bring after optimization.
  • FIG. 1 is a schematic flowchart of a method according to a specific embodiment of the present invention.
  • Figure 2 is a schematic diagram of the roofline model curve.
  • FIG. 3 is a schematic block diagram of the structure of the second embodiment of the present invention.
  • CPU Central Processing Unit central processing unit
  • ASIC Application Specific Integrated Circuit
  • Roofline model a method to visually compare the theoretical performance of hardware with the performance of the tested software
  • this embodiment provides a selection of the optimization direction of the deep learning benchmark test
  • the realization of the method not only reflects the longer the running time of a certain computing link, the more it should be listed as an optimization object; it also reflects the benefits that a computing link can bring after optimization.
  • the method of this embodiment includes the following steps.
  • S1 calculate the optimization benefit of each computing link; the optimization benefit is positively correlated with the running time of the computing link, and is also positively correlated with the degree of improvement that can be obtained after the computing link is optimized;
  • the optimization benefit which is positively correlated with the running time and the degree of improvement that can be obtained after optimization, is taken as a comprehensive measure, and the calculation link for optimization is selected through the optimization benefit, so as to realize the comprehensive consideration of the running time and the benefit brought by optimization.
  • the optimization benefit of each calculation link can be calculated by collecting the performance parameters of the calculation link and using the roofline model. Specifically include the following steps:
  • the performance parameters include running time, the number of performance factors executed, and the amount of data flowing through the registers;
  • the roofline model is used to determine the maximum performance of each computing link according to the performance strength of each computing link;
  • the effective benefit is calculated by the following formula, and the optimized benefit is denoted as ⁇ save .
  • ⁇ save_i is the optimization benefit of the i-th computing link.
  • T exp_i represents the measured running time of computing link i
  • T roofline_i represents the time required if computing link i runs at its maximum performance
  • T save_i represents if the optimization Compute link i, the maximum running time it can save
  • P exp_i represents the actual performance of computing link i
  • P roofline_i represents the maximum performance of computing link i
  • P gap_i represents the performance that can be improved by computing link i.
  • ⁇ save_i is the comprehensive index to measure and select the optimization object proposed by the present invention, that is, the maximum benefit brought by the optimization of a certain link—the reduction of the running time, which reflects the guidance that the longer the running time of the link, the more it should be listed as the optimization object.
  • the target ( ⁇ exp_i ) also reflects the room for improving the performance of the link itself At the same time, the roofline model is quantitatively applied.
  • the operation types include not only floating-point operations, but also other types such as integers, and in some cases, the proportion of integers accounts for the vast majority. Therefore, in the roofline model, the performance indicators are not Only floating point arithmetic should be covered, and all types involved should be covered more fully.
  • the number of instructions is used as an index of performance, that is, the performance factor is the number of instructions.
  • the performance strength of this embodiment is the instruction strength (number of instructions per bit).
  • the amount of data (total_bytes) then the instruction strength of the calculation link i is total_instructions/total_bytes.
  • total_instructions and total_bytes can be provided by tracking tools in the hardware system, such as on NVIDIA GPUs, these two parameters can be provided by CUPTI (NVIDIA CUDAProfiling Tools Interface).
  • the performance of the computing link refers to the number of performance factors executed per second, and this embodiment specifically refers to the number of instructions executed per second.
  • the maximum performance of the computing link it is based on the roofline model of hardware performance and the instruction intensity of the computing link. As shown in Figure 2, the solid line is the performance roofline model curve of the hardware, the abscissa is the instruction strength (unit is the number of instructions per bit), and the ordinate is the performance (the unit is the number of instructions per second).
  • the instruction intensity II 1 a horizontal axis perpendicular to the straight line, the intersection of the straight line and roofline curve is the model calculates the maximum performance in part II 1 1 Videos position, i.e., P roofline_1; for link 2, the instruction intensity II 2, the intersection of a line perpendicular to the horizontal axis drawn II 2 position, the straight line and roofline curve model is the maximum performance part 2, i.e., P roofline_2.
  • each hardware has a performance roofline model curve, that is, the performance of the running software cannot break through the roofline performance of the hardware.
  • the solid line represents the performance roofline of a certain hardware.
  • the performance of running software is not above this roofline.
  • the roofline can be provided by the hardware supplier, or the user can measure it through some specific software.
  • this embodiment provides an apparatus for selecting an optimization direction for a deep learning benchmark test, including the following functional modules.
  • Optimization benefit calculation module 101 Calculate the optimization benefit of each calculation link; the optimization benefit is related to the running time of the calculation link and the degree of improvement that can be obtained after the calculation link is optimized;
  • Calculation link selection module 102 According to the calculation result, select the calculation link whose optimization effect is greater than the preset value as the optimization object.
  • the optimization benefit calculation module 101 includes the following functional units.
  • Parameter obtaining unit 101-1 obtains performance parameters of each computing link; the performance parameters include running time, the number of performance factors executed, and the amount of data flowing through the registers;
  • Performance intensity calculation unit 101-2 calculates the performance intensity of each calculation link according to the number of performance factors executed by each calculation link and the amount of data flowing through the register;
  • the maximum performance determination unit 101-3 adopt the roof line model, and determine the maximum performance of each calculation link according to the performance intensity of each calculation link;
  • the optimization benefit calculation execution unit 101-4 based on the performance parameters and the maximum performance of each calculation link, calculate the optimization benefit of each calculation link.
  • the performance intensity of the calculation link the number of performance factors executed/the amount of data flowing through the register.
  • the optimization benefit is calculated by the following formula, denoting the optimization benefit as ⁇ save , then
  • ⁇ save_i is the optimization benefit of the i-th calculation link
  • T exp_i is the running time of the i-th calculation link
  • T tatal is the running time of the whole process when the deep learning is not optimized
  • P gap_i is the i-th calculation link that can be improved performance
  • P roofline_i is the maximum performance of the i-th computing link
  • the operation types include not only floating-point operations, but also other types such as integers, and in some cases, the proportion of integers accounts for the vast majority. Therefore, in the roofline model, the performance indicators should not only be Including floating point arithmetic, but should cover all the types involved more comprehensively. Therefore, in the present invention, the number of instructions is used as an index of performance, that is, the performance factor is the number of instructions.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Strategic Management (AREA)
  • Human Resources & Organizations (AREA)
  • General Engineering & Computer Science (AREA)
  • Economics (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • General Business, Economics & Management (AREA)
  • Game Theory and Decision Science (AREA)
  • Operations Research (AREA)
  • Artificial Intelligence (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Tourism & Hospitality (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Development Economics (AREA)
  • Computing Systems (AREA)
  • Marketing (AREA)
  • Mathematical Physics (AREA)
  • Computer Hardware Design (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

一种深度学习基准测试优化方向的选择方法及装置，深度学习过程包含多个计算环节；本发明计算各个计算环节的优化效益；所述优化效益与计算环节的运行时长呈正相关，同时也与计算环节优化后所能获得的改善程度呈正相关；根据计算结果，选取优化效果大于预设值的计算环节作为优化对象。本发明使最终结果既能体现某个计算环节运行时间越长，其越应被列为优化对象，也体现某个计算环节经优化后能带来的效益。

Description

一种深度学习基准测试优化方向的选择方法及装置
本申请要求于2020年07月17日提交中国专利局、申请号为202010694596.5、发明名称为“一种深度学习基准测试优化方向的选择方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及深度学习基准测试优化领域，具体涉及一种深度学习基准测试优化方向的选择方法及装置。
背景技术
在计算机领域，基准测试在硬件及软件的迭代中扮演着重要的角色，它的测试结果不仅可以衡量所设计的硬件或软件是否达到预期目标，还可以为开发者提供进一步优化的方向。随着深度学习领域的发展，计算速度的需求推动了计算平台从CPU到GPU及各种ASICs的发展，同时，很多相关评测基准框架也相继而生。这些基准测试框架的发展趋势包括覆盖更加全面的深度学习模型、支持更多的深度学习框架、提供更加全面的优化方向等。
针对基准测试的优化方向，现有技术一般有两种解决方案：(1)基于各个计算环节(深度学习计算中，存在conv、pooling、relu、fc等各个计算环节)的运行时长来决定，某个计算环节的运行时间越长，就越该对此计算环节进行优化；(2)在方案(1)的基础上，引入roofline模型辅助判断，即通过各个计算环节的运行时间决定优化对象，再通过roofline模型判断各个计算环节是处于内存带宽受限还是计算受限，如果是内存带宽受限，利用增强缓存重使用等优化方法来提高性能；如果是计算受限，通过增加算法的并行度等优化方法来提高性能。
但方案(1)只考虑了各个计算环节的运行时长，没有考虑各个计算环节是否存在可进一步优化的可能性即其优化带来的性能提升空间。比如，确定conv运算的运行时间最长，conv需要进一步优化，但是如果conv已经达到了硬件所能支持的最大性能，那么就没有必要对其进行优化；如果conv 没有达到最大性能，那么通过优化，其带来的理论收益是否值得做进一步优化。方案(2)只是定性地利用其来判断相关计算环节是内存带宽受限还是计算受限，没有解决方案(1)的局限性。
发明内容
为解决上述问题，本发明提供一种深度学习基准测试优化方向的选择方法及装置，既考虑各个计算环节的运行时间，又考虑其可能取得的优化效果，综合判断选取优化对象。
本发明的技术方案是：一种深度学习基准测试优化方向的选择方法，深度学习过程包含多个计算环节，该方法包括以下步骤：
计算各个计算环节的优化效益；所述优化效益与计算环节的运行时长呈正相关，同时也与计算环节优化后所能获得的改善程度呈正相关；
根据计算结果，选取优化效果大于预设值的计算环节作为优化对象。
进一步地，所述计算各个计算环节的优化效益，具体包括：
获取各个计算环节的性能参数；所述性能参数包括运行时间、执行的性能因子数量和流经寄存器的数据量；
根据各个计算环节执行的性能因子数量和流经寄存器的数据量，计算各个计算环节的性能强度；
采用屋顶线模型，根据各个计算环节的性能强度确定各个计算环节的最大性能；
基于各个计算环节的性能参数和最大性能，计算各个计算环节的优化效益。
进一步地，计算环节的性能强度＝执行的性能因子数量/流经寄存器的数据量。
进一步地，记优化效益为η save，则
Figure PCTCN2021077359-appb-000001
其中，η save_i是第i个计算环节的优化效益，T exp_i是第i计算环节的运行时间，T tatal是深度学习未优化时整个流程的运行时间，P gap_i为第i个计算环 节可提升的性能，P roofline_i是第i个计算环节的最大性能；
其中P gap_i＝P roofline_i-P exp_i，
Figure PCTCN2021077359-appb-000002
进一步地，执行的性能因子数量是指执行的指令数。
本发明的技术方案还包括一种深度学习基准测试优化方向的选择装置，深度学习过程包含多个计算环节，该装置包括，
优化效益计算模块：计算各个计算环节的优化效益；所述优化效益与计算环节的运行时长及计算环节优化后所能获得的改善程度有关；
计算环节选取模块：根据计算结果，选取优化效果大于预设值的计算环节作为优化对象。
进一步地，优化效益计算模块包括，
参数获取单元：获取各个计算环节的性能参数；所述性能参数包括运行时间、执行的性能因子数量和流经寄存器的数据量；
性能强度计算单元：根据各个计算环节执行的性能因子数量和流经寄存器的数据量，计算各个计算环节的性能强度；
最大性能确定单元：采用屋顶线模型，根据各个计算环节的性能强度确定各个计算环节的最大性能；
优化效益计算执行单元：基于各个计算环节的性能参数和最大性能，计算各个计算环节的优化效益。
进一步地，计算环节的性能强度＝执行的性能因子数量/流经寄存器的数据量。
进一步地，记优化效益为η save，则
Figure PCTCN2021077359-appb-000003
其中，
η save_i是第i个计算环节的优化效益，T exp_i是第i计算环节的运行时间，T tatal是深度学习整个流程的运行时间，P gap_i为第i个计算环节可提升的性能，P roofline_i是第i个计算环节的最大性能；
其中P gap_i＝P roofline_i-P exp_i，
Figure PCTCN2021077359-appb-000004
进一步地，执行的性能因子数量是指执行的指令数。
本发明提供的一种深度学习基准测试优化方向的选择方法及装置，根据各个计算环节的优化效益选择优化对象，其中优化效益与计算环节的运行时长及计算环节优化后所能获得的改善程度有关，且优化效益与计算环节的运行时长呈正相关性，与计算环节优化后所能获得的改善程度也呈正相关性，使最终结果既能体现某个计算环节运行时间越长，其越应被列为优化对象，也体现某个计算环节经优化后能带来的效益。
附图说明
图1是本发明具体实施例一方法流程示意图。
图2是roofline模型曲线示意图。
图3是本发明具体实施例二结构示意框图。
具体实施方式
下面结合附图并通过具体实施例对本发明进行详细阐述，以下实
施例是对本发明的解释，而本发明并不局限于以下实施方式。
以下对本发明涉及的英文名词进行解释。
(1)CPU:Central Processing Unit中央处理器；
(2)GPU：Graphical Processing Unit图像处理器；
(3)ASIC：Application Specific Integrated Circuit专用集成电路；
(4)Roofline模型：屋顶线模型，一种将硬件理论性能与所测试软件的性能进行可视化比较的方法；
(5)Conv:convolution深度学习计算中的卷积运算；
(6)Pooling：深度学习计算中的池化运算；
(7)Relu:rectified linear unit深度学习计算中的线性整；
流函数；
(8)Fc:fully connected深度学习计算中的全链接运算。
实施例一
在决定优化对象时，如何合理地考虑各个计算环节的运行时间及其可能取得的优化效果，需要一个定量的综合指标去衡量，基于此，本实施例提供一种深度学习基准测试优化方向的选择方法，实现既体现某计算环节运行时间越长，其越应该被列为优化对象；也体现某计算环节经优化后能带来的效益。
如图1所示，本实施例的方法包括以下步骤。
S1，计算各个计算环节的优化效益；优化效益与计算环节的运行时长呈正相关，同时也与计算环节优化后所能获得的改善程度呈正相关；
S2，根据计算结果，选取优化效果大于预设值的计算环节作为优化对象。
本方法将与运行时长、优化后所能获得的改善程度均呈正相关的优化效益作为综合衡量指标，通过优化效益选取进行优化的计算环节，实现运行时长和优化后带来效益的综合考量。
其中，各个计算环节的优化效益可通过采集计算环节的性能参数，借助roofline模型进行计算。具体包括以下步骤：
S101，获取各个计算环节的性能参数；所述性能参数包括运行时间、执行的性能因子数量和流经寄存器的数据量；
S102，根据各个计算环节执行的性能因子数量和流经寄存器的数据量，计算各个计算环节的性能强度；
S103，采用roofline模型，根据各个计算环节的性能强度确定各个计算环节的最大性能；
S104，基于各个计算环节的性能参数和最大性能，计算各个计算环节的优化效益。
本实施例具体通过以下公式计算有效效益，记优化效益为η save。
Figure PCTCN2021077359-appb-000005
其中，η save_i是第i个计算环节的优化效益。
上式(1)推导过程如下：
Figure PCTCN2021077359-appb-000006
其中i表示各个计算环节中的某个计算环节；T exp_i表示计算环节i经测量得到的运行时间；T roofline_i表示如果计算环节i以其最大性能运行，其所需要的时间；T save_i表示如果优化计算环节i，其能节省的最大运行时间；P exp_i表示计算环节i的实际性能；P roofline_i表示计算环节i的最大性能；P gap_i表示计算环节i可提升的性能。
其中，P gap_i＝P roofline_i-P exp_i,
Figure PCTCN2021077359-appb-000007
上式(2)两边同时除以未优化时整个流程的运行时间T tatal，得到
Figure PCTCN2021077359-appb-000008
其中η save_i＝T save_i/T tatal表示计算环节i经优化后运行时间的减少占总时间的比值；η exp_i＝T exp_i/T tatal表示计算环节i在优化之前的运行时间占总时间的比值；
Figure PCTCN2021077359-appb-000009
表示计算环节i的性能提升占最大性能的比值。
η save_i即为本发明提出的衡量选择优化对象的综合指标，即某环节的优化所带来的最大效益—运行时间的减少，其体现了环节运行时间越长越应该被列为优化对象的指导方针(η exp_i)，也体现了环节本身性能可提升的 空间
Figure PCTCN2021077359-appb-000010
同时定量地应用了roofline模型。
另外，在深度学习计算中，运算类型不仅包含浮点数运算，还包括整型等其它类型，且在有些情况下，整型的比例占绝大部分，因此，在roofline模型中，性能的指标不应该仅包括浮点数运算，而应该更全面地覆盖所有涉及到的类型。在本发明中，使用指令数作为性能的指标，即性能因子为指令数。
本实施例的性能强度即指令强度(指令数每比特)，在计算某个环节i的计算指令强度时，需要测量该计算环节所涉及到的所有指令数(total_instructions)和该环节中流经寄存器的数据量(total_bytes)，那么该计算环节i的指令强度为total_instructions/total_bytes。total_instructions和total_bytes可由硬件系统中的跟踪工具提供，比如在英伟达的GPU上，这两个参数可由CUPTI(NVIDIA CUDAProfiling Tools Interface)提供。
计算环节的性能是指每秒所执行的性能因子数量，本实施例具体指每秒所执行的指令数。针对计算环节的最大性能，其是建立在硬件性能的roofline模型和计算环节的指令强度上的。如图2所示，实线为该硬件的性能roofline模型曲线，横坐标为指令强度(单位为指令数每比特)，纵坐标为性能(单位为指令数每秒)。
对于计算环节1，其指令强度为II 1，在II 1位置画一条与横轴垂直的直线，该直线与roofline模型曲线的交点即为计算环节1的最大性能，即P roofline_1；对于环节2，其指令强度为II 2，在II 2位置画一条与横轴垂直的直线，该直线与roofline模型曲线的交点即为环节2的最大性能，即P roofline_2。
需要说明的是，每个硬件都具有一个性能的roofline模型曲线，即运 行的软件性能不可能突破硬件的roofline性能，如图2所示，实线代表某硬件的性能roofline，所有在该硬件上运行的软件性能都不在该roofline之上。该roofline可由硬件的供应商提供，或者用户可以通过一些特定的软件去测量。
实施例二
如图3所示，基于实施例一，本实施例提供一种深度学习基准测试优化方向的选择装置，包括以下功能模块。
优化效益计算模块101：计算各个计算环节的优化效益；优化效益与计算环节的运行时长及计算环节优化后所能获得的改善程度有关；
计算环节选取模块102：根据计算结果，选取优化效果大于预设值的计算环节作为优化对象。
其中，优化效益计算模块101包括以下功能单元。
参数获取单元101-1：获取各个计算环节的性能参数；所述性能参数包括运行时间、执行的性能因子数量和流经寄存器的数据量；
性能强度计算单元101-2：根据各个计算环节执行的性能因子数量和流经寄存器的数据量，计算各个计算环节的性能强度；
最大性能确定单元101-3：采用屋顶线模型，根据各个计算环节的性能强度确定各个计算环节的最大性能；
优化效益计算执行单元101-4：基于各个计算环节的性能参数和最大性能，计算各个计算环节的优化效益。
计算环节的性能强度＝执行的性能因子数量/流经寄存器的数据量。
本实施例中，优化效益由以下公式计算，记优化效益为η save，则
Figure PCTCN2021077359-appb-000011
其中，η save_i是第i个计算环节的优化效益，T exp_i是第i计算环节的运行时间，T tatal是深度学习未优化时整个流程的运行时间，P gap_i为第i个计算环节可提升的性能，P roofline_i是第i个计算环节的最大性能；
其中P gap_i＝P roofline_i-P exp_i，
Figure PCTCN2021077359-appb-000012
上式推导过程如实施例一所示。
在深度学习计算中，运算类型不仅包含浮点数运算，还包括整型等其它类型，且在有些情况下，整型的比例占绝大部分，因此，在roofline模型中，性能的指标不应该仅包括浮点数运算，而应该更全面地覆盖所有涉及到的类型。因此在本发明中，使用指令数作为性能的指标，即性能因子为指令数。
以上公开的仅为本发明的优选实施方式，但本发明并非局限于此，任何本领域的技术人员能思之的没有创造性的变化，以及在不脱离本发明原理前提下所作的若干改进和润饰，都应落在本发明的保护范围内。

Claims (10)

  1. 一种深度学习基准测试优化方向的选择方法，深度学习过程包含多个计算环节，其特征在于，该方法包括以下步骤：
    计算各个计算环节的优化效益；所述优化效益与计算环节的运行时长呈正相关，同时也与计算环节优化后所能获得的改善程度呈正相关；
    根据计算结果，选取优化效果大于预设值的计算环节作为优化对象。
  2. 根据权利要求1所述的深度学习基准测试优化方向的选择方法，其特征在于，所述计算各个计算环节的优化效益，具体包括：
    获取各个计算环节的性能参数；所述性能参数包括运行时间、执行的性能因子数量和流经寄存器的数据量；
    根据各个计算环节执行的性能因子数量和流经寄存器的数据量，计算各个计算环节的性能强度；
    采用屋顶线模型，根据各个计算环节的性能强度确定各个计算环节的最大性能；
    基于各个计算环节的性能参数和最大性能，计算各个计算环节的优化效益。
  3. 根据权利要求2所述的深度学习基准测试优化方向的选择方法，其特征在于，计算环节的性能强度＝执行的性能因子数量/流经寄存器的数据量。
  4. 根据权利要求3所述的深度学习基准测试优化方向的选择方法，其特征在于，记优化效益为η save，则
    Figure PCTCN2021077359-appb-100001
    其中，η save_i是第i个计算环节的优化效益，T exp_i是第i计算环节的运行时间，T tatal是深度学习未优化时整个流程的运行时间，P gap_i为第i个计算环节可提升的性能，P roofline_i是第i个计算环节的最大性能；
    其中P gap_i＝P roofline_i-P exp_i，
    Figure PCTCN2021077359-appb-100002
  5. 根据权利要求2、3或4所述的深度学习基准测试优化方向的选择方法，其特征在于，执行的性能因子数量是指执行的指令数。
  6. 一种深度学习基准测试优化方向的选择装置，深度学习过程包含多个计算环节，其特征在于，该装置包括，
    优化效益计算模块：计算各个计算环节的优化效益；所述优化效益与计算环节的运行时长及计算环节优化后所能获得的改善程度有关；
    计算环节选取模块：根据计算结果，选取优化效果大于预设值的计算环节作为优化对象。
  7. 根据权利要求6所述的深度学习基准测试优化方向的选择装置，其特征在于，优化效益计算模块包括，
    参数获取单元：获取各个计算环节的性能参数；所述性能参数包括运行时间、执行的性能因子数量和流经寄存器的数据量；
    性能强度计算单元：根据各个计算环节执行的性能因子数量和流经寄存器的数据量，计算各个计算环节的性能强度；
    最大性能确定单元：采用屋顶线模型，根据各个计算环节的性能强度确定各个计算环节的最大性能；
    优化效益计算执行单元：基于各个计算环节的性能参数和最大性能，计算各个计算环节的优化效益。
  8. 根据权利要求7所述的深度学习基准测试优化方向的选择装置，其特征在于，计算环节的性能强度＝执行的性能因子数量/流经寄存器的数据量。
  9. 根据权利要求8所述的深度学习基准测试优化方向的选择装置，其特征在于，记优化效益为η save，则
    Figure PCTCN2021077359-appb-100003
    其中，η save_i是第i个计算环节的优化效益，T exp_i是第i计算环节的运行时间，T tatal是深度学习整个流程的运行时间，P gap_i为第i个计算环节可提升的性能，P roofline_i是第i个计算环节的最大性能；
    其中P gap_i＝P roofline_i-P exp_i，
    Figure PCTCN2021077359-appb-100004
  10. 根据权利要求7、8或9所述的深度学习基准测试优化方向的选择装置，其特征在于，执行的性能因子数量是指执行的指令数。
PCT/CN2021/077359 2020-07-17 2021-02-23 一种深度学习基准测试优化方向的选择方法及装置 Ceased WO2022012046A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010694596.5A CN111967606A (zh) 2020-07-17 2020-07-17 一种深度学习基准测试优化方向的选择方法及装置
CN202010694596.5 2020-07-17

Publications (1)

Publication Number Publication Date
WO2022012046A1 true WO2022012046A1 (zh) 2022-01-20

Family

ID=73361888

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/077359 Ceased WO2022012046A1 (zh) 2020-07-17 2021-02-23 一种深度学习基准测试优化方向的选择方法及装置

Country Status (2)

Country Link
CN (1) CN111967606A (zh)
WO (1) WO2022012046A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111967606A (zh) * 2020-07-17 2020-11-20 苏州浪潮智能科技有限公司 一种深度学习基准测试优化方向的选择方法及装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110276096A (zh) * 2019-05-08 2019-09-24 北京百度网讯科技有限公司 提高深度学习模型预测能力的方法、电子设备和存储介质
US20200042419A1 (en) * 2018-07-31 2020-02-06 Alibaba Group Holding Limited System and method for benchmarking ai hardware using synthetic ai model
CN111242314A (zh) * 2020-01-08 2020-06-05 中国信息通信研究院 深度学习加速器基准测试方法和装置
US20200218985A1 (en) * 2019-01-03 2020-07-09 Alibaba Group Holding Limited System and method for synthetic-model-based benchmarking of ai hardware
CN111967606A (zh) * 2020-07-17 2020-11-20 苏州浪潮智能科技有限公司 一种深度学习基准测试优化方向的选择方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200042419A1 (en) * 2018-07-31 2020-02-06 Alibaba Group Holding Limited System and method for benchmarking ai hardware using synthetic ai model
US20200218985A1 (en) * 2019-01-03 2020-07-09 Alibaba Group Holding Limited System and method for synthetic-model-based benchmarking of ai hardware
CN110276096A (zh) * 2019-05-08 2019-09-24 北京百度网讯科技有限公司 提高深度学习模型预测能力的方法、电子设备和存储介质
CN111242314A (zh) * 2020-01-08 2020-06-05 中国信息通信研究院 深度学习加速器基准测试方法和装置
CN111967606A (zh) * 2020-07-17 2020-11-20 苏州浪潮智能科技有限公司 一种深度学习基准测试优化方向的选择方法及装置

Also Published As

Publication number Publication date
CN111967606A (zh) 2020-11-20

Similar Documents

Publication Publication Date Title
JP5961971B2 (ja) シミュレーション装置，方法，およびプログラム
US9207916B2 (en) Simulation apparatus, method and medium
Ilic et al. Cache-aware roofline model: Upgrading the loft
US8898390B2 (en) Scheduling workloads based on cache asymmetry
WO2023050671A1 (zh) 服务器故障定位方法、装置、电子设备及存储介质
JP6090327B2 (ja) ボトルネック検出装置、方法及びプログラム
US10564992B2 (en) Simulation apparatus and storage medium
Tiwari et al. Predicting optimal power allocation for cpu and dram domains
WO2022012046A1 (zh) 一种深度学习基准测试优化方向的选择方法及装置
Lawson et al. Energy evaluation for applications with different thread affinities on the Intel Xeon Phi
JP2017167930A (ja) 情報処理装置、電力測定方法及び電力測定プログラム
US7515152B2 (en) Adaptive sampling of a static data set
JP2013041513A (ja) 補正装置、補正方法、および補正プログラム
CN111090506A (zh) 一种基于软错误相关性的异构多核调度方法
JP5521687B2 (ja) 分析装置、分析方法および分析プログラム
Lee et al. Accurately approximating superscalar processor performance from traces
CN108846248B (zh) 一种应用建模及性能预测方法
Denoyelle et al. Automatic cache aware roofline model building and validation using topology detection
JP4983238B2 (ja) 模擬ソースプログラム生成プログラム及び装置、並びに情報処理方法
US7237166B2 (en) System and method for evaluating a multiprocessor system using a random bus traffic generation technique
TW201351129A (zh) 智慧測試系統及方法
CN115904839B (zh) 一种内存带宽的检测方法、装置、设备及存储介质
CN112231156B (zh) 一种spec cpu2017测试成绩预估方法、系统、装置及介质
CN116302900B (zh) 多接入边缘计算系统的算力可靠性评估方法
Adhianto et al. Hpctoolkit: Performance measurement and analysis for supercomputers with node-level parallelism

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21841416

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21841416

Country of ref document: EP

Kind code of ref document: A1