WO2024060916A1

WO2024060916A1 - Operator operation mode configuration method and apparatus, and related system

Info

Publication number: WO2024060916A1
Application number: PCT/CN2023/114411
Authority: WO
Inventors: 俞郑中
Original assignee: 华为技术有限公司
Priority date: 2022-09-22
Filing date: 2023-08-23
Publication date: 2024-03-28
Also published as: CN117785260A

Abstract

Embodiments of the present application provide an operator operation mode configuration method and apparatus, and a related system. The operator operation mode configuration method comprises: on the basis of an algorithm model, determining M operators corresponding to M operator nodes, wherein one operator node corresponds to one operator, each operator belongs to one type of N types of operators, each type of operators correspond to a plurality of operation modes, the operation precision and/or the operation speed of the operation modes are different, M is an integer greater than or equal to N, and N is an integer greater than or equal to 1; and on the basis of configuration parameters corresponding to each type of operators or configuration options corresponding to the operator nodes, configuring operation modes of the M operators in the algorithm model, wherein the configuration parameters or the configuration options are used for indicating operation modes of corresponding operators. By implementing the embodiments of the present application, the operation performance of operators can be improved while the operation precision of the operators in an operator model is ensured.

Description

Configuration method, device and related system for operator operation mode

This application claims the priority of the Chinese patent application submitted to the China Patent Office on September 22, 2022, with the application number 202211155932.4 and the application title "A configuration method, device and related system for operator operation mode", and its entire content incorporated herein by reference.

Technical field

The present application relates to the field of compilers, and in particular to a configuration method, device and related systems for an operator operation mode.

Background technique

When it comes to running AI models on dedicated artificial intelligence (AI) chips, the two most important points are the accuracy and performance of the chip, that is, whether the calculation results are accurate and whether the calculation speed is fast. The computing power of an AI chip is fixed. For example, the Ascend 710 AI processor can calculate float16 twice as fast as float32 in vector calculations. In some calculations, the computing power gap is even greater than that of float32. 2 times. To deal with this common computing power gap, the industry uses mixed precision methods. For example, the calculation types of some layers of the AI model are converted to fp16 calculations, and the calculations of some layers still maintain fp32 calculations. This kind of entire AI The model includes both fp16 and fp32 mixed precision calculation methods to improve the execution efficiency of the entire AI model without significant loss of accuracy.

However, the computing efficiency of the same chip in different types of operations is also very different. For example: exp, log, the computing efficiency of these computing chips for division is much lower than that of addition, subtraction, multiplication, and reciprocal operations. The existing AI framework does not provide a good expansion capability and can only perform operations according to a fixed processing method, which brings inconvenience to the performance and accuracy tuning of the AI model.

Contents of the invention

Embodiments of the present application provide an operator operation mode configuration method, device and related systems to ensure the operation accuracy of the operators in the operator model and improve the operation performance of the operators.

In the first aspect, embodiments of the present application provide a method for configuring operator operation modes, including: determining M operators corresponding to M operator nodes based on an algorithm model, where one operator node corresponds to one operator, and each The operators belong to one of the N types of operators. Each type of operator corresponds to multiple operating modes. The operating accuracy and/or operating speed between each of the operating modes are different; M is greater than or An integer equal to N, where N is an integer greater than or equal to 1; configure the M operators in the algorithm model based on the configuration parameters corresponding to each type of operator or the configuration options corresponding to each operator node The operating mode; wherein the configuration parameters or the configuration options are used to indicate the operating mode of the corresponding operator.

In the embodiment provided in the first aspect, after determining the operator corresponding to each of the M operator nodes of the operator model and the N types of operators corresponding to the M operators, the operator can be Configuration parameters corresponding to the operator or configuration options corresponding to each operator node configure the operation mode of the M operators in the algorithm model. Among them, the operators of each type of N types of operators can have a variety of operation modes with different operation accuracy and/or operation speed, which leads to the algorithm model also having multiple operation modes. In order to ensure that the operators in the operator model The operation accuracy and the operation performance of the operator can be improved at the same time. The embodiments of this application can be based on the operation characteristics of the operator model, based on the configuration parameters corresponding to each type of operator in the algorithm model or based on each operator node in the algorithm model. The corresponding configuration options configure the operation mode of the M operators in the algorithm model. The above-mentioned method of configuring the operation mode of the algorithm model through configuring parameters or configuration options can prevent the algorithm model in the existing technology from only performing operations according to a fixed processing method, making the optimization of the performance and accuracy of the algorithm model more flexible.

In one possible implementation, the method also includes: setting the configuration parameters corresponding to each type of operator in the operator interface corresponding to each type of operator in the N types of operators; configuring the operating modes of the M operators in the algorithm model based on the configuration parameters corresponding to each type of operator, including: calling the N types of operators and obtaining the corresponding configuration parameters through the operator interface corresponding to each type of operator in the N types of operators; configuring the operating modes of each type of operator in the algorithm model based on the configuration parameters corresponding to each type of operator in the N types of operators.

In the embodiment of this application, setting the configuration parameters corresponding to each type of operator in the operator implementation interface can realize operator-level precision mode setting. When calling each type of operator through the operator interface corresponding to this type of operator, operators with different operating modes can be compiled according to different configuration parameters in the operator implementation interface, and finally the accuracy and/or performance of the operator model can be adjusted. This makes the optimization of algorithm model performance and accuracy more flexible.

In a possible implementation manner, the configuration parameters are one of target configuration parameters, default configuration parameters, and tuning configuration parameters; Wherein, the target configuration parameter is used to instruct the corresponding operator to run in the target operation mode, and the target operation mode is one of the multiple operation modes; the default configuration parameter is used to instruct the corresponding operator to run in the highest operation mode. The precision operation mode or the fastest operation mode operation mode; the tuning configuration parameters are used to instruct the corresponding operator to run in the optimal operation mode, and the optimal operation mode is determined based on the operator model and the The above-mentioned operation mode with the highest operation accuracy has an operation mode with the operation result error within the preset threshold range and the fastest operation speed.

In the embodiment of the present application, the configuration parameters can be divided into three categories, for example, the configuration parameters can be any one of the target configuration parameters, the default configuration parameters, and the tuning configuration parameters. Among them, the target configuration parameters can be used to indicate an operation mode set by the user; the default configuration parameters can be used to indicate that the operator of this type is operated in the operation mode with the highest computing accuracy or the fastest computing speed; the tuning configuration parameters can be used to indicate that the operator is operated in the optimal operation mode under the computing characteristics of the current operator model.

In a possible implementation manner, the configuration parameters include a configuration path corresponding to the operating mode; and based on the configuration parameters corresponding to each type of the N types of operators, the configuration parameters in the algorithm model are The operation mode corresponding to each type of operator in the N types of operators includes: based on the configuration parameters corresponding to each type of operators in the N types of operators, obtaining the operation mode of each type of operators in the N types of operators. The configuration path of the corresponding operating mode of the operator; the corresponding configuration file is read based on the configuration path to configure the operating mode of the corresponding type of operator during the operation of the algorithm model.

In this embodiment of the present application, the configuration parameters may include corresponding operating mode configuration paths. The configuration path is used to indicate the configuration file corresponding to the operation mode. The configuration file includes relevant configuration information, so that after reading the configuration file, the operation mode of the corresponding type of operator can be configured during the operation of the algorithm model. For example: when the configuration parameters of the first type of operator are target configuration parameters, they can include the configuration path of the target operating mode, and read the configuration file of the target operating mode based on the configuration path of the target operating mode; and then configure the first operating mode based on the configuration file. Class operators run in target run mode.

In a possible implementation manner, determining the M operators corresponding to the M operator nodes based on the algorithm model includes: receiving input information and determining the M operator nodes and each of the M operator nodes based on the input information. The configuration options corresponding to the operator nodes; wherein the configuration options include one of target configuration parameters, default configuration parameters, and tuning configuration parameters; based on the configuration options corresponding to each operator node, Configuring the operation mode of the M operators in the algorithm model includes: calling the N types of operators respectively through the operator interface corresponding to each type of operator in the N types of operators; based on each of the The configuration options corresponding to the operator nodes configure the operation mode of the operator corresponding to each of the M operator nodes in the algorithm model.

In the embodiment of this application, in addition to setting the configuration parameters corresponding to each type of operator in the operator implementation interface, the configuration options corresponding to each operator node can also be set based on the user's input information to achieve operator node level Precision mode setting, that is, operators of the same type in different operator nodes can be configured with different operating modes. This configuration option is the same as the configuration parameter. You can also choose one of the three configuration parameters to set. When compiling the model, when each operator node calls this type of operator through the corresponding type of operator interface, it can be set according to the operator The configuration options corresponding to the node configure the operation mode of the operator of the operator node, and ultimately realize the adjustment of the accuracy and/or performance of the operator model, making the optimization of the performance and accuracy of the algorithm model more flexible.

In a possible implementation manner, the method further includes: compiling and running the algorithm model based on the operation modes of the M operators.

In the embodiment of the present application, after the operating modes of the M operators are determined, the operator model can be compiled according to the operating modes of the M operators, and finally the accuracy and/or performance of the operating operator model can be achieved.

In the second aspect, embodiments of the present application provide an operator operation mode configuration device, which may include:

A determination unit, configured to determine M operators corresponding to M operator nodes based on an algorithm model, wherein one operator node corresponds to one operator, each of the operators belongs to one of N types of operators, each type of operator corresponds to a plurality of operation modes, and each operation mode has a different operation precision and/or operation speed; M is an integer greater than or equal to N, and N is an integer greater than or equal to 1;

A configuration unit configured to configure the operating modes of the M operators in the algorithm model based on the configuration parameters corresponding to each type of operator or the configuration options corresponding to each operator node; wherein, the configuration Parameters or configuration options are used to indicate the operation mode of the corresponding operator.

In a possible implementation manner, the device further includes: a setting unit configured to set the configuration parameters corresponding to each type of operator in the operator interface corresponding to each type of operator in the N types of operators; The configuration unit is specifically used to: call the N types of operators through the operator interface corresponding to each type of operator in the N types of operators and obtain the corresponding configuration parameters; based on the N types of operators Configuration parameters corresponding to each type of operator in the algorithm model configure the corresponding operating mode of each type of operator in the N types of operators in the algorithm model.

In a possible implementation manner, the configuration parameter is one of a target configuration parameter, a default configuration parameter, and a tuning configuration parameter; wherein the target configuration parameter is used to instruct the corresponding operator to run in the target operation mode, The target operating mode is one of the multiple operating modes; the default configuration parameters are used to instruct the corresponding operator to operate in the operating mode with the highest operating accuracy or the fastest operating speed. OK; the tuning configuration parameters are used to instruct the corresponding operator to run in the optimal operation mode. The optimal operation mode is the operation result error determined based on the operator model and the operation mode with the highest operation accuracy. An operation mode that is within the preset threshold range and has the fastest operation speed.

In a possible implementation manner, the configuration parameters include a configuration path corresponding to the operating mode; the configuration unit is specifically configured to: based on the configuration parameters corresponding to each type of the N types of operators, obtain The configuration path of the corresponding operation mode of each type of operator in the N types of operators; read the corresponding configuration file based on the configuration path to configure the operation of the corresponding type of operator in the operation process of the algorithm model Way.

In a possible implementation manner, the determining unit is further configured to: receive input information and determine the M operator nodes and the configuration options corresponding to each operator node based on the input information; Wherein, the configuration options include one of target configuration parameters, default configuration parameters, and tuning configuration parameters; the configuration unit is specifically used to: use the operator interface corresponding to each type of operator in the N types of operators , respectively calling the N types of operators; based on the configuration options corresponding to each of the operator nodes, configure the operation of the operator corresponding to each of the M operator nodes in the algorithm model Way.

In a possible implementation manner, the device further includes: a compilation unit, configured to compile and run the algorithm model based on the operation mode of the M operators.

In the third aspect, embodiments of the present application provide an operator operation mode configuration system, including: a model compiler, used to determine M operators corresponding to M operator nodes based on the algorithm model, where one operator node corresponds to An operator. Each operator belongs to one of the N types of operators. Each type of operator corresponds to multiple operating modes. Each of the operating modes has different operating accuracy and/or operating speed. ; M is an integer greater than or equal to N, and N is an integer greater than or equal to 1; an operator compiler is used based on the configuration parameters corresponding to each type of operator or the configuration options corresponding to each operator node, Configure the operating modes of the M operators in the algorithm model; wherein the configuration parameters or the configuration options are used to indicate the operating modes of the corresponding operators.

In the fourth aspect, embodiments of the present application provide a computer storage medium for storing computer software instructions used for configuring a device for an operator operation mode provided in the second aspect, which includes instructions for executing the operations designed in the above aspect. program.

In a fifth aspect, embodiments of the present application provide a computer program. The computer program includes instructions. When the computer program is executed by a computer, the computer can execute the operations performed by the configuration device of the operator operation mode in the second aspect. process.

In a sixth aspect, the present application provides a chip system, which includes a processor and is used to support a terminal device to implement the functions involved in the above-mentioned first aspect, for example, in generating or processing the configuration method of the above-mentioned operator operation mode. the information involved. In a possible design, the chip system further includes a memory, and the memory is used to store necessary program instructions and data for the data sending device. The chip system may be composed of chips, or may include chips and other discrete devices.

Description of the drawings

In order to more clearly explain the technical solutions in the embodiments of the present application or the background technology, the drawings required to be used in the embodiments or the background technology of the present application will be described below.

Figure 1 is a partial structural schematic diagram of an algorithm model provided by an embodiment of the present application.

Figure 2 is a schematic diagram of the configuration architecture of an operator operation mode provided by the embodiment of the present application.

FIG. 3 is a schematic flowchart of a method for configuring an operator operation mode provided by an embodiment of the present application.

Figure 4 is a partial structural schematic diagram of an algorithm model provided by an embodiment of the present application.

Figure 5 is a schematic diagram of an algorithm model of several operating modes provided by embodiments of the present application.

Figure 6 is a configuration device for an operator operation mode provided by an embodiment of the present application.

Detailed ways

The embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.

The terms “first”, “second”, “third” and “fourth” in the description, claims and drawings of this application are used to distinguish different objects, rather than to describe a specific sequence. . Furthermore, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes Other steps or units inherent to such processes, methods, products or devices.

It should be understood that in this application, "at least one (item)" refers to one or more, and "plurality" refers to two or more. "and/ "Or" is used to describe the association of associated objects, indicating that there can be three relationships. For example, "A and/or B" can mean: only A exists, only B exists, and A and B exist simultaneously, where A , B can be singular or plural. The character "/" generally indicates that the related objects are an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including Any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c" , "b and c", or "a and b and c", where a, b, c can be single or multiple.

Reference herein to "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of this phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those skilled in the art understand, both explicitly and implicitly, that the embodiments described herein may be combined with other embodiments.

The terms "component", "module", "system", etc. used in this specification are used to refer to computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to, a process, a processor, an object, an executable file, a thread of execution, a program and/or a computer running on a processor. Through the illustrations, both applications running on the computing device and the computing device may be components. One or more components can reside in a process and/or thread of execution and a component can be localized on one computer and/or distributed between 2 or more computers. Additionally, these components can execute from various computer-readable media having various data structures stored thereon. A component may, for example, be based on a signal having one or more data packets (eg, data from two components interacting with another component, a local system, a distributed system, and/or a network, such as the Internet, which interacts with other systems via signals) Communicate through local and/or remote processes.

First of all, in order to facilitate understanding of the embodiments of the present application, the technical problems and application scenarios that need to be solved by the embodiments of the present application are analyzed in detail below.

In the existing technology, when it comes to running AI models on dedicated artificial intelligence (AI) chips, the two most important points are the accuracy and performance of the chip, that is, whether the calculation results are accurate and whether the calculation speed is fast. The computing power of an AI chip for certain aspects is fixed. For example, the Ascend 710AI processor's calculation speed for fp16 is twice as fast as that for fp32 in vector calculations, and even the computing power for some calculations is The difference is more than 2 times. fp16 refers to a data type that uses 2 bytes (16 bits) for encoding and storage; similarly, fp32 refers to using 4 bytes (32 bits).

To deal with this common computing power gap, the industry uses mixed precision methods. For example, the calculation types of some layers of the AI model are converted to fp16 calculations, and the calculations of some layers still maintain fp32 calculations. This kind of entire AI The model includes both fp16 and fp32 mixed precision calculation methods to improve the execution efficiency of the entire AI model without significant loss of accuracy.

Please refer to Figure 1. Figure 1 is a partial structural schematic diagram of an algorithm model provided by an embodiment of the present application. As shown in Figure 1, this part of the algorithm model includes three operators, namely MatMul, Add and Softmax. During the compilation process, if the accuracy mode of the operator is not specified, each operator node will use the high-precision operation mode by default (the high-precision operation mode can ensure that the accuracy of all value ranges is correct), such as fp32, etc. .

However, there are also great differences in the computing efficiency of the same chip on different types of operations. For example, the following are examples of different calculation methods for some operators:

Example: Div operator. The calculation logic of the Div operator is: find the division result of two tensors (Tensor), that is, find x÷y, x and y are a set of Tensors that satisfy the broadcast relationship.

The calculation efficiency of division on some AI chips is relatively low (i.e., the calculation performance is poor), but the calculation accuracy is high (i.e., the calculation result is accurate and the error is small), while the calculation efficiency of reciprocal calculation on the above-mentioned AI chips is relatively high, but the calculation The accuracy is slightly worse. However, the AI chip can use the Newton iteration method to improve calculation accuracy. The more Newton iterations are used, the accuracy will go up, but the performance will go down. Due to these characteristics, the Div operator can provide multiple implementation versions suitable for this AI chip. For example:

High-precision operation mode: Use the Div instruction of the AI chip for calculation, which features high precision but poor performance.

High-performance operation mode: Use the AI chip to find the reciprocal, and then do multiplication, that is, calculate x*(1/y), with slightly lower accuracy and higher performance.

Illustrative: Softmax operator. The calculation logic of the Softmax operator is:

Step 1: Find the max value on the reduce axis.

Step 2: Subtract the max value from the data on the reduce axis.

Step 3: Find the exp value of the data on the reduce axis.

Step 4: Find the cumulative sum of exp values of the data on the reduce axis.

Step 5: Find the exp value of the data on the reduce axis divided by its cumulative sum.

For the first step, the efficiency of calculating the max value of fp32 on the AI chip is much lower than that of fp16. Therefore, when calculating the Softmax result of an fp32 Tensor, if the second step is calculated using fp32, the efficiency will be not tall. If you convert fp32 to fp16 and then perform calculations, the performance will be improved a lot. However, converting to fp16 may result in poor calculation accuracy. For example: when the number entered When the data value size does not exceed the maximum value of fp16 (65504), this conversion has no effect on the final accuracy. When the input data value size exceeds the maximum value of fp16 (65504), this conversion will lead to the final result of the calculation. Incorrect. Due to these characteristics, the Softmax operator can provide 2 implementation versions:

High-precision operation mode: for max value calculation, no conversion from fp32 to fp16 is performed.

High-performance operation mode: Convert fp32 to fp16 for max value calculation.

For the above scenarios, existing AI frameworks do not provide effective methods to support this extension. Model developers need to provide customized operators in the form of custom operators, and then use the customized operators in the model to improve model running performance. For example: For the above Softmax operator, the model developer needs to provide the "My Softmax" operator that implements a high-performance operation mode in the AI framework, and then use the new operator in the model.

Disadvantage 1: Although operators with different operating modes are implemented, redefining an operator end-to-end requires many modifications. For example, a MySoftmax operator needs to be fully defined and implemented, and it also needs to be modified based on the AI model to use this new operator instead of the old one.

Disadvantage 2: The AI framework does not have the built-in capability of this new operator, that is, the new operator is not stored in the operator library. Operator developers need to identify which operators can have multiple operating modes. During development, new operators with multiple operating modes are provided. This method increases learning costs and requires AI model developers to be familiar with chip characteristics for optimization.

Therefore, the embodiments of the present application provide a configuration method, device and related system for operator operation modes. There is no need to define new operators, and multiple operation modes of operators can be implemented on the original operator framework. For example: the embodiment of this application provides a configuration method of operator operation mode, which can determine the operator corresponding to each of the M operator nodes of the operator model, and the N categories corresponding to the M operators. After the operator is added, the operation mode of the M operators in the algorithm model can be configured according to the configuration parameters corresponding to each type of operator or the configuration options corresponding to each operator node. Among them, the operators of each type of N types of operators can have a variety of operation modes with different operation accuracy and/or operation speed, which leads to the algorithm model also having multiple operation modes. In order to ensure that the operators in the operator model The operation accuracy and the operation performance of the operator can be improved at the same time. The embodiments of this application can be based on the operation characteristics of the operator model, based on the configuration parameters corresponding to each type of operator in the algorithm model or based on each operator node in the algorithm model. The corresponding configuration options configure the operation mode of the M operators in the algorithm model. The above-mentioned method of configuring the operation mode of the algorithm model through configuring parameters or configuration options can prevent the algorithm model in the existing technology from only performing operations according to a fixed processing method, making the optimization of the performance and accuracy of the algorithm model more flexible.

Based on the technical issues raised above and the corresponding application scenarios in this application, and to facilitate understanding of the embodiments of this application, the configuration system architecture of one of the operator operation modes based on the embodiments of this application is first described below. Please refer to Figure 2. Figure 2 is a schematic diagram of the configuration architecture of an operator operation mode provided by an embodiment of the present application. As shown in Figure 2, the configuration system architecture of this operator operation mode includes: a model compiler 101 and an operator compiler 102; it may also include: an operator library 103 and a tuner 104. in:

The model compiler 101 is used to determine M operators corresponding to M operator nodes based on the algorithm model, where one operator node corresponds to one operator, and each of the operators belongs to one of the N types of operators, Each type of operator corresponds to multiple operation modes, and the operation accuracy and/or operation speed between each operation mode are different; M is an integer greater than or equal to N, and N is an integer greater than or equal to 1.

For example, the model compiler 101 can be an AI model compiler (Ascend Tensor Complier, ATC). The operator development interface is provided to model developers, so that model developers can use ATC to compile their models, so that the models can be used in Ascend. Wait for the platform to run.

The operator compiler 102 is used to configure the operation mode of the M operators in the algorithm model based on the configuration parameters corresponding to each type of operator or the configuration options corresponding to each operator node; wherein, The configuration parameters or configuration options are used to indicate the operation mode of the corresponding operator.

For example, the operator compiler 102 can be an operator compiler included in a graph compilation engine and a fusion engine (Graph Engine&Fusion Engine, GE&FE). In which, after each operator node in the AI model undergoes various processing, it is finally The operator compiler needs to be used to compile operators that can be run on the AI chip. The operator compiler 102 provided in the embodiment of this application can read the corresponding operator precision mode configuration (that is, the corresponding operating mode of the operator) according to the configuration of each operator node in the model, and compile according to the configuration a program that can be used in the AI Implementation version running on the chip.

It should be noted that the AI model mentioned in the embodiments of this application and the following related embodiments is equivalent to an algorithm model.

The operator library 103 provides multiple types of operators and multiple operating modes of each type of operators for the operator compiler 102 to compile. That is, in this embodiment of the present application, the operator library 103 can provide multiple versions of an operator (that is, each type of operator corresponds to multiple operating modes) and implementation information (that is, multiple operating modes). Configuration information corresponding to each mode), a default high-performance mode configuration file and a default high-precision mode configuration file corresponding to all types of operators can also be provided to achieve high precision or high performance of the operators. of operation. For example: the operator library provides the default high-precision mode configuration file high_performance.ini including: SoftmaxV2=high_performance, and RealDiv=high_performance, that is, the high-precision mode configuration file includes the configuration information of the high-precision operation mode of the SoftmaxV2 operator and the RealDiv operator. Another example: the operator library provides the default high-precision mode configuration file high_precision.ini including: SoftmaxV2=high_performance, and RealDiv=high_performance, that is, the high-performance mode configuration file includes the high-performance operation mode of the SoftmaxV2 operator and the RealDiv operator. Configuration information.

The tuner 104 can be called Auto Tune. The multi-version implementation information provided in the operator library 103 can be read, and the precision mode configuration is generated by automatically tuning the operator based on model-based computing features or chip-based computing power features. For example: Compared with division operations, the calculation efficiency of reciprocal calculation on related AI chips is relatively high.

For example, the tuner 104 can read the configuration file in the operator library 103 and obtain multiple operating modes corresponding to various operators. That is, obtain which operators can adjust the operating mode (precision mode configuration), and which precision modes can be configured. Among them, the configuration file can include multiple operating modes. For example, the order from left to right is: the higher the performance is to the left, the higher the accuracy is to the right.

In an embodiment of the present application, the tuner 104 can run all the various operation modes of an operator in parallel, and then compare the output results of the various operation modes with the output results of the high-precision operation mode for accuracy, and select the operation mode with the highest performance as the tuning operation mode of the operator from the operation modes whose accuracy comparison results are less than a preset threshold (the user can also specify a threshold, the default threshold is double one thousandth, that is, the absolute value of the difference between the values of each element in the two result Tensors is compared point by point and does not exceed one thousandth (abs(x-y)/min(abs(x), abs(y))<0.001), and the number of elements with an absolute value difference exceeding one thousandth accounts for no more than one thousandth of the total number of elements in the total Tensor). For example: If the high performance and high precision comparison of an operator meet the threshold requirements, it will be adjusted to a high-performance implementation to improve the final AI model operation performance.

Therefore, the embodiment of the present application provides an operator operation mode configuration system, which does not need to define new operators and can implement multiple operation modes of operators on the original operator framework. For example: after the model compiler determines the operator corresponding to each of the M operator nodes of the operator model and the N types of operators corresponding to the M operators, the operator compiler can determine the The configuration parameters corresponding to the above operators or the configuration options corresponding to each of the operator nodes configure the operation mode of the M operators in the algorithm model. Among them, each type of N-type operators can have a variety of operating modes with different operating accuracy and/or operating speed, which leads to the algorithm model also having multiple operating modes. In order to ensure that the operators in the operator model The operation accuracy is improved while the operation performance of the operator is improved. Embodiments of this application can configure the algorithm model based on the operation characteristics of the operator model, based on the configuration parameters corresponding to each type of operator in the algorithm model, or based on the configuration options corresponding to each operator node in the algorithm model. How M operators operate. The above-mentioned method of configuring the operation mode of the algorithm model through configuring parameters or configuration options can prevent the algorithm model in the existing technology from only performing operations according to a fixed processing method, making the optimization of the performance and accuracy of the algorithm model more flexible.

It should be noted that each functional module in the configuration architecture of the operator operation mode described in the embodiment of the present application is only an exemplary description in the embodiment of the present application, and the embodiment of the present application does not limit this.

It should also be noted that for the functions of each functional module in the configuration architecture of the operator operation mode described in the embodiments of this application, please refer to the relevant descriptions of steps S201 to S204 in the method embodiments described below, here No further details for now.

Based on the configuration system architecture of the operator operation mode provided in the above embodiments, combined with the configuration method of the operator operation mode provided in this application, the technical problems raised in this application are specifically analyzed and solved.

Referring to Figure 3, Figure 3 is a schematic flow chart of a method for configuring an operator operating mode provided by an embodiment of the present application. This method can be applied to the configuration system architecture of the operator operating mode described in Figure 1, wherein The method includes step S201 to step S204 of the method flow shown in Figure 3. Among them, the model compiler 101 in the configuration system architecture of the operator operation mode shown in Figure 1 can be used to support and execute the method flow steps S201 to S202 and S204 shown in Figure 3. The operator compiler 102 may be used to support and execute step S203 of the method flow shown in FIG. 3 . in,

Step S201: Determine M operators corresponding to M operator nodes based on the algorithm model.

Specifically, M operators corresponding to M operator nodes are determined based on the algorithm model, wherein one operator node corresponds to one operator, each of the operators belongs to one of N types of operators, each type of operator corresponds to multiple operating modes, and the operation accuracy and/or operation speed of each operating mode is different; M is an integer greater than or equal to N, and N is an integer greater than or equal to 1.

Please refer to FIG. 4 , which is a partial structural diagram of an algorithm model provided by an embodiment of the present application. As shown in Figure 4, this part of the algorithm model includes three operator nodes, namely, a first operator node 301, a second operator node 302, and a third operator node 303. The first operator node 301 corresponds to the MatMul operator, the second operator node 302 corresponds to the RealDiv operator, and the third operator node 303 corresponds to the Softmax operator. Among them, the operators corresponding to the first operator node 301, the second operator node 302, and the third operator node 303 respectively belong to three different types of operators. Each type of operator can correspond to multiple operation modes, such as: high-precision operation mode, high-performance operation mode, half-precision operation mode, etc. Each of the above operation modes has different operation accuracy and/or operation speed. For example: RealDiv operator can provide high-precision operation mode: Use the Div instruction with fp32 precision for calculations. Half-precision operation mode: Use the Div instruction of fp16 precision for calculation. High-performance operation mode: use fp16 precision to find the reciprocal, and then do multiplication, that is, calculate x*(1/y). The embodiments of this application do not specifically limit the operating modes of various operators.

Step S202: Set the configuration parameters corresponding to each type of operator in the operator interface corresponding to each type of operator among the N types of operators.

Specifically, the configuration parameters corresponding to each type of operator are set in the operator interface corresponding to each type of operator in the N types of operators. Each type of operator corresponds to multiple operating modes. Setting the configuration parameters corresponding to each type of operator in the operator interface corresponding to each type of N types of operators can realize the operating mode without changing the operator structure. configuration.

Exemplary: The operator interface of the Softmax operator in the prior art can be def softmax_v2 (input_x, output_y, axis=-1, kernel_name="softmax_v2"). In the embodiment of this application, the operator interface of the Softmax operator can be ：def softmax_v2(input_x, output_y, axis＝-1, kernel_name="softmax_v2", impl_mode="high_performance"). Among them, compared with the existing technology, this application adds a new configuration parameter impl_mode="high_performance" in the operator interface to indicate that the configured operation mode when calling the Softmax operator is the high-performance operation mode.

Exemplary: The configuration parameters corresponding to each type of operator are set in the operator interface corresponding to each type of operator mentioned in the embodiment of the present application. It is also possible to set the configuration parameters based on the global parameter global_build_context in the compiled context information before calling the operator interface for compilation, so that when calling the operator interface, the operation mode of the operator can be directly configured based on the configuration parameters corresponding to the global parameter. For example: the pre-set global parameters when calling the Softmax operator are: global_build_context.set_impl_mode("high_performance"), and the operator compiler calls the operator interface of the Softmax operator according to the compiled context information as def softmax_v2(input_x, output_y, axis=-1, kernel_name="softmax_v2"): global_build_context.set_impl_mode("high_performance"), which is used to indicate that the operation mode configured when calling the Softmax operator is a high-performance operation mode.

Optionally, the configuration parameters are one of target configuration parameters, default configuration parameters, and tuning configuration parameters; wherein, the target configuration parameters are used to instruct the corresponding operator to run in the target operation mode, and the target operation mode It is one of the multiple operating modes; the default configuration parameters are used to instruct the corresponding operator to run in the operating mode with the highest operating accuracy or the fastest operating speed; the tuning configuration parameters are used to indicate the corresponding The operator operates in an optimal operation mode, which is one in which the operation result error determined based on the operator model and the operation mode with the highest operation accuracy is within a preset threshold range and has the fastest operation speed. mode of operation.

It can be understood that the configuration parameters in the embodiments of the present application can be divided into three categories. For example, the configuration parameters can be any one of target configuration parameters, default configuration parameters, and tuning configuration parameters. Among them, the target configuration parameters can be used to indicate an operating mode set by the user; that is, the user can directly set the configuration parameters to the target configuration parameters of the required target operating mode. For example: the target configuration parameters can indicate an ini file. Path, indicating that the configuration information in this ini file is used to select the corresponding precision implementation version (that is, the running mode) for the corresponding operator. Default configuration parameters can be used to instruct this type of operator to run in the highest computing accuracy or fastest computing speed mode; for example: the default configuration parameters can indicate the default high-precision and high-performance configuration built into the operator library, such as The corresponding configuration parameters in the operator interface of the above Softmax operator. Tuning configuration parameters can be used to indicate running in the optimal operating mode under the operating characteristics of the current operator model. Another example: the tuning configuration parameters represent the running mode after automatic tuning using the tuner (auto_tune module). This tuning configuration parameter can be used to instruct the tuner to select an operating mode from multiple operating modes that has an error within a preset threshold and has the highest performance. Please refer to Figure 5. Figure 5 is a schematic diagram of an algorithm model of several operating modes provided by embodiments of the present application. As shown in Figure 5, this application runs the high-precision operation mode and the high-performance operation mode respectively based on the algorithm model structure shown in Figure 4 above. The results of the two operation modes can be compared to select the optimal operation mode.

Among them, the preset threshold range can represent a specified threshold, or it can be a default threshold. The default threshold is one in two thousand, that is, two results (the operation result of the optimal operation mode and the operation result of the high-precision operation mode) Tensor The values in are compared point by point. The absolute value of the difference between each element value does not exceed one thousandth (abs(x-y)/min(abs(x), abs(y))<0.001), and the absolute value difference exceeds one thousandth. The number of elements of one accounts for no more than one thousandth of the total number of elements in the total Tensor.

Optionally, input information is received and the M operator nodes and the configuration options corresponding to each operator node are determined based on the input information. Specifically, input information is received and the M operator nodes and the configuration options corresponding to each operator node are determined based on the input information; wherein the configuration options include target configuration parameters, default configuration parameters, One of the tuning configuration parameters. The input information may be user input or selection operation, and in response to the operation, the configuration options corresponding to each operator node may be set. Correspondingly, the configuration option also includes one of target configuration parameters, default configuration parameters, and tuning configuration parameters. The above three configuration parameters can be referred to the relevant descriptions of the above embodiments, and will not be described again in the embodiments of this application.

It can be understood that setting configuration parameters at the operator interface means that the operator model will run in the mode indicated by the configuration parameters when calling and compiling this type of operator; setting configuration options at the operator node means Refers to the operating mode indicated by this configuration option when compiling the operator corresponding to the operator node of the operator model.

Step S203: Configure the operating modes of the M operators in the algorithm model based on the configuration parameters corresponding to each type of operator or the configuration options corresponding to each operator node.

Specifically, based on the configuration parameters corresponding to each type of operator, the operation mode of the M operators in the algorithm model is configured. Or, configure the operation mode of M operators in the algorithm model based on the configuration options corresponding to each operator node. Correspondingly, after the operator compiler reads/obtains the operation modes corresponding to the M operators, it reads the corresponding configuration files in the operator library according to the operation modes to compile the operators.

In one possible implementation method, when the configuration options corresponding to the operator node and the configuration parameters corresponding to each type of operator are set at the same time, the corresponding operating mode in the configuration options corresponding to the operator node is used as the corresponding operation mode of the current operator node. How the operator operates.

In another possible implementation, when the configuration options corresponding to the operator node and the configuration parameters corresponding to each type of operator are set at the same time, the corresponding operating mode in the configuration parameters corresponding to each type of operator is used as the current operator. The node corresponds to the operation mode of the operator.

In another possible implementation method, when the configuration options corresponding to the operator node and the configuration parameters corresponding to each type of operator are not set, the corresponding high-precision operation mode in the configuration parameters corresponding to each type of operator is used as the The current operator node corresponds to the operating mode of the operator.

In a possible implementation manner, configuring the operating modes of the M operators in the algorithm model based on the configuration parameters corresponding to each type of operators includes: using the N types of operators The operator interface corresponding to each type of operator calls the N types of operators respectively and obtains the corresponding configuration parameters; configure the algorithm based on the configuration parameters corresponding to each type of the N types of operators. The corresponding operating mode of each type of operator in the N types of operators described in the model.

Setting the configuration parameters corresponding to each type of operator in the operator implementation interface enables operator-level precision mode setting. When calling each type of operator through the operator interface corresponding to this type of operator, operators with different operating modes can be compiled according to different configuration parameters in the operator implementation interface, and finally the accuracy and/or performance of the operator model can be adjusted. This makes the optimization of algorithm model performance and accuracy more flexible.

In a possible implementation manner, the configuration parameters include a configuration path corresponding to the operating mode; and based on the configuration parameters corresponding to each type of the N types of operators, the configuration parameters in the algorithm model are The operation mode corresponding to each type of operator in the N types of operators includes: based on the configuration parameters corresponding to each type of operators in the N types of operators, obtaining the operation mode of each type of operators in the N types of operators. The configuration path of the corresponding operating mode of the operator; the corresponding configuration file is read based on the configuration path to configure the operating mode of the corresponding type of operator during the operation of the algorithm model. For example, the configuration parameters may include corresponding running mode configuration paths. The configuration path is used to indicate the configuration file corresponding to the operation mode. The configuration file includes relevant configuration information, so that after reading the configuration file, the operation mode of the corresponding type of operator can be configured during the operation of the algorithm model. For example: when the configuration parameters of the first type of operator are target configuration parameters, they can include the configuration path of the target operating mode, and read the configuration file of the target operating mode based on the configuration path of the target operating mode; and then configure the first operating mode based on the configuration file. Class operators run in target run mode.

It should be noted that the target operation mode may be a high-precision operation mode, which is not specifically limited in this application.

Optionally, call the N types of operators respectively through the operator interface corresponding to each type of N types of operators; configure the algorithm model based on the configuration options corresponding to each of the operator nodes. The operation mode of the operator corresponding to each of the M operator nodes. Among them, based on each operator node, the operator interface of the corresponding type of operator is called, and based on the configuration option corresponding to each operator node, the running mode of the called operator is configured.

In addition to setting the configuration parameters corresponding to each type of operator in the operator implementation interface, you can also set the configuration options corresponding to each operator node based on the user's input information to implement the precision mode setting at the operator node level, that is, operators of the same type in different operator nodes can be configured to run in different ways. The configuration options and configuration parameters can also be set by selecting one of the three configuration parameters. When compiling the model, when each operator node calls the operator of this type through the operator interface of the corresponding type, the operation mode of the operator of the operator node can be configured according to the configuration options corresponding to the operator node, and finally the precision and/or performance of the operator model can be adjusted, making the optimization of the performance and precision of the algorithm model more flexible.

Step S204: Compile and run the algorithm model based on the operation modes of the M operators.

Specifically, based on the operation mode of M operators, the algorithm model is compiled and run. After obtaining the operation mode of each operator, the operators that can be run on the chip can be compiled sequentially or in parallel to obtain and run the algorithm model.

In the embodiment provided by this application, after determining the operator corresponding to each of the M operator nodes of the operator model, and the N types of operators corresponding to the M operators, the operator can be determined according to the description of each type. The configuration parameters corresponding to the operator or the configuration options corresponding to each operator node configure the operation mode of the M operators in the algorithm model. Among them, each type of N-type operators can have a variety of operating modes with different operating accuracy and/or operating speed, which leads to the algorithm model also having multiple operating modes. In order to ensure that the operators in the operator model The operation accuracy of the operator is improved while the operation performance of the operator is improved. The embodiments of this application can be based on the operation characteristics of the operator model, based on the configuration parameters corresponding to each type of operator in the algorithm model or based on each operator node in the algorithm model. The corresponding configuration options configure the operation mode of the M operators in the algorithm model. The above method of configuring the operation mode of the algorithm model through configuring parameters or configuration options can avoid that the algorithm model in the prior art can only operate according to a fixed processing method, so that the performance and accuracy of the algorithm model in the embodiment of the present application are improved. Optimization is more flexible.

The methods of the embodiments of the present application are described in detail above, and the relevant devices of the embodiments of the present application are provided below.

Please refer to Figure 6. Figure 6 is an operator operation mode configuration device provided by an embodiment of the present application. The operator operation mode configuration device 60 is applied to the configuration system structure of the operator operation mode and is suitable for the above operator operation. configuration method. The operator operation mode configuration device 60 may include a determination unit 401 and a configuration unit 402, and may also include a setting unit 403 and a compilation unit 404. Among them, the detailed description of each unit is as follows.

The determination unit 401 is used to determine M operators corresponding to M operator nodes based on the algorithm model, where one operator node corresponds to one operator, and each operator belongs to one of the N types of operators. The operators described in this class respectively correspond to multiple operating modes, and the operation accuracy and/or operation speed between each of the operation modes are different; M is an integer greater than or equal to N, and N is an integer greater than or equal to 1;

The configuration unit 402 is configured to configure the operating modes of the M operators in the algorithm model based on the configuration parameters corresponding to each type of operator or the configuration options corresponding to each operator node; wherein, Configuration parameters or configuration options are used to indicate the operation mode of the corresponding operator.

In a possible implementation manner, the device further includes: a setting unit 403, configured to set the configuration parameters corresponding to each type of operator in the operator interface corresponding to each type of operator in the N types of operators. ; The configuration unit 402 is specifically configured to: call the N types of operators through the operator interface corresponding to each type of operator in the N types of operators and obtain the corresponding configuration parameters; based on the N The configuration parameters corresponding to each type of operator in the operator type configure the operation mode corresponding to each type of operator in the N types of operators in the algorithm model.

In a possible implementation manner, the configuration parameter is one of a target configuration parameter, a default configuration parameter, and a tuning configuration parameter; wherein the target configuration parameter is used to instruct the corresponding operator to run in the target operation mode, The target operating mode is one of the multiple operating modes; the default configuration parameters are used to instruct the corresponding operator to run in the operating mode with the highest operating accuracy or the fastest operating speed; the tuning configuration The parameter is used to indicate that the corresponding operator runs in the optimal operation mode. The optimal operation mode is that the operation result error determined based on the operator model and the operation mode with the highest operation accuracy is within the preset threshold range and the operation result The fastest way to run.

In a possible implementation manner, the configuration parameters include a configuration path corresponding to the operating mode; the configuration unit 402 is specifically configured to: based on the configuration parameters corresponding to each type of the N types of operators, Obtain the configuration path corresponding to the operation mode of each type of operator in the N types of operators; read the corresponding configuration file based on the configuration path to configure the operation of the corresponding type of operator in the algorithm model operation process. Operation mode.

In one possible implementation, the determination unit 401 is also used to: receive input information and determine the M operator nodes and the configuration options corresponding to each of the operator nodes based on the input information; wherein the configuration options include one of target configuration parameters, default configuration parameters, and tuning configuration parameters; the configuration unit 402 is specifically used to: respectively call the N types of operators through the operator interfaces corresponding to each type of operator in the N types of operators; and configure the operation mode of the operator corresponding to each of the M operator nodes in the algorithm model based on the configuration options corresponding to each of the operator nodes.

In a possible implementation manner, the device further includes: a compiling unit 404, configured to compile and run the algorithm model based on the operating modes of the M operators.

It should be noted that the functions of each functional unit in the operator operation mode configuration device 60 described in the embodiment of the present application can be referred to step S201 in the embodiment of the operator operation mode configuration method described in Figure 3. The relevant description of S204 will not be repeated here.

Embodiments of the present application provide a computer storage medium for storing computer software instructions used for configuring a device for an operator operation mode provided in the above-mentioned related embodiments, which includes a program designed for executing the above aspect.

An embodiment of the present application provides a computer program. The computer program includes instructions. When the computer program is executed by a computer, the computer can execute the process executed by the configuration device of the operator operation mode in the above related embodiments.

The present application provides a chip system. The chip system includes a processor and is used to support the terminal device to implement the functions involved in the above-mentioned related embodiments, for example, generating or processing the information involved in the configuration method of the above-mentioned operator operation mode. . In a possible design, the chip system further includes a memory, and the memory is used to store necessary program instructions and data for the data sending device. The chip system may be composed of chips, or may include chips and other discrete devices.

In the above embodiments, each embodiment is described with its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

It should be noted that for the sake of simple description, the foregoing method embodiments are expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence. Because according to this application, certain steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are preferred implementations. The actions and modules involved in the embodiment are not necessarily necessary for this application.

In the several embodiments provided in this application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the above units is only a logical function division. In actual implementation, there may be other divisions. For example, multiple units or components may be combined or integrated. to another system, or some features can be ignored, or not implemented. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, and the indirect coupling or communication connection of the devices or units may be in electrical or other forms.

The units described above as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or software functional units.

If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to cause a computer device (which can be a personal computer, a server, a network device, etc., specifically a processor in a computer device) to execute all or part of the steps of the above methods in various embodiments of the present application. Among them, the aforementioned storage media may include: U disk, mobile hard disk, magnetic disk, optical disk, read-only memory (Read-Only Memory, abbreviation: ROM) or random access memory (Random Access Memory, abbreviation: RAM), etc. A medium on which program code can be stored.

As mentioned above, the above embodiments are only used to illustrate the technical solution of the present application, but not to limit it. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still make the foregoing technical solutions. The technical solutions described in each embodiment may be modified, or some of the technical features thereof may be equivalently replaced; however, these modifications or substitutions shall not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of each embodiment of the present application.

Claims

A method for configuring operator operation mode, which is characterized by including:

M operators corresponding to M operator nodes are determined based on the algorithm model. Among them, one operator node corresponds to one operator. Each of the operators belongs to one of the N types of operators. The operators of each type are respectively Corresponding to multiple operation modes, the operation accuracy and/or operation speed between each operation mode are different; M is an integer greater than or equal to N, and N is an integer greater than or equal to 1;

Based on the configuration parameters corresponding to each type of the operator or the configuration options corresponding to each of the operator nodes, the operation modes of the M operators in the algorithm model are configured; wherein the configuration parameters or the configuration options are used to indicate the operation mode of the corresponding operator.
The method according to claim 1, characterized in that, the method further includes:

Set the configuration parameters corresponding to each type of operator in the operator interface corresponding to each type of operator in the N types of operators;

The operation mode of the M operators in the algorithm model is respectively configured based on the configuration parameters corresponding to each type of operator, including:

Through the operator interface corresponding to each type of operator in the N types of operators, call the N types of operators respectively and obtain the corresponding configuration parameters;

Based on the configuration parameters corresponding to each type of the N types of operators, configure the corresponding operating mode of each type of the N types of operators in the algorithm model.
The method according to claim 1 or 2, characterized in that the configuration parameters are one of target configuration parameters, default configuration parameters, and tuning configuration parameters;

The target configuration parameter is used to instruct the corresponding operator to run in a target operation mode, and the target operation mode is one of the multiple operation modes;

The default configuration parameters are used to instruct the corresponding operator to operate in the operation mode of the highest computing accuracy or the operation mode of the fastest computing speed;

The tuning configuration parameters are used to instruct the corresponding operator to run in the optimal operation mode. The optimal operation mode is the preset error between the operation result determined based on the operator model and the operation mode with the highest operation accuracy. An operation mode that is within the threshold range and has the fastest operation speed.
The method according to any one of claims 1-3, characterized in that the configuration parameters include a configuration path corresponding to the operating mode;

Configuring the operation mode corresponding to each type of the N types of operators in the algorithm model based on the configuration parameters corresponding to each type of the N types of operators includes:

Based on the configuration parameters corresponding to each type of the N types of operators, obtain a configuration path corresponding to the operation mode of each type of the N types of operators;

The corresponding configuration file is read based on the configuration path to configure the operation mode of the corresponding type of operator during the operation of the algorithm model.
The method according to claim 1, characterized in that determining M operators corresponding to M operator nodes based on an algorithm model includes:

Receive input information and determine the M operator nodes and the configuration options corresponding to each of the operator nodes based on the input information; wherein the configuration options include one of target configuration parameters, default configuration parameters, and tuning configuration parameters;

Configuring the operation mode of the M operators in the algorithm model based on the configuration options corresponding to each operator node includes:

Call the N types of operators respectively through the operator interface corresponding to each type of operator in the N types of operators;

Based on the configuration options corresponding to each operator node, configure the operating mode of the operator corresponding to each of the M operator nodes in the algorithm model.
The method according to any one of claims 1-5, characterized in that the method further includes:

Based on the operation mode of the M operators, the algorithm model is compiled and run.
A configuration device for operator operation mode, which is characterized by including:

Determination unit, used to determine M operators corresponding to M operator nodes based on the algorithm model, where one operator node corresponds to one operator, and each operator belongs to one of the N types of operators. Each type The operators respectively correspond to multiple operation modes, and the operation accuracy and/or operation speed between each operation mode are different; M is an integer greater than or equal to N, and N is an integer greater than or equal to 1;

A configuration unit configured to configure the operator based on the configuration parameters corresponding to each type of operator or the configuration options corresponding to each operator node. The operation mode of the M operators in the method model; wherein the configuration parameters or the configuration options are used to indicate the operation mode of the corresponding operator.
The device according to claim 7, characterized in that, the device further includes:

A setting unit configured to set the configuration parameters corresponding to each type of operator in the operator interface corresponding to each type of operator in the N types of operators;

The configuration unit is specifically used for:

Through the operator interface corresponding to each type of operator in the N types of operators, call the N types of operators respectively and obtain the corresponding configuration parameters;

Based on the configuration parameters corresponding to each type of the N types of operators, configure the corresponding operating mode of each type of the N types of operators in the algorithm model.
The device according to claim 7 or 8, characterized in that the configuration parameters are one of target configuration parameters, default configuration parameters, and tuning configuration parameters;

Wherein, the target configuration parameter is used to instruct the corresponding operator to run in a target operation mode, and the target operation mode is one of the multiple operation modes;

The default configuration parameters are used to instruct the corresponding operator to operate in the operation mode of the highest computing accuracy or the operation mode of the fastest computing speed;

The tuning configuration parameters are used to instruct the corresponding operator to run in the optimal operation mode. The optimal operation mode is the preset error between the operation result determined based on the operator model and the operation mode with the highest operation accuracy. An operation mode that is within the threshold range and has the fastest operation speed.
The device according to any one of claims 7-9, wherein the configuration parameters include a configuration path corresponding to the operating mode;

The configuration unit is specifically used for:

Based on the configuration parameters corresponding to each type of operator in the N types of operators, obtain the configuration path corresponding to the operating mode of each type of operator in the N types of operators;

The corresponding configuration file is read based on the configuration path to configure the operation mode of the corresponding type of operator during the operation of the algorithm model.
The device according to claim 7, characterized in that the determining unit is also used to:

Receive input information and determine the M operator nodes and the configuration options corresponding to each of the operator nodes based on the input information; wherein the configuration options include one of target configuration parameters, default configuration parameters, and tuning configuration parameters;

The configuration unit is specifically used for:

Call the N types of operators respectively through the operator interface corresponding to each type of operator in the N types of operators;

Based on the configuration options corresponding to each operator node, configure the operating mode of the operator corresponding to each of the M operator nodes in the algorithm model.
The device according to any one of claims 7-11, characterized in that the device further includes:

The compilation unit is used to compile and run the algorithm model based on the operation mode of the M operators.
A configuration system for operator operation mode, which is characterized by including:

The model compiler is used to determine M operators corresponding to M operator nodes based on the algorithm model, where one operator node corresponds to one operator, and each of the operators belongs to one of the N types of operators. The operators described in this class respectively correspond to multiple operating modes, and the operation accuracy and/or operation speed between each of the operation modes are different; M is an integer greater than or equal to N, and N is an integer greater than or equal to 1;

An operator compiler is used to configure the operation mode of the M operators in the algorithm model based on the configuration parameters corresponding to each type of operator or the configuration options corresponding to each operator node; wherein, The above configuration parameters or the above configuration options are used to indicate the operation mode of the corresponding operator.
A computer program product, characterized in that the computer program product includes a computer program. When the computer program is executed by a computer or a processor, the computer or the processor causes the computer or the processor to execute any of claims 1-6. The method described in one item.
A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, and when the computer program is executed by a computer or processor, the method described in any one of claims 1-6 is implemented.