WO2022142662A1 - Operator mapping method and apparatus for computational graph - Google Patents

Abstract

The present application relates to an operator mapping method and apparatus for a computational graph, the method comprising: on the basis of a first footprint matrix and a second footprint matrix, repeating the following process until stop conditions are reached: calculating a first matrix according to the first footprint matrix and the second footprint matrix, adjusting the order of rows and columns in the first footprint matrix according to the first matrix, and recalculating the first matrix according to the adjusted first footprint matrix and the second footprint matrix; and according to the first footprint matrix and the second footprint matrix when the stop conditions are reached, obtaining operators in a second computational graph that respectively correspond to a plurality of operators in a first computational graph. The operator mapping method and apparatus of the embodiments provided in the present application can support operator mapping between computational graphs of similar structures, and can be applied to a scenario in which a deep learning framework performs model migration.

Description

Operator mapping method and device for computational graph

This application claims the priority of the Chinese patent application with the application number 202011588279.1 and the invention titled "Operator Mapping Method and Apparatus for Computational Graphs" filed with the China Patent Office on December 29, 2020, the entire contents of which are incorporated herein by reference Applying.

technical field

The present application relates to the technical field of artificial intelligence, and in particular, to a method and device for operator mapping of a computational graph.

Background technique

Computational graph, as an intermediate representation between the front-end and back-end of a deep learning framework, can bring good interactivity. Computational graph is a way of expressing data functions through graph theory language. In graph theory, nodes are connected by edges, nodes represent things, and the edges connecting two nodes represent the relationship between two things, while nodes in computational graphs represent nerves. The input data or operator in the network, the edge connecting two nodes represents the input-output relationship between the two points, etc.

The deep learning framework can convert the script of the model written according to the deep learning framework into a computational graph, and the operators in the model can be converted into a corresponding node in the computational graph. When the scripts of the same model are processed under different deep learning frameworks, the generated computational graphs are often different. When migrating models between different deep learning frameworks, for example, when migrating a model from TensorFlow (TF for short) to MindSpore (MS for short), the structure and computation graph of the models corresponding to different deep learning frameworks before and after the migration The operators may be different. Therefore, it may be necessary to compare the structure of the calculation graph corresponding to the model before and after the migration, and whether the operators in the calculation graph are consistent.

The structure of the computational graph is complex and there are many operators. For example, the number of operators in the computational graph of ResNet50 (Residual Network) in the deep learning framework TensorFlow is 2000, and the computational graph in the deep learning framework MindSpore The number of subs is 5000. It is very necessary to design an automatic calculation graph comparison tool to realize the comparison of two calculation graphs and the operator relationship mapping.

SUMMARY OF THE INVENTION

In view of this, a method and device for operator mapping of computational graphs are proposed, which can support operator mapping between computational graphs with similar structures, and can be applied to scenarios where a deep learning framework performs model migration.

In a first aspect, embodiments of the present application provide an operator mapping method for a computation graph, for performing operator mapping on operators in a first computation graph and a second computation graph, the first computation graph and all The second calculation graph is the calculation graph of the same neural network model, and the operator mapping method includes:

Processes the first footprint matrix and the second footprint matrix, and repeats the following process until a stop condition is reached: calculating a first matrix according to the first footprint matrix and the second footprint matrix, and adjusting all the parameters according to the first matrix. the order of the rows and columns in the first footprint matrix, and recalculate the first matrix according to the adjusted first footprint matrix and the second footprint matrix; wherein the first footprint matrix indicates the first calculation the topological relationship between operators of the graph, the second footprint matrix indicates the topological relationship between the operators of the second computation graph, the first matrix indicates the first computation graph and the second computation graph similarity of the topological relationships of the graphs; the processor obtains, according to the first footprint matrix and the second footprint matrix when the stopping condition is reached, that multiple operators in the first computation graph are respectively in the second computation graph corresponding operator.

The operator mapping method of the embodiments provided by the present application constructs footprint matrices of two computation graphs, where the footprint matrix is a matrix representing the topological relationship between operators in the computation graph. The first matrix may be iteratively calculated according to footprint matrices corresponding to the two calculation graphs, and the first matrix may indicate the similarity of the topological relationship of the two calculation graphs. After the iteration is stopped, the preferred one or more mapping relationships can be determined according to the footprint matrix. Therefore, the operator mapping method provided by the present application can support operator mapping between calculation graphs with similar structures, and can be applied to a scenario where a deep learning framework performs model migration.

According to a first possible implementation manner of the first aspect, the processor calculates the first matrix according to the first footprint matrix and the second footprint matrix, including: the processor calculates the first matrix according to a plurality of first eigenvectors and a plurality of second eigenvectors, The first matrix is obtained, wherein each of the first eigenvectors corresponds to an operator in the first calculation graph, and the operator corresponding to each of the first eigenvectors and the first eigenvector is in the The corresponding row vectors in the first footprint matrix are related to column vectors, each of the second eigenvectors corresponds to an operator in the second calculation graph, and each of the second eigenvectors is related to the second eigenvectors The row vector corresponding to the corresponding operator in the second footprint matrix is related to the column vector, and the first eigenvector and the second eigenvector are used to calculate the operator corresponding to the first eigenvector and the The similarity of the topological relationship between the operators corresponding to the second eigenvector.

According to the first possible implementation manner of the first aspect, in the second possible implementation manner, the value of the element of the first matrix represents the operator corresponding to the row where the element is located and the operator corresponding to the column where the element is located similarity of the topological relationship of the elements, wherein the operator corresponding to the row where the element is located and the operator corresponding to the column where the element is located are respectively the operator in the first calculation graph and the operator in the second calculation graph operator.

The eigenvectors of the operator are constructed by using the corresponding row and column vectors in the footprint matrix corresponding to the operator in the calculation graph, and the topological relationship characteristics of the operator in the calculation graph are extracted. The similarity of the topological relationship of the operator is calculated according to the eigenvector of the operator, and the similarity of the topological relationship of the operator of the first calculation graph and the operator of the second calculation graph forms a similarity matrix, and the similarity matrix is used to measure the two calculations. The similarity of the topological relationships of the graphs.

According to the first aspect, or the first or second possible implementation manner of the first aspect, in a third possible implementation manner, the first footprint matrix, the adjusted first footprint matrix and the third The operator corresponding to the row at the same position and the operator corresponding to the column at the same position of the two footprint matrix are of the same type, and the dimensions of the tensors output by the operators are the same. That is, the first footprint matrix and the second footprint matrix have the same type. The operator corresponding to the row at the same position and the operator corresponding to the column at the same position of the footprint matrix are of the same type, and the dimension of the tensor output by the operator is the same, and the adjusted first footprint matrix and the second footprint matrix are the same. The operator corresponding to the row of the position, the operator corresponding to the column of the same position are of the same type, and the dimension of the tensor output by the operator is the same.

When constructing the first footprint matrix and the second footprint matrix, and adjusting the first footprint matrix, ensure the type of the operator corresponding to the row at the same position and the type of the operator corresponding to the column at the same position of the first footprint matrix and the second footprint matrix The same, the dimensions of the tensors output by the operators are the same, that is, the operators corresponding to the rows in the same position of the first footprint matrix and the second footprint matrix and the operators corresponding to the columns in the same position are functionally the same. Combine the similarity matrix to measure the similarity of the topological relationship between operators. According to the operator mapping method provided by the present application, the mapping relationship between two operators can be reflected in terms of functions and topological relationships, thereby ensuring the correctness of the mapping result.

According to the first aspect, or the first or second possible implementation manner of the first aspect, in a fourth possible implementation manner, the operators in the first computation graph and the second computation graph are divided into Belonging to multiple first partitions, the same first partition has the same operator type, and the dimensions of the tensors output by the operators are the same, and each first partition may include at least part of the first computation graph and/or the second computation graph Operators; some operators in the first calculation graph and the second calculation graph belong to a second partition, and the types of operators in the second partition are different and/or the dimensions of the tensors output by the operators are different; The operators corresponding to the rows in the same position of the first footprint matrix and the second footprint matrix are in the same first partition, and the operators corresponding to the columns in the same position are in the same first partition; and, belong to the same first partition. The operators of the first computation graph are arranged in consecutive rows or columns in the first footprint matrix, and the operators of the second computation graph belonging to the same first partition are arranged in consecutive rows or columns in the second footprint matrix arrangement.

Because it is more likely that there is a mapping relationship between homogeneous operators, and in the embodiment of the present application, there is a mapping relationship between the operators corresponding to the rows or columns in the same position of the first footprint matrix and the second footprint matrix . Therefore, according to the method of the above-mentioned embodiment provided by the present application, the operators in the first calculation graph and the second calculation graph are homogeneously partitioned, and when adjusting the order of the rows and columns in the first footprint matrix, adjust the The order of the rows and columns corresponding to the operators located in the same homogeneous partition can make the iterative process converge more quickly and speed up the mapping process.

For operators without homogeneous operators, the corresponding operator similarity can be obtained by setting mixed partitions for classification and participating in the iterative calculation process. When determining the mapping operator, mapping suggestions can be output according to the calculated similarity. Therefore, it can support operator mapping between computational graphs with similar structures, and can also support one-to-many or many-to-many mapping, which can be applied to scenarios where deep learning frameworks perform model migration.

According to a third possible implementation manner of the first aspect, in a fifth possible implementation manner, the homogeneous depths of the operators corresponding to the rows in the same position of the first footprint matrix and the second footprint matrix are the same, The homogeneity depths of the operators corresponding to the columns in the same position are the same, and the corresponding rows or columns of the operators of the first calculation graph belonging to the same first partition in the first footprint matrix are the same as those of the operators. Homogeneous depth sorting, the rows or columns corresponding to the operators of the second computation graph of the same first partition in the second footprint matrix are sorted according to the homogeneous depth of the operators; wherein, the homogeneous depth is: The maximum number of operators before the operator that have the same type as the operator and the same output tensor dimension among all the branches to which the operator belongs in the calculation graph.

The operators in the homogeneous partition are sorted according to the homogeneous depth, and the first footprint matrix of the first computation graph and the second footprint matrix of the second computation graph are respectively determined according to the homogeneous partition. Since it is more likely that there is a mapping relationship between operators with the same homogeneous depth, the process of calculating the similarity matrix according to the above-mentioned embodiments of the present application can quickly converge, and the mapping relationship between operators can be determined more quickly. the process of mapping. In addition, by distinguishing operators with the same attributes by homogeneous depth, the unreliability of distinguishing operators with the same attributes by the names of the operators can be avoided.

According to a third possible implementation manner of the first aspect, in a sixth possible implementation manner, if the number of operators of the first computation graph in the first partition and the number of operators of the second computation graph The number of operators is different, and the first partition also includes pseudo operators; wherein, the pseudo operators are operators that do not have operator types and operands; if the first calculation graph in the first partition The number of operators is less than the number of operators in the second computation graph in the first partition, and the pseudo operator in the first footprint matrix corresponding to the row or column is located in the first After the row or column corresponding to the operator belonging to the first partition in the graph; if the number of operators in the second computation graph in the first partition is less than the number of operators in the first partition The number of operators in the calculation graph. The row or column corresponding to the pseudo operator in the second footprint matrix is located after the row or column corresponding to the operator belonging to the first partition in the second calculation graph.

According to the third or fifth possible implementation manner of the first aspect, in a seventh possible implementation manner, if the number of operators of the first computation graph in the second partition and the number of the The number of operators in the two computation graphs is different, the second partition also includes the pseudo-operator; if the number of operators in the first computation graph in the second partition is less than that in the second partition The number of operators in the second computation graph in the pseudo-operator, the rows and columns corresponding to the pseudo-operators in the first footprint matrix correspond to the operators belonging to the second partition in the first computation graph Rows and columns are randomly ordered in the same way; if the number of operators in the second computation graph in the second partition is less than the number of operators in the second computation graph in the second partition The number of operators, the rows and columns corresponding to the pseudo operators in the second footprint matrix and the rows and columns corresponding to the operators belonging to the second partition in the second calculation graph are randomly ordered, and the rows and columns Random sorting works the same way.

When the number of operators in the first computation graph and the second computation graph is different, the total number of operators in the first computation graph and the second computation graph is equal by adding pseudo-operators, thereby constructing footprint matrices of the same size, so that Subsequent matrix operations proceed normally. In this way, the operator mapping method provided by the present application can be applied to the scenarios of frameworks or migration models between systems using different computational graph representations.

According to any one of the third to sixth possible implementation manners of the first aspect, in an eighth possible implementation manner, the row and column values of the first footprint matrix are adjusted according to the first matrix. The order includes: adjusting the order of the rows and columns corresponding to the operators located in the same first partition in the first footprint matrix according to the first matrix.

According to any one of the third to sixth possible implementation manners of the first aspect, in a ninth possible implementation manner, the first footprint matrix belonging to the same first partition and when the stopping condition is reached There is a mapping relationship between the operators of the first calculation graph and the operators of the second calculation graph that have the same position as the corresponding row or column in the second footprint matrix; the first calculation belonging to the second partition There is a mapping relationship between the operators of the graph and multiple operators of the second calculation graph belonging to the second partition, wherein the multiple operators of the second calculation graph belonging to the second partition are: reaching the stop condition In the first matrix when , among the values of the elements corresponding to the operators of the first computation graph belonging to the second partition, the operators of the second computation graph corresponding to the largest values.

According to the mapping method of the above-mentioned embodiments of the present application, it is possible to support operator mapping between computational graphs with similar structures, support one-to-one, one-to-many, and many-to-many mappings, and can be applied to scenarios where deep learning frameworks perform model migration middle.

According to the first aspect or any one of the first to eighth possible implementation manners of the first aspect, in a tenth possible implementation manner, the stopping condition is that the number of times of repeatedly calculating the first matrix is greater than first threshold.

By performing homogeneous partitioning on the operators in the first computational graph and the second computational graph, and when adjusting the order of the rows and columns in the first footprint matrix, adjust the rows and columns corresponding to the operators located in the same homogeneous partition order can make the iterative process converge faster and speed up the mapping process. This is because the similarity of homogenous operators is higher, and there is a greater possibility of a mapping relationship between homogenous operators. Therefore, the process of homogenous partitioning can accelerate the speed of iterative convergence and the process of mapping.

In a second aspect, embodiments of the present application provide an operator mapping device for a computation graph, which is used to perform operator mapping on operators in a first computation graph and a second computation graph, the first computation graph and all The second calculation graph is the calculation graph of the same neural network model, and the operator mapping device includes: an iterative module for repeating the following process based on the first footprint matrix and the second footprint matrix until the stopping condition is reached: The first footprint matrix and the second footprint matrix calculate a first matrix, and adjust the order of rows and columns in the first footprint matrix according to the first matrix, and according to the adjusted first footprint matrix and all recompute the first matrix with the second footprint matrix; wherein, the first footprint matrix indicates the topological relationship between the operators of the first computation graph, and the second footprint matrix indicates the second computation graph The topological relationship between the operators of The footprint matrix and the second footprint matrix are used to obtain the operators corresponding to each of the multiple operators in the first calculation graph in the second calculation graph.

The operator mapping apparatus of the embodiments provided by the present application constructs footprint matrices of two computation graphs, where the footprint matrix is a matrix representing the topological relationship between operators in the computation graph. The first matrix can be iteratively calculated according to the footprint matrices corresponding to the two computation graphs, and the first matrix indicates the similarity of the topological relationship between the two computation graphs. After the iteration is stopped, one or more preferred mapping relationships can be determined according to the footprint matrix. Therefore, the operator mapping method provided by the present application can support operator mapping between calculation graphs with similar structures, and can be applied to a scenario where a deep learning framework performs model migration.

According to a first possible implementation manner of the second aspect, the iteration module includes:

The calculation unit is configured to obtain the first matrix according to a plurality of first eigenvectors and a plurality of second eigenvectors, wherein each of the first eigenvectors corresponds to an operator in the first calculation graph, and each Each of the first eigenvectors and the operators corresponding to the first eigenvectors are related to the row vectors and column vectors in the first footprint matrix, and each of the second eigenvectors corresponds to the second calculation graph The operators in , each of the second eigenvectors and the operators corresponding to the second eigenvectors are related to the row vectors and column vectors in the second footprint matrix, and the first eigenvectors and the The second eigenvector is used to calculate the similarity of the topological relationship between the operator corresponding to the first eigenvector and the operator corresponding to the second eigenvector.

According to the first possible implementation manner of the second aspect, in the second possible implementation manner, the value of the element of the first matrix represents the operator corresponding to the row where the element is located and the operator corresponding to the column where the element is located similarity of the topological relationship of the elements, wherein the operator corresponding to the row where the element is located and the operator corresponding to the column where the element is located are respectively the operator in the first calculation graph and the operator in the second calculation graph operator.

The eigenvectors of the operator are constructed by using the corresponding row and column vectors in the footprint matrix corresponding to the operator in the calculation graph, and the topological relationship characteristics of the operator in the calculation graph are extracted. The similarity of the topological relationship of the operator is calculated according to the eigenvector of the operator, and the similarity of the topological relationship of the operator of the first calculation graph and the operator of the second calculation graph forms a similarity matrix, and the similarity matrix is used to measure the two calculations. The similarity of the topological relationships of the graphs.

According to the second aspect, or the first or second possible implementation manner of the second aspect, in a third possible implementation manner, the first footprint matrix, the adjusted first footprint matrix and the third The operator corresponding to the row at the same position and the operator corresponding to the column at the same position of the two footprint matrix are of the same type, and the dimensions of the tensors output by the operators are the same. That is, the first footprint matrix and the second footprint matrix have the same type. The operator corresponding to the row at the same position and the operator corresponding to the column at the same position of the footprint matrix are of the same type, and the dimension of the tensor output by the operator is the same, and the adjusted first footprint matrix and the second footprint matrix are the same. The operator corresponding to the row of the position, the operator corresponding to the column of the same position are of the same type, and the dimension of the tensor output by the operator is the same.

When constructing the first footprint matrix and the second footprint matrix, and adjusting the first footprint matrix, ensure the type of the operator corresponding to the row at the same position and the type of the operator corresponding to the column at the same position of the first footprint matrix and the second footprint matrix The same, the dimensions of the tensors output by the operators are the same, that is, the operators corresponding to the rows in the same position of the first footprint matrix and the second footprint matrix and the operators corresponding to the columns in the same position are functionally the same. Combine the similarity matrix to measure the similarity of the topological relationship between operators. According to the operator mapping device provided by the present application, the mapping relationship between two operators can be reflected in terms of functions and topological relationships, thereby ensuring the correctness of the mapping result.

According to the second aspect, or the first or second possible implementation manner of the second aspect, in a fourth possible implementation manner, the operators in the first computation graph and the second computation graph are divided into Belonging to multiple first partitions, the same first partition has the same operator type, and the dimensions of the tensors output by the operators are the same, and each first partition may include at least part of the first computation graph and/or the second computation graph An operator, where some operators in the first calculation graph and the second calculation graph belong to a second partition, and the types of operators in the second partition are different and/or the dimensions of the tensors output by the operators are different; The operators corresponding to the rows in the same position of the first footprint matrix and the second footprint matrix are in the same first partition, and the operators corresponding to the columns in the same position are in the same first partition; and, belong to the same first partition. The operators of the first computation graph are arranged in consecutive rows or columns in the first footprint matrix, and the operators of the second computation graph belonging to the same first partition are arranged in consecutive rows or columns in the second footprint matrix arrangement.

Because it is more likely that there is a mapping relationship between homogeneous operators, and in the embodiment of the present application, there is a mapping relationship between the operators corresponding to the rows or columns in the same position of the first footprint matrix and the second footprint matrix . Therefore, according to the above-mentioned embodiment provided by the present application, the operators in the first calculation graph and the second calculation graph are homogeneously partitioned, and when adjusting the order of the rows and columns in the first footprint matrix, adjust the The order of the rows and columns corresponding to the operators of the homogeneous partition can make the iterative process converge more quickly and speed up the mapping process.

For operators without homogeneous operators, the corresponding operator similarity can be obtained by setting mixed partitions for classification and participating in the iterative calculation process. When determining the mapping operator, mapping suggestions can be output according to the calculated similarity. Therefore, it can support operator mapping between computational graphs with similar structures, and can also support one-to-many or many-to-many mapping, which can be applied to scenarios where deep learning frameworks perform model migration.

According to a third possible implementation manner of the second aspect, in a fifth possible implementation manner, the homogeneous depths of the operators corresponding to the rows in the same position of the first footprint matrix and the second footprint matrix are the same, The homogeneity depths of the operators corresponding to the columns in the same position are the same, and the corresponding rows or columns of the operators of the first calculation graph belonging to the same first partition in the first footprint matrix are the same as those of the operators. Homogeneous depth sorting, the rows or columns corresponding to the operators of the second computation graph of the same first partition in the second footprint matrix are sorted according to the homogeneous depth of the operators; wherein, the homogeneous depth is: The maximum number of operators before the operator that have the same type as the operator and the same output tensor dimension among all the branches to which the operator belongs in the calculation graph.

The operators in the homogeneous partition are sorted according to the homogeneous depth, and the first footprint matrix of the first computation graph and the second footprint matrix of the second computation graph are respectively determined according to the homogeneous partition. Since it is more likely that there is a mapping relationship between operators with the same homogeneous depth, the process of calculating the similarity matrix according to the above-mentioned embodiments of the present application can quickly converge, and the mapping relationship between operators can be determined more quickly. the process of mapping. In addition, by distinguishing operators with the same attributes by homogeneous depth, the unreliability of distinguishing operators with the same attributes by the names of the operators can be avoided.

According to a third possible implementation manner of the second aspect, in a sixth possible implementation manner, if the number of operators of the first computation graph in the first partition and the number of operators of the second computation graph The number of operators is different, and the first partition also includes pseudo operators; wherein, the pseudo operators are operators that do not have operator types and operands; if the first calculation graph in the first partition The number of operators is less than the number of operators in the second computation graph in the first partition, and the pseudo operator in the first footprint matrix corresponding to the row or column is located in the first After the row or column corresponding to the operator belonging to the first partition in the graph; if the number of operators in the second computation graph in the first partition is less than the number of operators in the first partition The number of operators in the calculation graph. The row or column corresponding to the pseudo operator in the second footprint matrix is located after the row or column corresponding to the operator belonging to the first partition in the second calculation graph.

According to the third or fifth possible implementation manner of the second aspect, in a seventh possible implementation manner, if the number of operators of the first computation graph in the second partition and the number of the The number of operators in the two computation graphs is different, the second partition also includes the pseudo-operator; if the number of operators in the first computation graph in the second partition is less than that in the second partition The number of operators in the second computation graph in the pseudo-operator, the rows and columns corresponding to the pseudo-operators in the first footprint matrix correspond to the operators belonging to the second partition in the first computation graph Rows and columns are randomly ordered in the same way; if the number of operators in the second computation graph in the second partition is less than the number of operators in the second computation graph in the second partition The number of operators, the rows and columns corresponding to the pseudo operators in the second footprint matrix and the rows and columns corresponding to the operators belonging to the second partition in the second calculation graph are randomly ordered, and the rows and columns Random sorting works the same way.

When the number of operators in the first computation graph and the second computation graph is different, the total number of operators in the first computation graph and the second computation graph is equal by adding pseudo-operators, thereby constructing footprint matrices of the same size, so that Subsequent matrix operations proceed normally. In this way, the operator mapping method provided by the present application can be applied to the scenarios of frameworks or migration models between systems using different computational graph representations.

According to any one of the third to sixth possible implementation manners of the second aspect, in an eighth possible implementation manner, the iteration module further includes: an adjustment unit, configured to adjust according to the first matrix The order of the rows and columns corresponding to the operators located in the same first partition in the first footprint matrix.

According to any one of the third to sixth possible implementation manners of the second aspect, in a ninth possible implementation manner, the first footprint matrix belonging to the same first partition and when the stopping condition is reached There is a mapping relationship between the operators of the first calculation graph and the operators of the second calculation graph that have the same position as the corresponding row or column in the second footprint matrix; the first calculation belonging to the second partition There is a mapping relationship between the operators of the graph and multiple operators of the second calculation graph belonging to the second partition, wherein the multiple operators of the second calculation graph belonging to the second partition are: reaching the stop condition In the first matrix when , among the values of the elements corresponding to the operators of the first computation graph belonging to the second partition, the operators of the second computation graph corresponding to the largest values.

According to the mapping method of the above-mentioned embodiments of the present application, it is possible to support operator mapping between computational graphs with similar structures, support one-to-one, one-to-many, and many-to-many mappings, and can be applied to scenarios where deep learning frameworks perform model migration middle.

According to the second aspect or any one of the first to eighth possible implementation manners of the second aspect, in a tenth possible implementation manner, the stopping condition is that the number of times of repeatedly calculating the first matrix is greater than or equal to first threshold.

By performing homogeneous partitioning on the operators in the first computational graph and the second computational graph, and when adjusting the order of the rows and columns in the first footprint matrix, adjust the rows and columns corresponding to the operators located in the same homogeneous partition order can make the iterative process converge faster and speed up the mapping process. This is because the similarity of homogeneous operators is higher, and there is a greater possibility of a mapping relationship between homogeneous operators. Therefore, the process of homogeneous partitioning can accelerate the speed of iterative convergence and the process of mapping.

In a third aspect, an embodiment of the present application provides an operator mapping apparatus for a computational graph, including:

A processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the first aspect or one or more of various possible implementations of the first aspect when executing the instructions A kind of operator mapping method.

In a fourth aspect, embodiments of the present application provide a non-volatile computer-readable storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above-mentioned first aspect or the first aspect is implemented One or more of the multiple possible implementations of the operator mapping method.

In a fifth aspect, an embodiment of the present application provides a terminal device, where the terminal device can execute the first aspect or one or more of the operator mapping methods in multiple possible implementations of the first aspect.

In a sixth aspect, embodiments of the present application provide a computer program product, comprising computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are stored in an electronic When running in the device, the processor in the electronic device executes the first aspect or one or more of the operator mapping methods in the multiple possible implementation manners of the first aspect.

These and other aspects of the present application will be more clearly understood in the following description of the embodiment(s).

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features and aspects of the application and together with the description, serve to explain the principles of the application.

1a and 1b respectively show schematic diagrams of application scenarios according to an embodiment of the present application.

FIG. 2 shows a block diagram of a device for performing an operator mapping method of a computational graph according to an embodiment of the present application.

FIG. 3 shows a flowchart of an operator mapping method for a computation graph according to an embodiment of the present application.

FIG. 4 shows a schematic diagram of a calculation graph according to an example of the present application.

Fig. 5a shows a flowchart of a process of calculating similarity according to an embodiment of the present application.

FIG. 5b shows a flowchart of a method for adjusting the order of rows and columns of a footprint matrix according to an embodiment of the present application.

FIG. 6 shows a flowchart of the method of step S300 according to an embodiment of the present application.

FIG. 7 shows a schematic diagram of a homogeneous partition according to an embodiment of the present application.

FIG. 8 and FIG. 9 respectively show schematic diagrams of determining the homogeneity depth of a homogeneity operator according to an embodiment of the present application.

FIG. 10 shows an example of an insertion pseudo-operator according to an embodiment of the present application.

FIG. 11 shows a flowchart of an operator mapping method according to an embodiment of the present application.

FIG. 12 shows a schematic diagram of determining the shortest distance according to an embodiment of the present application.

FIG. 13 shows a schematic diagram of adjusting the footprint matrix according to an embodiment of the present application.

FIG. 14 shows a schematic diagram of the effect of iterative calculation according to an embodiment of the present application.

FIG. 15 shows a schematic diagram of the effect of iterative calculation according to an embodiment of the present application.

FIG. 16 shows a block diagram of an operator mapping apparatus for a computation graph according to an embodiment of the present application.

FIG. 17 shows a block diagram of an operator mapping apparatus according to an embodiment of the present application.

Detailed ways

Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numbers in the figures denote elements that have the same or similar functions. While various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

In addition, in order to better illustrate the present application, numerous specific details are given in the following detailed description. It should be understood by those skilled in the art that the present application may be practiced without certain specific details. In some instances, methods, means, components and circuits well known to those skilled in the art have not been described in detail so as not to obscure the subject matter of the present application.

Glossary:

Operator: An operation unit in a calculation graph. An operator is used to complete a type of operation logic. An operator in this application can be understood as a mapping from one function space to another function space: y=f(x), also It is an operator in a broad sense. A calculation graph can include many different types of operators, and the type of the operator refers to the type of operation logic that the operator can perform. In this application, an operator may indicate a type of basic operation logic, and the basic operation logic cannot be split, such as Add (add), Multi (multiply), and so on. An operator can also indicate a kind of complex operation logic, and the complex operation logic can be expressed as a combination of a group of basic operation logic. In this way, complex operation logic can be indicated by a variety of operators, and each operator corresponds to a different combination of basic operation logic. These various operators have the same function. A group of operators composed of these various operators can be regarded as an operator that completes the above-mentioned complex operation logic. A group of operators composed of these various operators may be another operator. A homogeneous operator of a set of operators with the same function. For example: Conv (convolution) operator, sigmoid operator, etc., wherein, the conv operator can be composed of an addition operator and a multiplication operator.

Tensor: In this application, a tensor is a description of the characteristics of a piece of data stored, and a tensor records information such as the shape and type of the data. In this embodiment of the present application, a tensor may be understood as tensor data or an operand of an operator, which may include input operands and output operands of an operator in a neural network, and may also include feature tensor data and the like. Taking the artificial intelligence deep learning framework TensorFlow as an example, generally use rank (rank), shape (shape) and dimension (dimension number) to describe the dimension of the tensor.

Numerical file: A file that records the output tensor data (operands) of the deep learning model operator during the operation.

Operator mapping: In two computation graphs corresponding to the same model, an operator in one computation graph has an equivalent correspondence in function and topology with an operator in the other computation graph. That is to say, A functionally and topologically equivalent correspondence between two operators in two computation graphs.

Homogeneous operator: Two operators with identical properties are called homogeneous operators. In a deep learning model, the properties of an operator can refer to the type of the operator and the dimension of the output tensor. If the two operators are of the same type and the dimensions of the tensors output by the two operators are the same, then the two operators are homogeneous operators. That is, two homogenous operators are functionally identical. In this application, the operator referred to by a homogeneous operator may refer to an operator that completes basic, non-splittable operation logic, or may be a group of operators that complete an operation that can be split. It is divided into complex functions of basic operation logic. This group of operators and another group perform the same function. If the dimensions of the final output tensors of the two groups of operators are also the same, then these two groups of operators are also homogeneous operators.

Cosine similarity: the cosine value of the angle between two non-zero vectors. The larger the cosine value, the closer and similar the two vectors are.

Similarity matrix: The value in the similarity matrix represents the similarity between the operator corresponding to the row where the value is located and the operator corresponding to the column where the value is located. These two operators can be operators in different calculation graphs. For example, the operator corresponding to the row where the value is located is an operator of a calculation graph, and the operator corresponding to the column where the operator is located is an operator of another calculation graph. The similarity between two operators can be measured by the above cosine similarity. For example, a value in the matrix can represent the cosine similarity between the operator corresponding to the row where the value is located and the operator corresponding to the column where the value is located.

Cosine similarity of operators: the cosine value of the angle between the eigenvectors corresponding to two operators.

The eigenvector of the operator: a vector indicating the topological relationship between the operator and other operators in the computational graph.

Element: Each number in the matrix is called an element.

Calculate the distance from one operator to another in the graph: Calculate the length of the path from the node corresponding to one operator to the node corresponding to another operator in the graph. The connection between one node and another node in the computation graph is a reachable path. Since the computation graph is a directed graph, between two directly connected nodes in the computation graph: the path from the parent node to the child node (edge) is reachable, the path length is 1, the child node to the parent node is unreachable, and the length of the path from the child node to the parent node is 0. The length of the path from one node to another in the computation graph is the total length of a reachable path between the two nodes.

Calculate the shortest distance from one operator to another in the graph: Calculate the shortest length of the path from the node corresponding to one operator to the node corresponding to another operator in the graph.

In the related art, an isomorphic graph algorithm can be used to determine the mapping relationship between nodes of two graphs. However, the isomorphic graph algorithm is only suitable for the mapping of nodes between completely isomorphic graphs. For graphs with similar structures, node mapping cannot be performed, and the isomorphic graph algorithm does not support many-to-many node mapping. In different deep learning frameworks, the expression of computational graphs is different. Even the same neural network model may correspond to computational graphs of different structures after conversion in different deep learning frameworks. Therefore, the isomorphic graph algorithm cannot be applied to different deep learning frameworks for model transfer scenarios.

In order to solve the above-mentioned technical problems, the present application provides an operator mapping method of a computational graph. By constructing footprint matrices of two computational graphs, the footprint matrix is a matrix representing the topological relationship between the operators of the computational graph. The value of the element is the reciprocal of the shortest distance from the operator corresponding to the row where the element is located to the operator corresponding to the column where the element is located, or the value of the element in the footprint matrix is the operator corresponding to the column where the element is located to the row where the element is located. The reciprocal of the shortest distance of the corresponding operator. The similarity matrix can be iteratively calculated according to the footprint matrices corresponding to the two calculation graphs, and the similarity matrix indicates the similarity of the topological relationship between the two calculation graphs. After the iteration is stopped, one or more preferred mapping relationships can be determined according to the footprint matrix. Therefore, the operator mapping method provided by the present application can support operator mapping between calculation graphs with similar structures, and can be applied to a scenario where a deep learning framework performs model migration.

The operator mapping method of the computational graph in the embodiment of the present application is mainly applied to the computational graph of the neural network, and the neural network may be a neural network model used for image recognition, speech recognition, and the like. The operator mapping method in the embodiment of the present application can be applied to the scenario of performing neural network model migration between different deep learning frameworks, and can also be applied to the scenario of performing neural network model migration between different versions of the same deep learning framework In the scenario of comparing neural networks between different hardware, and in the scenario of comparing the results of training and inference, comparison of design diagrams related to integrated circuits, comparison of calculation diagrams of program codes, and so on. This application does not limit the specific application scenarios, as long as it is a scenario of comparison mapping between two graphs expressed in the form of graphs in graph theory.

Taking a scenario of model migration between different deep learning frameworks as an example, FIG. 1 a and FIG. 1 b respectively show schematic diagrams of an application scenario according to an embodiment of the present application. FIG. 2 shows a block diagram of a device for performing an operator mapping method of a computational graph according to an embodiment of the present application.

Figure 1a shows the application scenario of manual migration. When the model is migrated from deep learning framework 1 to deep learning framework 2, the corresponding calculation diagram 1 and numerical file 1 can be derived from the model script of deep learning framework 1, in which , the numerical file can include data such as the operand Tensor and weight of the operator in the calculation graph. The developer can write the model script of the deep learning framework 2 according to the model script of the deep learning framework 1, and derive the corresponding calculation diagram 2 and the numerical file 2 from the model script of the deep learning framework 2.

The device shown in FIG. 2 may be a computer, a server, etc. The device shown in FIG. 2 may include a processor and a memory, wherein the processor may be equipped with an operating system, and the upper layer of the operating system may be installed with calculation graph comparison software. The processor can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), or a central processing unit. It can be a central processor unit (CPU), a network processor (NP), a digital signal processing circuit (DSP), or a microcontroller (MCU) , it can also be a programmable logic device (PLD) or other integrated chips.

The memory can be read-only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (RAM) or other types of storage devices that can store information and instructions Dynamic storage device, the memory can also include non-volatile memory (non-volatile memory), such as flash memory (flash memory), hard disk drive (hard disk drive, HDD) or solid-state drive (solid-state drive, SSD); memory Combinations of the above kinds of memories may also be included. The memory can also be electrically erasable programmable read-only memory (electrically erasable programmable read-only memory, EEPROM), compact disc read-only memory (CD-ROM) or other optical disk storage, optical disk storage (including compressed optical disks) , laser disc, compact disc, digital versatile disc, Blu-ray disc, etc.), magnetic disk storage medium or other magnetic storage device, or any other device capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer Other media, but not limited to this. The memory may exist independently and be connected to the processor through a communication line, such as the hard disk storage device shown in FIG. 2 . Memory can also be integrated with the processor, as shown in Figure 2. The memory provided by the embodiments of the present application may generally be non-volatile. The memory is used to store the computer-executed instructions involved in executing the solution of the present application, and the execution is controlled by the processor. The processor is configured to execute the computer-executed instructions stored in the memory, thereby implementing the method provided by the embodiments of the present application.

The calculation graph comparison software can be a set of instructions that integrates the method provided by the present application, and the processing of the input calculation graph by the processor running the instruction can realize the mapping relationship between the operators of the calculation graph to obtain the mapping result, and according to the mapping result , computational graphs, and numerical archives to detect mapping results.

Specifically, the derived calculation diagram 1 and numerical file 1 of the deep learning framework 1, and the calculation diagram 2 and numerical file 2 of the deep learning framework 2 can be input into the processor, and the processor executes the operators of the calculation diagram provided by this application. The mapping method can find out the mapping relationship between the operators in the calculation Figure 1 of the deep learning framework 1 and the calculation Figure 2 of the deep learning framework 2 (the steps shown in the solid line box in Figure 1a are provided by the present application. The operator mapping method of FIG. 1a is realized. The steps shown in the dotted box in Figure 1a are the process of preprocessing to obtain the calculation graph or the process of subsequent adjustment and processing according to the mapping result, which belong to a part of the application scenario of the present application. The dotted box shows steps are not the focus of this application). Then the processor can compare whether the Tensor corresponding to the operator in the corresponding numerical file 1 and numerical file 2 is consistent according to the mapping relationship, and output the comparison result between the mapping relationship and the numerical file as a comparison report. If the comparison result of the numerical files is inconsistent, or the developer judges that the main structure of the calculation graph is inconsistent according to the mapping relationship, the model script of the deep learning framework 2 can be modified, and then the above process is performed again until the comparison results of the numerical files are consistent and The main structure of the computation graph is consistent.

Figure 1b shows the application scenario of automatic migration. The model script of the deep learning framework 1 can be converted into a file in the Open Neural Network Exchange (ONNX, Open Neural Network Exchange) format, which is a file used to represent deep learning models. A standard that enables models to be transferred between different deep learning frameworks. Then use the script conversion tool to convert the file in ONNX format to obtain the model script of the deep learning framework 2. According to the file in ONNX format, the calculation diagram 1 can be exported, and the calculation diagram 2 can be derived according to the model script of the deep learning framework 2. The calculation diagram 1 and the calculation diagram 2 are input into the device shown in FIG. 2, and the processor executes the application. The operator mapping method can obtain the mapping relationship of the operators in the calculation diagram 1 and the calculation diagram 2, and verify the mapping result according to the mapping relationship.

In the application scenarios shown in Figure 1a and Figure 1b, the deep learning framework 1 and the deep learning framework 2 may be any of TensorFlow, PyTorch, MindSpore, and the like. It should be noted that the above FIG. 1a and FIG. 1b are only examples of application scenarios of the present application, and do not limit the present application in any way. The operator mapping methods provided in the embodiments of the present application may also be applied to other application scenarios.

FIG. 3 shows a flowchart of an operator mapping method for a computation graph according to an embodiment of the present application. The operator mapping method provided by the present application can be used to perform operator mapping on operators in a first calculation graph and a second calculation graph, wherein the first calculation graph and the second calculation graph are the same neural network The computation graph of the model, the first computation graph and the second computation graph may be computation graphs corresponding to different versions of the same deep learning framework, or may be computation graphs corresponding to different deep learning frameworks, which are not limited in this application. The method steps shown in FIG. 3 may be performed by the processor shown in FIG. 2 .

As shown in FIG. 3 , the operator mapping method for a computational graph according to an embodiment of the present application may include the following steps:

Step S300, based on the first footprint matrix and the second footprint matrix, repeat the following process until a stop condition is reached: calculate a first matrix according to the first footprint matrix and the second footprint matrix, and calculate the first matrix according to the first footprint matrix The order of rows and columns in the first footprint matrix is adjusted, and the first matrix is recalculated according to the adjusted first footprint matrix and the second footprint matrix.

Wherein, the first footprint matrix indicates a topological relationship between operators in the first computation graph, the second footprint matrix indicates a topological relationship among operators in the second computation graph, and the first footprint matrix indicates a topological relationship between operators in the second computation graph. The matrix indicates the similarity of the topological relationship of the first computational graph and the second computational graph.

Step S301 , according to the first footprint matrix and the second footprint matrix when the stopping condition is reached, obtain operators corresponding to each of the multiple operators in the first calculation graph in the second calculation graph.

In a possible implementation manner, the value of an element of the first footprint matrix may be the reciprocal of the shortest distance from the operator corresponding to the row where the element is located to the operator corresponding to the column where the element is located. The reciprocal of the shortest distance from operator to operator in a computation graph describes the topological relationship between operators in the first computation graph. The value of an element in the second footprint matrix can be the reciprocal of the shortest distance from the operator corresponding to the row where the element is located to the operator corresponding to the column where the element is located. The reciprocal of the shortest distance of the children describes the topological relationship between the operators of the second computational graph.

In a possible implementation manner, the processor may traverse each operator in the first computation graph and the second computation graph respectively, and determine the shortest distance from each operator to other operators in the computation graph. Specifically, the processor may use the traversed operator as a starting point, perform a depth-first-search (DFS) on the computation graph, and determine the shortest distance from the operator to other operators. Since the first computation graph and the second computation graph are both directed graphs, some operators are unreachable from other operators. In this case, the shortest distance from the operator to the unreachable operator can be determined to be infinite. The reciprocal of the shortest distance from the operator to the unreachable operator is 0, so the corresponding element value of the operator in the footprint matrix is 0. In addition, if there is more than one path from an operator to another operator, and the distances of these paths are different, it can be determined that the distance corresponding to the path with the shortest distance is the shortest distance from the operator to the other operator. The reciprocal of the operator's shortest distance to itself is 0.

FIG. 4 shows a schematic diagram of a calculation graph according to an example of the present application. Taking FIG. 4 as an example, the process of determining the footprint matrix according to the calculation graph in the embodiment of the present application is described. It is assumed that the calculation graph shown in FIG. 4 is the first calculation graph.

The processor performs depth-first traversal with node (operator) A as the starting point, and the shortest distances from operator A to other operators B, C, D, E, and F are obtained as: 1, 2, 3, 2, and 3, respectively. For operator E, A has two paths to reach operator E, and the distances of the two paths are 2 and 3 respectively. Therefore, the shortest distance from operator A to operator E is 2.

The processor performs depth-first traversal with node B as the starting point, and the shortest distances from node B to nodes A, C, D, E, and F can be obtained: ∞, ∞, ∞, 1, and 2, respectively.

The processor performs depth-first traversal with node C as the starting point, and the shortest distances from node C to nodes A, B, D, E and F can be obtained: ∞, ∞, 1, 2, and 3, respectively.

The processor performs depth-first traversal with node D as the starting point, and the shortest distances from node D to nodes A, B, C, E, and F can be obtained: ∞, ∞, ∞, 1, and 2, respectively.

The processor performs depth-first traversal with node E as the starting point, and the shortest distances from node E to nodes A, B, C, D, and F can be obtained: ∞, ∞, ∞, ∞, and 1, respectively.

The processor performs depth-first traversal with node F as the starting point, and the shortest distances from node F to nodes A, B, C, D, and E can be obtained as ∞, ∞, ∞, ∞, and ∞, respectively. The shortest distance from each node to itself is ∞.

According to the above process, the first footprint matrix of the first calculation graph shown in FIG. 4 may be:

	A	B	C	D	E	F
A	0	1	1/2	1/3	1/2	1/3
B	0	0	0	0	1	1/2
C	0	0	0	1	1/2	1/3
D	0	0	0	0	1	1/2
E	0	0	0	0	0	1
F	0	0	0	0	0	0

According to the above process, the first footprint matrix of the first computation graph and the second footprint matrix of the second computation graph can be determined. It should be noted that, what is shown in FIG. 4 is only an example of the present application, and does not limit the present application in any way.

In a possible implementation manner, when the processor traverses the shortest distance from an operator to other operators, if the shortest distance exceeds the maximum access distance, the processor may set the reciprocal of the shortest distance between the two operators to 0. The maximum access distance may be a distance threshold set to limit the amount of calculation. For example, in the embodiment of the present application, the maximum access distance may be set to 15. When the shortest distance from other operators to the operator at the starting point exceeds the maximum access distance, the influence of the other operators on the topology of the operator as the starting point is relatively small, so it can be ignored. In this way, the sparsity of the footprint matrix can be increased and the amount of computation can be reduced.

In one possible implementation, the similarity between two non-zero vectors can be measured by cosine similarity. Among them, cosine similarity refers to the cosine value of the angle between two non-zero vectors, and the larger the cosine value, the closer the two vectors are pointing. For example, suppose two non-zero vectors

and

The cosine similarity can be expressed as the following formula (1),

In the embodiment of the present application, the first matrix is used to indicate the similarity of the topological relationship between the first calculation graph and the second calculation graph. Therefore, in the embodiment of the present application, the first matrix may also be called Make a similarity matrix (the similarity matrix of the first calculation graph and the second calculation graph). In a possible implementation manner, the value of the element in the first matrix represents the similarity of the topological relationship between the operator corresponding to the row where the element is located and the operator corresponding to the column where the element is located. In the embodiment of the present application, the feature of the operator of the first calculation graph and the feature of the operator of the second calculation graph can be extracted and expressed in the form of a feature vector, according to the feature vector of the operator of the first calculation graph and The eigenvectors of the operators of the second calculation graph are obtained, and the cosine similarity of the operators of the first calculation graph and the operators of the second calculation graph is obtained to describe the difference between the operators of the first calculation graph and the operators of the second calculation graph. similarity of topological relationships.

Since the footprint matrix is a matrix representing the topological relationship between the operators of the computation graph, in a possible implementation manner, the processor may construct the eigenvectors of the operators in the computation graph according to the footprint matrix, and according to the two operators The cosine similarity of the eigenvectors can measure the similarity of the topological relationship between two operators.

In a possible implementation manner, calculating the first matrix according to the first footprint matrix and the second footprint matrix in step S300 may include:

The first matrix is obtained according to a plurality of first eigenvectors and a plurality of second eigenvectors, wherein each of the first eigenvectors corresponds to an operator in the first calculation graph, and each of the first eigenvectors corresponds to an operator in the first calculation graph. The row vector corresponding to the operator corresponding to the eigenvector and the first eigenvector in the first footprint matrix is related to the column vector, and each of the second eigenvectors corresponds to the operator in the second calculation graph, Each of the second eigenvectors and the operator corresponding to the second eigenvectors are related to row vectors and column vectors in the second footprint matrix, and the first eigenvectors and the second eigenvectors are represented by for calculating the similarity of the topological relationship between the operator corresponding to the first eigenvector and the operator corresponding to the second eigenvector.

That is to say, each operator in the first calculation graph has a first eigenvector corresponding to the operator, and the first eigenvector is related to the row vector and column vector corresponding to the operator in the first footprint matrix. The first calculation The row vector corresponding to the operator of the graph in the first footprint matrix may refer to the row vector of the operator of the first calculation graph in the row corresponding to the first footprint matrix, and the corresponding column vector may refer to the operator of the first calculation graph Column vector of the corresponding column in the first footprint matrix. Each operator in the second calculation graph has a second eigenvector corresponding to the operator. The second eigenvector is related to the row and column corresponding to the operator in the second footprint matrix. The operator of the second calculation graph is in The corresponding row vector in the second footprint matrix may refer to the row vector of the operator of the second computation graph in the row corresponding to the second footprint matrix, and the corresponding column vector may refer to the operator of the second computation graph in the second footprint matrix A column vector of corresponding columns.

In other words, the processor can obtain the first eigenvector of the operator in the first calculation graph according to the row vector and the column vector in the first footprint matrix, and obtain the second calculation graph according to the row vector and column vector in the second footprint matrix The second characteristic of the operators is the same.

In a possible implementation manner, the first eigenvector and the second eigenvector are used to calculate the similarity of the topological relationship between the operator corresponding to the first eigenvector and the operator corresponding to the second eigenvector The degree may mean that the similarity of the topological relationship between the two operators can be obtained by calculating the cosine similarity of the first eigenvector and the second eigenvector corresponding to the two operators. For example, calculate the cosine similarity of the first eigenvector of operator A in the first calculation graph and the second eigenvector of operator B in the second calculation graph as the similarity of the topological relationship between operator A and operator B Spend. In this way, the value of the element of the first matrix represents the similarity of the topological relationship between the operator corresponding to the row where the element is located and the operator corresponding to the column where the element is located, wherein the operator corresponding to the row where the element is located and the operator corresponding to the column where the element is located are the operators in the first calculation graph and the operators in the second calculation graph, respectively.

In a possible implementation manner, the operator corresponding to the row of the first matrix may be the operator of the first calculation graph, and the operator corresponding to the column may be the operator of the second calculation graph, or, in another possibility In the implementation manner of , the operator corresponding to the row of the first matrix may be the operator of the second calculation graph, and the operator corresponding to the column may be the operator of the first calculation graph.

FIG. 5a shows a flowchart of a process of calculating a first matrix according to an embodiment of the present application.

As shown in FIG. 5a, in a possible implementation manner, obtaining the first matrix according to multiple first eigenvectors and multiple second eigenvectors may specifically include the following process:

Step S3010, determining the first eigenvector of the operator in the first calculation graph according to the first footprint matrix, and determining the second eigenvector of the operator in the second calculation graph according to the second footprint matrix;

Step S3011: Calculate the first matrix according to the first eigenvector and the second eigenvector.

In a possible implementation manner, the processor may concatenate the row vector of the row where the operator is located in the corresponding footprint matrix and the column vector of the column where the operator is located to obtain the feature vector of the operator. The elements in the vector are spliced behind the column vector to obtain the eigenvector of the operator, or the elements in the column vector can be spliced behind the row vector to obtain the eigenvector of the operator. This application does not limit the specific splicing method, but the processor The way of splicing the row vector and the column vector of the operator in the first calculation graph is the same as the way of splicing the row vector and the column vector of the operator in the second calculation graph.

For example, taking operator A in the example shown in FIG. 4 as an example, the row where operator A is located in the footprint matrix is the first row, and the first row vector is [0,1,1/2,1/3, 1/2,1/3], the column where operator A is located in the footprint matrix is the first column, the first column vector is [0,0,0,0,0,0], and the column vector and row vector are spliced The eigenvectors of operator A can be obtained as [0,0,0,0,0,0,0,1,1/2,1/3,1/2,1/3].

After the eigenvectors of the operators are determined, the cosine similarity between the first eigenvectors of the operators in the first calculation graph and the second eigenvectors of the corresponding operators in the second calculation graph can be calculated according to formula (1). , to get the similarity of the operators. The similarity between the operators in the first calculation graph and the operators in the second calculation graph may form a first matrix.

For example, it is assumed that the first computational graph is Q and the second computational graph is R.

The first footprint matrix of the first computation graph Q is:

	q(1)	q(2)	q(3)	…	q(n-1)	q(n)
q(1)	0	…	…	…	…	…
q(2)	…	0	…	…	…	…
q(3)	…	…	0	…	…	…
…	…	…	…	0	…	…
q(n-1)	…	…	…	…	0	…
q(n)	…	…	…	…	…	0

The second footprint matrix of the second computation graph R is:

	r(1)	r(2)	r(3)	…	r(n-1)	r(n)
r(1)	0	…	…	…	…	…
r(2)	…	0	…	…	…	…
r(3)	…	…	0	…	…	…
…	…	…	…	0	…	…
r(n-1)	…	…	…	…	0	…
r(n)	…	…	…	…	…	0

It should be noted that the operator corresponding to the row in the footprint matrix can be used as the starting point of the access path, the operator corresponding to the column can be used as the end point of the access path, or the operator corresponding to the column in the footprint matrix can be used as the starting point of the access path. The operator corresponding to the row is used as the end point of the access path, which is not limited in this application.

The processor may determine the first eigenvectors of the operators q(1), q(2)...q(n) in the first calculation graph according to the above-mentioned first footprint matrix, and determine the second calculation graph according to the above-mentioned second footprint matrix. The second eigenvectors of the operators r(1), r(2)...r(n).

The first eigenvectors of operators q(1), q(2)...q(n) can be expressed as:

In the above table, a vector formed by a column of data where the operator is located can represent the first eigenvector of the operator. Above the thick solid line can be the column vector of the column where the operator is located, and below the thick solid line can be the row of the row where the operator is located. The vector is concatenated into a column vector.

The second eigenvectors of operators r(1), r(2)...r(n) can be expressed as:

In the above table, the vector formed by a column of data where the operator is located can represent the second eigenvector of the operator. Above the thick solid line can be the column vector of the column where the operator is located, and below the thick solid line can be the row of the row where the operator is located. The vector is concatenated into a column vector.

For example, the similarity of operators q(1) and r(1) can be calculated by the following formula (2):

in,

represents the first eigenvector of the operator q(1),

Represents the second eigenvector of operator r(1). According to the above formula (2), the similarity between operators q(1) and r(1) can be calculated. The operator of the first calculation graph and the second calculation graph of The similarity of operators can form the similarity matrix of the first calculation graph and the second calculation graph, as shown in the following table:

In step S300, the processor adjusts the order of rows and columns in the first footprint matrix according to the first matrix, which may refer to adjusting the order between rows and rows and between columns and columns in the first footprint matrix according to the first matrix , while maintaining the order of rows and columns in the second footprint matrix. It should be noted that, in the embodiments of the present application, adjusting the order of rows and columns in the first footprint matrix is only an example of the present application, and it is also possible to adjust the order of rows and columns in the second footprint matrix while maintaining the first footprint matrix. The order of rows and columns remains unchanged.

In a possible implementation manner, the specific process that the processor adjusts the order of the rows and columns in the first footprint matrix according to the similarity matrix may include: the processor determines the first calculation graph and the second calculation graph according to the similarity matrix A mapping relationship between operators, according to which the order of rows and columns in the first footprint matrix corresponding to the operators of the first calculation graph is adjusted.

Exemplarily, the processor may determine the degree of similarity between the operator in the first calculation graph and the operator in the second calculation graph according to the value of the elements in the similarity matrix, and determine the first calculation graph and the second calculation graph. To calculate the mapping relationship between the operators of the graph, for example, you can select the largest element values in the similarity matrix, and determine that the two operators corresponding to the largest element values have a mapping relationship. If the operators correspond to different rows and columns in the first footprint matrix and the second footprint matrix, the relationship between the rows and columns in the first footprint matrix can be adjusted, so that the two operators with a mapping relationship after adjustment are in the The first footprint matrix and the second footprint matrix correspond to the same rows and columns. If it is determined that the two operators with a mapping relationship correspond to the same rows and columns in the first footprint matrix and the second footprint matrix, then the order of the rows and columns of the first footprint matrix may not be adjusted first, and continue from the similarity Select the larger element value in the matrix and repeat the above process.

It should be noted that the above is just an example of adjusting the order of the rows and columns of the first footprint matrix according to the similarity matrix, and does not limit the present application in any way.

When adjusting the row and column order of the first footprint matrix, the rows and columns with the same serial number should be adjusted at the same time. The positions of the first and second rows, and the first and second columns in the footprint matrix, the adjusted first footprint matrix can be expressed as:

	q(2)	q(1)	q(3)	…	q(n-1)	q(n)
q(2)	0	…	…	…	…	…
q(1)	…	0	…	…	…	…
q(3)	…	…	0	…	…	…
…	…	…	…	0	…	…
q(n-1)	…	…	…	…	0	…
q(n)	…	…	…	…	…	0

After adjusting the order of rows and columns in the first footprint matrix, the mapping relationship between the operators in the first calculation graph and the operators in the second calculation graph has changed, and the eigenvectors of the operators are also different from those before the adjustment. Therefore, the similarity matrix can be recalculated according to the adjusted first footprint matrix and the second footprint matrix.

The above process is repeated until a stop condition is reached, where the stop condition may be a condition indicating that the process of adjusting the first footprint matrix and calculating the first matrix is stopped. In a possible implementation manner, the stopping condition may refer to a condition that is satisfied by the first matrix obtained by multiple calculations, for example, the sum of the element values in the first matrix is greater than the similarity threshold, or it can also be set to be similar to Degree-dependent objective function, when the value of the objective function is maximum, the stopping condition is reached.

For example, the objective function of iteration can be set as

Among them, M represents a mapping relationship set between operators, and m represents a mapping relationship between two specific operators in this mapping relationship set. If, according to the mapping relationship between the operators corresponding to the adjusted first footprint matrix and the second footprint matrix, the calculated value of the objective function satisfies a certain range, or the calculated value of the objective function is no longer changes, then the processor may determine that the stop condition is reached.

In another possible implementation manner, the stop condition may also be the number of iterations. For example, the stop condition may be that the number of times the first matrix is repeatedly calculated is greater than the first threshold, that is, the number of times corresponding to the first threshold is performed. After the process of computing the first matrix, the order of the rows and columns of the first footprint matrix is no longer adjusted. This application does not limit this.

In a possible implementation manner, when the stopping condition is reached, the first footprint matrix and the second footprint matrix have the same row or column position as the operator of the first computation graph and the operator of the second computation graph. There is a mapping relationship between them.

After the stop condition is reached, for step S301, the processor may determine that the operators corresponding to the rows or columns of the same position of the first footprint matrix and the second footprint matrix when the stop condition is reached are mapping operators to each other, and the row of the same position It can refer to the same row number, for example, the first row of the first footprint matrix and the second row of the second footprint matrix are in the same position, the operator corresponding to the first row of the first footprint matrix and the first row of the second footprint matrix. There is a mapping relationship between the operators corresponding to the two rows. Alternatively, multiple mapping operators can also be determined for one operator according to the calculated similarity matrix. For example, the similarity between one operator and other operators can be sorted according to the calculated similarity matrix value. The multiple operators with the highest similarity with the operator are determined as the mapping operators of the operator.

In a possible implementation manner, the type of the first footprint matrix, the adjusted first footprint matrix, and the operator corresponding to the row at the same position and the type of the operator corresponding to the column at the same position of the second footprint matrix The same, the dimensions of the tensors output by the operators are the same, that is, the operators corresponding to the rows at the same position of the first footprint matrix and the second footprint matrix and the operators corresponding to the columns at the same position are of the same type , the dimensions of the tensors output by the operators are the same, the operators corresponding to the rows at the same position and the columns at the same positions of the adjusted first footprint matrix and the second footprint matrix are of the same type, and the operators output The dimensions of the tensors are the same.

When constructing the first footprint matrix and the second footprint matrix, and adjusting the first footprint matrix, ensure the type of the operator corresponding to the row at the same position and the type of the operator corresponding to the column at the same position of the first footprint matrix and the second footprint matrix The same, the dimensions of the tensors output by the operators are the same, that is, the operators corresponding to the rows in the same position of the first footprint matrix and the second footprint matrix and the operators corresponding to the columns in the same position are functionally the same. Combine the similarity matrix to measure the similarity of the topological relationship between operators. According to the operator mapping method provided by the present application, the mapping relationship between two operators can be reflected in terms of functions and topological relationships, thereby ensuring the correctness of the mapping result.

According to the operator mapping method for computational graphs provided by the present application, one or more preferred mapping relationships can be obtained through iterative calculation, so it can support operator mapping between computational graphs with similar structures, and can be applied to deep learning frameworks for In the scenario of model migration.

In a possible implementation manner, the operators in the first calculation graph and the second calculation graph belong to multiple first partitions, the operators in the same first partition are of the same type, and the operator outputs The dimensions of the quantities are the same; some operators in the first calculation graph and the second calculation graph belong to the second partition, the types of operators in the second partition are different and/or the dimensions of the tensors output by the operators are different different.

Among them, operators with the same type and the same dimension of output tensors can be called homogeneous operators, so the first partition can also be called homogeneous partition. The types of operators in the second partition are different and/or the dimensions of the tensors output by the operators are different. Therefore, the operators in the second partition are not homogeneous operators, and the second partition may also be called a hybrid partition.

In a possible implementation manner, the processor may classify the operators of the first calculation graph and the operators of the second calculation graph according to the type of the operator and the dimension of the tensor output by the operator, and classify the homogeneous operators Divided into one class, each class of homogeneous operators belongs to a first partition. Each of the first partitions may include at least part of the operators of the first computation graph and/or the second computation graph.

There may be no homogeneous operators in some operators in the first calculation graph and the second calculation graph. Such operators may belong to the second partition (hybrid partition), and the second partition may include the first calculation graph and/or the second partition. 2. Partial operators of computational graphs. For the operators in the first calculation graph and the second calculation graph, the operators without homogeneous operators all belong to the mixed partition. For example, some operators in the calculation graph are unmappable operator types, or there are no operators with the same dimension of output operands, and there are no homogeneous operators for such operators. For example, MS's tuple_getitem operator and TF's Identity operator cannot find corresponding operator types with the same functions from each other's framework. Such operators can be divided into mixed partitions, and when a mapping operator is determined, a mapping suggestion can be output according to the first footprint matrix, the second footprint matrix and the calculated similarity matrix when the stopping condition is reached.

In a possible implementation manner, the operator and the first computation graph belong to the same first partition and have the same row or column position in the first footprint matrix and the second footprint matrix when the stopping condition is reached. There is a mapping relationship between the operators of the two computation graphs.

In a possible implementation manner, there is a mapping relationship between the operators of the first computation graph belonging to the second partition and multiple operators of the second computation graph belonging to the second partition, wherein the operators belonging to the first computation graph The multiple operators of the second computation graph of the second partition are: in the first matrix when the stopping condition is reached, among the values of the elements corresponding to the operators of the first computation graph belonging to the second partition, The operator of the second computation graph corresponding to the largest multiple values. According to the first footprint, the second matrix, and the first matrix when the stopping condition is reached, a mapping suggestion between the operator of the first computation graph and the multiple operators of the second computation graph in the mixed partition can be output, or the first computation graph in the mixed partition can be output. The operator of the second calculation graph and the mapping suggestion of multiple operators in the first calculation graph. For example, for the operator of the first calculation graph in the mixed partition, according to the similarity value corresponding to the operator in the similarity matrix, multiple operators (top-k, k is the number of operators suggested when outputting mapping suggestions, and k is a positive integer) output.

Because it is more likely that there is a mapping relationship between homogeneous operators, and in the embodiment of the present application, there is a mapping relationship between the operators corresponding to the rows or columns in the same position of the first footprint matrix and the second footprint matrix . Therefore, when determining the first footprint matrix and the second footprint matrix, the processor may arrange the operators of the first computation graph in the rows corresponding to the first footprint matrix and the operators of the second computation graph in the second footprint according to the first partition. The corresponding rows in the matrix, and the columns corresponding to the operators of the first computation graph in the first footprint matrix and the columns corresponding to the operators of the second computation graph in the second footprint matrix are arranged according to the first partition.

Specifically, in a possible implementation manner, the operators corresponding to the rows in the same position of the first footprint matrix and the second footprint matrix are in the same first partition, and the operators corresponding to the columns in the same position are in the same The first partition; and, the operators of the first computation graph belonging to the same first partition are arranged consecutively in the corresponding rows or columns in the first footprint matrix, and the operators of the second computation graph belonging to the same first partition are in the same first partition. The corresponding rows or columns in the second footprint matrix are arranged consecutively. That is to say, the row corresponding to the operator of the first computation graph belonging to the same first partition in the first footprint matrix and the row corresponding to the operator of the second computation graph in the second footprint matrix have the same sequence number range, and belong to the same The columns corresponding to the operators of the first computation graph of the first partition in the first footprint matrix and the columns corresponding to the operators of the second computation graph in the second footprint matrix have the same serial number range. For example, the operators q(1), q(2) and q(3) of the first computation graph belonging to the same first partition are in the rows 1-3 and columns 1-3 corresponding to the first footprint matrix. 3 columns, the operators r(1), r(2) and r(3) of the second computation graph belonging to the same first partition as the operators q(1), q(2) and q(3) are in the second The footprint matrix corresponds to rows 1-3 and columns 1-3.

FIG. 6 shows a flowchart of constructing a footprint matrix according to an embodiment of the present application. As shown in FIG. 6, in a possible implementation manner, the process of respectively determining the first footprint matrix of the first computation graph and the second footprint matrix of the second computation graph may include:

Step S3000, according to the type of the operator and the dimension of the tensor output by the operator, partition the operators in the first calculation graph and the second calculation graph to obtain a first partition; The operator is a homogeneous operator, the type of the homogeneous operator is the same, and the dimension of the tensor output by the operator is the same;

Step S3002, respectively determining a first footprint matrix of the first computation graph and a second footprint matrix of the second computation graph according to the first partition.

In this implementation manner, adjusting the order of rows and columns in the first footprint matrix according to the first matrix in step S300 may include: adjusting according to the first matrix the order of rows and columns in the first footprint matrix that are located in the same The order of the rows and columns corresponding to the operators in the first partition.

By performing homogeneous partitioning on the operators in the first computational graph and the second computational graph, and when adjusting the order of the rows and columns in the first footprint matrix, adjust the rows and columns corresponding to the operators located in the same homogeneous partition order can make the iterative process converge faster and speed up the mapping process. This is because the similarity of homogeneous operators is higher, and there is a greater possibility of a mapping relationship between homogeneous operators. Therefore, the process of performing homogeneous partitioning can speed up the speed of convergence in step S300 and speed up the mapping process.

For step S3000, the processor may divide the operators of the first computation graph and the second computation graph with the same type and the same dimension of the tensors output by the operators into the same homogeneous partition. That is to say, operators with the same type and the same dimension of the tensors output by the operators in the first calculation graph and the second calculation graph are located in the same homogeneous partition.

FIG. 7 shows a schematic diagram of a homogeneous partition according to an embodiment of the present application. As shown in FIG. 7 , operators q(1), q(2), q(3), q(4) and r(1), r(2), r(3), and r(4) are in the same homogeneous partition, the type of the operator in the homogeneous partition is convolution operation, and the dimension of the output operand is 64×55×55. q(5), q(6), q(7) and r(5), r(6), r(7) are in the same homogeneous partition, and the type of operator in this homogeneous partition is convolution operation , the dimension of the output operand is 256×55×55. The operators q(n) and r(n) are in the same homogeneous partition, the type of the operators in the homogeneous partition is the pooling operation, and the dimension of the output operand is m×n×k. It should be noted that the types of operators in the homogeneous partition, the dimensions of output operands, the number and number of operators in the homogeneous partition in FIG. 7 are only examples of the present application and do not limit the present application in any way.

For step S3002, the first footprint matrix of the first computation graph and the second footprint matrix of the second computation graph are determined according to the first partition, mainly for placing operators located in the same homogeneous partition adjacent to each other in the footprint matrix row and column positions.

The operators corresponding to the rows in the same position of the first footprint matrix and the second footprint matrix are in the same first partition, and the operators corresponding to the columns in the same position are in the same first partition; The operators of the computation graph are arranged consecutively in corresponding rows or columns in the first footprint matrix, and the operators of the second computation graph belonging to the same first partition are arranged consecutively in corresponding rows or columns in the second footprint matrix. That is, when determining the first footprint matrix and the second footprint matrix, for operators located in the same homogeneous partition in the first computation graph and the second computation graph, the operators in the first computation graph are in the first footprint matrix The sequence number range of the row and column in is the same as the sequence number range of the row and column of the operator in the second footprint matrix in the second computation graph. For example, still taking the example shown in FIG. 7 as an example, it is assumed that the operators q(1), q(2), q(3), and q(4) of the first calculation graph correspond to the first footprint matrix in the first footprint matrix. Rows 1-4, columns 1-4, then the operators r(1), r(2), r(3), and r(4) of the second calculation graph correspond to rows 1-4 in the second footprint matrix , columns 1-4. In this way, it is more likely that there is a mapping relationship between operators corresponding to the same rows and columns of the first footprint matrix and the second footprint matrix. When adjusting the order of rows and columns in the first footprint matrix, by adjusting the first footprint The order of the rows and columns corresponding to the operators located in the same homogeneous partition in the footprint matrix can quickly adjust the operators with mapping relationships to the same rows and columns.

In the above-mentioned embodiments of the present application, the operators in the calculation graph are homogeneously partitioned, and the corresponding footprint matrix is determined according to the homogeneous partition, so that operators with a higher possibility of having a mapping relationship can be quickly determined, and the mapping can be accelerated. the process of.

In a possible implementation manner, adjusting the order of rows and columns corresponding to operators located in the same first partition in the first footprint matrix according to the first matrix may include:

Determine the mapping relationship between the operator in the first calculation graph and the operator in the second calculation graph in the same first partition according to the first matrix;

The order of rows and columns in the first footprint matrix corresponding to the operators in the first computation graph is adjusted according to the mapping relationship.

The mapping relationship between the operators in the first calculation graph and the operators in the second calculation graph in the same first partition is determined according to the first matrix, which may specifically include two traversals. The following process is for the same first partition. The process of determining the mapping relationship by the operator.

In the first traversal process: start from the operator in the first calculation graph corresponding to the smallest element in a homogeneous partition of the similarity matrix to traverse all the operators in the homogeneous partition, according to the current traversed operator in the similarity The largest element in the row or column of the degree matrix determines the operator most similar to the currently traversed operator in the second calculation graph, if the operator most similar to the currently traversed operator in the second calculation graph has been determined is the most similar operator of the operators that have been traversed, then in the second traversal, the most similar operator is re-determined for the operators that have been traversed, that is, in the process of the first traversal, after the traversal When the determined most similar operator conflicts with the previously determined most similar operator, the result of the later traversal overwrites the result of the previous traversal.

In the process of the second traversal: for the operator that has not yet matched the most similar operator in the homogeneous partition, start the traversal from the operator corresponding to the largest element in the homogeneous partition of the similarity matrix and have not yet matched. The operator of the most similar operator, according to the largest element in the row or column where the currently traversed operator is located in the similarity matrix, to determine the operator that is most similar to the currently traversed operator in the second calculation graph. If the operator most similar to the currently traversed operator in the calculation graph has been determined to be the most similar operator of the operator that has been traversed, then according to the order of the current traversed operator in the row or column of the similarity matrix The large element value determines the most similar operator to the currently traversed operator in the second calculation graph until the most similar operator to the currently traversed operator is matched.

FIG. 5b shows a flowchart of a method for adjusting the order of rows and columns of a footprint matrix according to an embodiment of the present application. Assume that in this example, the operator corresponding to the row of the first footprint matrix is the operator of the first calculation graph, the operator corresponding to the column is the operator of the second calculation graph, and the row of the similarity matrix is the same as the first calculation graph. The operators correspond, and the columns correspond to the operators of the second calculation graph. The process of adjusting the footprint matrix in this embodiment of the present application will be described with reference to the similarity matrix in the above example.

Fig. 5b shows the process of determining the mapping relationship of the operators for an operator of a homogeneous partition. As shown in Fig. 5b, in step S500, the processor can first determine whether it is the first traversal or the second traversal, if yes For the first time to traverse the operators in the first calculation graph, the processor may execute step S510 to first determine whether all operators of the homogeneous partition have been traversed, and if the operators of the homogeneous partition have been traversed, the processor may return Step S500, continue to judge whether it is the first traversal or the second traversal; if the operator of the homogeneous partition has not been traversed, the processor can perform step S511 to determine that the operator that has not been traversed corresponds to the similarity matrix. The smallest element in the row of , determines the operator corresponding to the smallest element as the current traversed operator, that is, in the first traversal process: start traversal from the operator corresponding to the smallest element value in the similarity matrix.

After determining the currently traversed operator, the processor may execute step S512, and determine the most similar operator to the currently traversed operator in the second calculation graph according to the largest element value in the row where the currently traversed operator is located in the similarity matrix For the operator, for example, the operator in the second calculation graph corresponding to the largest element value of the currently traversed operator in the row where the similarity matrix is located is determined as the most similar operator to the currently traversed operator. In other words, because it has been assumed that the operator corresponding to the row of the first footprint matrix is the operator of the first calculation graph, the operator corresponding to the column is the operator of the second calculation graph, and the row of the similarity matrix is the operator of the first calculation graph. The sub-correspondence, the column corresponds to the operator of the second calculation graph, therefore, in the similarity matrix, the value of a row of elements respectively represents the cosine of an operator in the first calculation graph and each operator in the second calculation graph Similarity, the larger the value in this row of elements, the more similar the operator in the first calculation graph is to the operator in the second calculation graph. Therefore, the processor may determine the operator in the second calculation graph corresponding to the largest element value of the currently traversed operator in the row of the similarity matrix as the operator most similar to the currently traversed operator.

After determining the most similar operator of the currently traversed operator, the processor may execute step S513 to determine whether the most similar operator has been matched with the previously traversed operator, and if the most similar operator does not match the previously traversed operator If the sub-matches, the processor may perform step S515 to record the mapping relationship between the currently traversed operator and the most similar operator, mark the currently traversed operator as the traversed operator, and return to step S510 to continue to determine whether All operators of the homogeneous partition have been traversed.

If the processor determines that the most similar operator matches the previously traversed operator, the processor may execute step S514 to delete the mapping relationship between the previously traversed operator and the most similar operator, and add the previously traversed operator to the The second traversal of the queue. Then proceed to step S515. For steps S514 and S515, the way that the processor records and deletes the mapping relationship is only an example of the present application, and the present application is not limited to this, and other methods can also be used to realize the most similar operator determined by traversal and the most similar operator determined before. When similar operators conflict, the result of the later traversal overwrites the result of the previous traversal. Similarly, adding the previously traversed operator to the queue for the second traversal is only a way to implement the second traversal. The processor can also implement this process by adding a special identifier to the operator. The implementation method is not limited.

After the operators of the homogeneous partition are traversed for the first time according to the above process, the process of the second traversal is started. The processor may execute step S520 to determine whether all operators of the homogeneous partition have been traversed, that is, whether the operators that have not been matched after the first traversal have been traversed. After judging that all operators of the homogeneous partition have been traversed, the processor may end the process of determining the mapping relationship. After judging all the operators of the homogeneous partition that have not traversed the operators, the processor may execute step S521 to determine the largest element in the row corresponding to the similarity matrix of the operator that has not yet been traversed, and calculate the operator corresponding to the largest element. child as the current traversal operator. That is, in the process of the second traversal described above, for the operators that have not yet matched the most similar operator, the traversal starts from the operator corresponding to the largest element in the similarity matrix.

After the currently traversed operator is determined, the processor may execute step S522 to determine the most similar operator to the currently traversed operator in the second calculation graph according to the maximum element value in the row where the currently traversed operator is located in the similarity matrix the operator. For the specific determination method, reference may be made to the content of step S512, and details are not repeated here. The processor may execute step S523 to determine whether the most similar operator has been matched with the previously traversed operator. For the specific process, please refer to the content of step S513. The difference is that when the processor determines that the most similar operator matches the previously traversed operator, it no longer covers the previously traversed result, but executes step S524, according to the row in the similarity matrix where the currently traversed operator is located. The value of the second largest element in the second calculation graph determines the operator that is most similar to the currently traversed operator, that is to say, continue to find other operators that are most similar to the currently traversed operator until it finds an operator that is not the same as the one traversed before. After the operator matches the operator that is "most similar" to the currently traversed operator, step S525 is executed.

In the process of performing the above two traversals on all the operators in the homogeneous partition, the processor can more quickly determine the local (stage) optimal mapping relationship of the operators in the first calculation graph and the second calculation graph, and according to The mapping relationship adjusts the order of the rows and columns in the first footprint matrix corresponding to the operators in the first computation graph. Specifically, according to the arrangement order of operators corresponding to each row (each column) of the second footprint matrix, the mapping operator of the operator in the second calculation graph in the first calculation graph is determined according to the above-mentioned mapping relationship, and the mapping The corresponding row (column) of the operator in the first footprint matrix is adjusted to be the same as the row (column) of the operator in the second footprint matrix in the second calculation graph.

According to the method for adjusting the first footprint matrix provided by the above embodiments of the present application, the calculation result can be more quickly converged to the stop condition of the iteration, and the mapping relationship between operators can be obtained more quickly, which is beneficial to speed up the mapping process. It should be noted that the above example is only an example of adjusting the order of the rows and columns of the first footprint matrix provided by the present application, and the present application is not limited thereto.

In a possible implementation manner, the homogeneous depths of the operators corresponding to the rows in the same position of the first footprint matrix and the second footprint matrix are the same, and the homogeneous depths of the operators corresponding to the columns of the same position are the same, And the corresponding rows or columns of the operators of the first computation graph belonging to the same first partition in the first footprint matrix are sorted according to the homogeneous depth of the operators, and the second computation graph of the same first partition The rows or columns corresponding to the operators in the second footprint matrix are sorted according to the homogeneity depth of the operators; wherein, the homogeneity depth is: in all branches to which the operator belongs in the calculation graph, in The maximum number of operators that have the same type and output tensor dimensions as the operator before the operator.

In a possible implementation manner, the operator mapping method provided by the present application may further include the following steps:

Determine the homogeneity depth of the operator according to the type of the operator, the dimension of the tensor output by the operator, and the order of the topological structure of the operator in the first computation graph or the second computation graph;

The operators are partitioned to obtain homogeneous partitions, and in each homogeneous partition, the operators of the first computational graph and the second computational graph are sorted and numbered according to the homogeneous depth of the operator; that is, in each In the homogenous partition, the processor may sort and number the operators of the first calculation graph according to the homogeneity depth of the operators of the first calculation graph, and sort and number the operators of the second calculation graph according to the homogeneity depth of the operators of the second calculation graph. The operators are sorted and numbered according to the homogeneous depth, and the method of sorting and numbering is not limited. The depth is sorted from deep to shallow, and the numbers are numbered from small to large.

In this way, when the processor determines the first footprint matrix, it can be determined according to the sequence of the numbers of the operators of the first calculation graph in the first partition; when the processor determines the second footprint matrix, it can be determined according to the second calculation in the first partition. The order of the numbering of the operators of the graph is determined. In this way, the operators corresponding to the rows in the same position of the first footprint matrix and the second footprint matrix can have the same homogeneous depth, the operators corresponding to the columns in the same position have the same homogeneous depth, and belong to the first calculation of the same first partition. The corresponding rows or columns of the operators of the graph in the first footprint matrix are sorted according to the homogeneous depth of the operators, and the corresponding rows or columns of the operators of the second calculation graph of the same first partition in the second footprint matrix are sorted according to the operator's homogeneous depth. Homogeneous depth ordering of .

Distinguishing operators with the same attributes by the homogeneous depth can avoid the unreliability of distinguishing operators with the same attributes by the names of the operators, and numbering and sorting homogeneous operators according to the homogeneous depth is also conducive to speeding up the mapping process, because it is more likely that there is a mapping relationship between operators that belong to the same first region and have the same homogeneous depth.

Due to the subjectivity or lack of user naming of operators, if the operators with the same attributes are distinguished according to the names of the operators named by the user, there is a problem that it may not be well differentiated (renamed or missing names) case), unreliable, and another problem is the possibility of not reflecting the mapping relationship between homogeneous operators. In the embodiments provided in this application, the above two problems can be well solved by distinguishing homogeneous operators with the same properties by homogeneous depth.

FIG. 8 and FIG. 9 respectively show schematic diagrams of determining the homogeneity depth of a homogeneity operator according to an embodiment of the present application. In the calculation diagram shown in Figure 8, there are four homogenous operators Conv, the homogeneity depth of the first operator Conv is 1, and the homogeneity depth of the second Conv operator is 2, because from this operator The first homogenous operator (the first Conv operator) starts to the operator (the second Conv operator), and the second Conv operator has a total of two homogenous operators (one is the first Conv operator, one is itself), therefore, the homogeneity depth of the second Conv operator is 2. Similarly, the homogeneity depths of other operators in the computation graph shown in FIG. 8 can be determined.

As shown in Figure 9, the fourth Conv operator from top to bottom (the operator filled with patterns in Figure 9) belongs to two branches. In two different branches, the corresponding homogeneous depth of the operator are different, for example, there is a Conv operator on the left branch before the 4th Conv operator, the homogeneity depth of the 4th Conv operator is 2, and on the right branch before the 4th Conv operator There are two Conv operators, and the fourth Conv operator has a homogeneity depth of 3. At this time, from the first homogenous operator of the operator to the operator, the maximum number of homogenous operators of the operator is 3, so the homogeneity depth of the fourth Conv operator is 3.

After determining the homogeneity operators and the homogeneity depths of the homogeneity operators in the first calculation graph and the second calculation graph, the homogeneity operators can be partitioned to obtain homogeneity partitions, and in the homogeneity partitions, according to the operator The homogeneity depth of , sorts and numbers the operators of the two computation graphs respectively.

In the above-mentioned embodiments of the present application, the homogeneous depth is used as an example of the representation of the topology structure of the operator in the calculation graph. The present application is not limited to this, and other ways can also be used to represent the topology structure of the operator in the calculation graph. .

According to the above-mentioned embodiments of the present application, the method of sorting and numbering by homogeneous depth can be used to distinguish different homogeneous operators well, and the mapping relationship between homogeneous operators can be obtained more quickly, which is conducive to speeding up the mapping process .

In a possible implementation manner, if the number of operators of the first computation graph in the first partition is different from the number of operators of the second computation graph, the first partition further includes Pseudo-operator; wherein, a pseudo-operator is an operator without operator type and operand. If the number of operators of the first computation graph in the second partition is different from the number of operators of the second computation graph, the second partition also includes the pseudo-operator.

In a possible implementation manner, if the number of operators of the first computation graph in the first partition is less than the number of operators of the second computation graph in the first partition, the The row or column corresponding to the pseudo operator in the first footprint matrix is located after the row or column corresponding to the operator belonging to the first partition in the first calculation graph; The number of operators of the second calculation graph is less than the number of operators of the first calculation graph in the first partition, and the corresponding row or column of the pseudo operator in the second footprint matrix is located in After the row or column corresponding to the operator belonging to the first partition in the second calculation graph.

In a possible implementation manner, if the number of operators of the first computation graph in the second partition is less than the number of operators of the second computation graph in the second partition, the The rows and columns corresponding to the pseudo operators in the first footprint matrix and the rows and columns corresponding to the operators belonging to the second partition in the first calculation graph are randomly sorted, and the rows and columns are randomly sorted in the same manner; If the number of operators of the second computation graph in the second partition is less than the number of operators of the second computation graph in the second partition, the pseudo operator is located in the second footprint The corresponding rows and columns in the matrix and the rows and columns corresponding to the operators belonging to the second partition in the second computation graph are randomly ordered, and the random ordering of the rows and columns is the same.

Since the model is transferred between different deep learning frameworks, the number of operators in the computation graph corresponding to the model may be different. When the similarity calculation is performed, the length of the feature vector is the same to be calculated. Therefore, when the dimensions of the first footprint matrix of the first computation graph and the second footprint matrix of the second computation graph are the same, the eigenvectors of the operators of the first computation graph determined according to the first footprint matrix and the eigenvectors of the operators of the first computation graph determined according to the second footprint matrix The lengths of the eigenvectors of the operators of the second computational graph are the same. In order to ensure that the footprints of the first footprint matrix of the first computation graph and the second footprint matrix of the second computation graph are the same, in the embodiment of the operator mapping method provided by the present application, it may further include:

If the number of operators in the first computation graph is different from the number of operators in the second computation graph in the homogeneous partition, a pseudo operator is added to the computation graph with a small number of operators in the homogeneous partition, so that the same The number of operators in the first computational graph and the second computational graph in the prime partition is the same;

If the number of operators in the first computation graph is different from the number of operators in the second computation graph in the mixed partition, a pseudo-operator is added to the computation graph with a small number of operators in the mixed partition, so that the mixed partition The number of operators of the first computational graph and the second computational graph is the same.

When adding pseudo-operators in homogeneous and mixed partitions, the pseudo-operators can be randomly sorted and numbered. In a possible implementation manner, when a pseudo operator is added to the homogeneous partition, the pseudo operator may be arranged after the operator of the calculation graph. In hybrid partitioning, both pseudo-operators and operators of the computation graph can be randomly ordered and numbered according to the ordering.

Pseudo-operators are fictitious fillers that do not have any meaningful operator properties or topology. The purpose of pseudo-operators is to make the total number of operators in the first computational graph and the second computational graph equal, so as to construct the same size. the footprint matrix, so that the subsequent matrix operations can be performed normally. In this way, the operator mapping method provided by the present application can be applied to the scenarios of frameworks or migration models between systems using different computational graph representations.

It should be noted that the above sequence of determining the homogeneous depth, partitioning, adding pseudo-operators, sorting and numbering is only an example of the present application, and the present application is not limited to the above sequence. For example, the processor may also perform the above process in the following order: the processor may determine the homogeneity depth of the operators in the first and second computation graphs; Partitioning is performed to obtain homogeneous partitions and/or mixed partitions. At the same time, the operators of the first computational graph and the second computational graph can be sorted according to the homogeneous depth of the operators in the homogeneous partition, and the sorting is based on the homogeneity. The depth can be from large to small or from small to large, which is not limited in this application; the processor adds pseudo-operators according to the number of operators in the two calculation graphs in the homogeneous partition, and the processor can The homogeneous depth is set to the maximum (true operator + total number of pseudo-operators). For homogeneous partitions, pseudo-operators can be sorted according to the homogeneous depth, and for mixed partitions, pseudo-operators can be randomly sorted; The operators are numbered.

FIG. 10 shows an example of an insertion pseudo-operator according to an embodiment of the present application. As shown in Figure 10, in the first homogeneous partition, two pseudo-operators q(3) and q(4) are added to the left computational graph, and in the second homogeneous partition, the right A pseudo-operator r(7) is added to the computational graph, and in the mixed partition, two pseudo-operators are added to the computational graph on the left. After the pseudo-operator is added, the number of operators in the first computation graph and the second computation graph is the same.

When determining the first footprint matrix of the first computation graph and the second footprint matrix of the second computation graph, the corresponding rows and columns of the two pseudo-operators q(3) and q(4) in the first footprint matrix are calculated. After the sub-q(1), q(2), the corresponding row and column of the pseudo-operator r(7) in the second footprint matrix are after the operator r(6). In addition, the pseudo operator is unreachable to other operators, that is, the shortest distance from the pseudo operator to other operators is infinite, and the reciprocal of the shortest distance is 0, that is, in the footprint matrix, the pseudo operator corresponds to The elements of are all 0.

Application example

The operator mapping method of the computation graph provided by the present application will be described below with reference to specific application scenarios and examples.

Application scenario and example description: Migrate the ResNet50 model from TensorFlow to MindSpore, and assist in locating the problematic script during the migration process. The number of operators in the calculation graph of the ResNet50 model on the TensorFlow v1.14 platform is 5176, and the number of operators in the calculation graph on the MindSpore v1.0.0 platform is 2513. In this example, denote the computation graph of TensorFlow as R and the computation graph of MindSpore as Q. The operator mapping method of the present application will be described below with reference to the application scenario and the example.

FIG. 11 shows a flowchart of an operator mapping method according to an embodiment of the present application. Combined with the example shown in Figure 2, after exporting the computational graph R from the model script of TensorFlow v1.14 and exporting the computational graph Q from the model script of MindSpore v1.0.0, input the computational graph to the processor. The processor may execute the operator mapping method shown in FIG. 11 .

The processor may compute the homogenous depth for the operators in the computation graph Q and the computation graph R.

Still taking the examples shown in Figure 8 and Figure 9 as an example, as shown in Figure 8, assuming that the dimensions of the tensors output by all operators in Figure 8 are the same, using each operator as a starting point, go to the edge Traverse upwards in the direction to find how many operators are homogeneous (with the same attributes) as the operator at the starting point, and add one to the number of homogeneous operators to obtain the homogeneity depth of the operator at the starting point, as shown in Figure 8 at the end A Conv operator is preceded by three Conv operators, and the last Conv operator has a homogeneity depth of 4. In the example shown in Figure 9, there are multiple branches. The fourth Conv operator (the operator filled with patterns in Figure 9) from top to bottom belongs to two branches. In two different branches, The homogeneity depths corresponding to the operators are different. For example, there is a Conv operator on the left branch before the fourth Conv operator, the homogeneity depth of the fourth Conv operator is 2, and on the right branch There are two Conv operators before the fourth Conv operator, and the homogeneity depth of the fourth Conv operator is 3. At this time, from the first homogenous operator of the operator to the operator, the maximum number of homogenous operators of the operator is 3, so the homogeneity depth of the fourth Conv operator is 3.

After calculating the homogeneous depth of the operators in the computation graph, the processor may perform homogeneous partitioning on the operators in the computation graph Q and the computation graph R. An example after the processor performs homogeneous partitioning can refer to FIG. 10 .

The processor may divide the operators in the computation graph Q and the computation graph R with the same type and the same dimension size of the output operands of the operators into the same homogeneous partition. For the operators that do not have homogeneous operators in the computational graph Q and the computational graph R, a mixed partition can be set, and for the operators without homogeneous operators, they can be divided into mixed partitions. For example, MS's tuple_getitem operator and TF's Identity operator cannot find corresponding operator types with the same functions from each other's framework. For this type of operator, it can be divided into mixed partitions.

After homogeneous partitioning, the two computation graphs have the same number of homogeneous partitions and one mixed partition, each homogeneous partition is different from the other homogeneous partitions in the type of operator and/or the dimension of the output operand, homogeneous In a partition, the order in which the different homogeneous partitions are arranged is not important. If in the same homogeneous partition, the number of operators in the two computation graphs is not the same, the processor can add a pseudo-operator to the computation graph with a relatively small number of operators in the homogeneous partition, so that the two computation graphs in the homogeneous partition have different numbers of operators. The number of operators in each computation graph is the same. Pseudo-operators are fictitious fillers that do not have any meaningful operator attributes or topology. The purpose of pseudo-operators is to make the total number of operators in the computational graph Q and the computational graph R equal, so as to construct footprints of the same size. matrix to enable subsequent matrix operations to be performed.

After the homogeneous partition is completed, the processor can sort and number the operators according to the predetermined order. In a homogeneous partition, the processor can sort and number the operators of the two computation graphs in the order of the homogeneous depth from small to large. . The pseudo operators added in the homogeneous partition are arranged after the real operators in the calculation graph, and the pseudo operators in the mixed partition can be arranged after the operators in the calculation graph, or the pseudo operators in the mixed partition and the operators in the calculation graph can be combined. The children are randomly sorted, which is not limited in this application.

After completing the homogeneous partition, the processor may construct the first footprint matrix of the first computation graph and the second footprint matrix of the second computation graph according to the sequence of the operator numbers in the homogeneous partition. After the homogeneous partition is completed, the total number of operators in both the computational graph Q and the computational graph R is 5176. The processor can construct two footprint matrices of 5176 × 5176 in the order of operator numbers, and combine all elements of the two footprint matrices. Values are initialized to 0.

Compute the footprint matrix q of the graph Q, all initialized to 0:

	q1	q2	q3	…	…	q5176
q1	0	…	…	…	…	0
q2	…	…	…	…	…	…
q3	…	…	…	…	…	…
…	…	…	…	…	…	…
…	…	…	…	…	…	…
q5176	0	…	…	…	…	0

Calculate the footprint matrix r of the graph Q, all initialized to 0:

	r1	r2	r3	…	…	r5176
r1	0	…	…	…	…	0
r2	…	…	…	…	…	…
r3	…	…	…	…	…	…
…	…	…	…	…	…	…
…	…	…	…	…	…	…
r5176	0	…	…	…	…	0

The footprint matrix of the computation graph Q and the computation graph R stores the reciprocal of the shortest reachable distance from the operator to the operator in the corresponding computation graph. Since the computation graph Q and the computation graph R are both directed graphs, some operators are are not reachable, so the footprint matrix can be asymmetric.

The processor can traverse the operators in the computation graph Q and the computation graph R respectively, find out the shortest distance from the operator to other operators, and determine the footprint matrix according to the shortest distance. Taking the computational graph Q as an example, the processor can use each operator in the computational graph Q as a starting point to perform a depth-first traversal that can repeatedly access the operators. The value (the inverse of the visit distance) is placed in the corresponding element of the column where the origin operator is located and the row where the visited operator is located in the footprint matrix. If the visited operator has already been visited in this traversal and has been granted a larger or equal footprint value, the processor may not update the high corresponding element in the footprint matrix, and will no longer visit the visited operator's Direct downstream operator (already visited). The associated footprint value of all pseudo-operators is 0.

FIG. 12 shows a schematic diagram of determining the shortest distance according to an embodiment of the present application. As shown in Figure 12, traverse other operators from the starting point operator to determine the shortest distance from the starting point operator to other operators and the corresponding footprint value. The distance from the start operator to the previous operator is infinite, so the footprint value is 0. The shortest distance from the origin operator to the operator corresponding to the two connected child nodes is 1, and the corresponding footprint value is 1. The last operator in Figure 12 is a pseudo-operator, so the footprint value from the starting point operator to the pseudo-operator is 0. According to the above process, the footprint values from the starting point operator to other operators in Fig. 12 are determined one by one. The footprint matrix corresponding to the calculation graph shown in FIG. 12 can be determined according to the footprint value.

The operators of the first calculation graph and the second calculation graph are homogeneously partitioned, the homogeneous depth of the operators is determined, and the operators are sorted and numbered according to the homogeneous depth of the operators in the homogeneous partition. The order of the homogeneity depths of the operators in the partition determines the first footprint matrix of the first computation graph and the second footprint matrix of the second computation graph, respectively. In this way, the functions of the operators corresponding to the rows and columns in the same position of the first footprint matrix and the second footprint matrix are the same or similar, and the first footprint matrix indicates the operators of the first calculation graph and other operators in the first calculation graph. The topological relationship between operators, the second footprint matrix indicates the topological relationship between the operators of the second calculation graph and other operators in the second calculation graph.

After determining the footprint matrices corresponding to the computation graph Q and the computation graph R, the processor may calculate the similarity matrix according to the footprint matrices corresponding to the computation graph Q and the computation graph R.

In one example, the cosine similarity of the operators in the computation graph Q and the computation graph R can be calculated according to the footprint matrix q and the footprint matrix r to measure the similarity between two operators in the two computation graphs. The specific process is: determining the first eigenvector of the operator in the calculation graph Q according to the footprint matrix q, determining the second eigenvector of the operator in the calculation graph R according to the footprint matrix r; according to the first eigenvector and the second eigenvector The vector computation computes the similarity matrix between the operators in the computation graph Q and the operators in the computation graph R.

Taking an operator qk in the calculation graph Q as an example, where k is an integer between 1 and 5176, the eigenvector of the operator qk can be expressed as:

Among them, col(qk) is the corresponding column vector in the footprint matrix to which the operator qk belongs, row(qk) is the corresponding row vector in the footprint matrix to which the operator qk belongs, and concat is the splicing operator. The row vector of the row in the footprint matrix is spliced with the column vector of the column to obtain the eigenvector of the operator

According to the determined footprint matrix q and footprint matrix r, the eigenvector matrix of the computational graph Q and the eigenvector matrix of the computational graph R can be obtained.

The eigenvector matrix of the calculation graph Q is shown in the following table. The part above the thick solid line can be the column vector corresponding to the operator in the calculation graph Q, and the part below the thick solid line can be the row vector corresponding to the operator in the calculation graph Q. Obtained below the column vector, the eigenvector matrix of the computational graph Q is 10352×5176:

q1	q2	q3	…	…	q5176
…	…	…	…	…	…
…	…	…	…	…	…
…	…	…	…	…	…
…	…	…	…	…	…
…	…	…	…	…	…
…	…	…	…	…	…
…	…	…	…	…	…
…	…	…	…	…	…

Similarly, the eigenvector matrix of the calculation graph R is shown in the following table, the part above the thick solid line can be the column vector corresponding to the operator in the calculation graph R, and the part below the thick solid line can be the row vector corresponding to the operator in the calculation graph R After splicing it under the column vector, the eigenvector matrix of the calculation graph R is 10352×5176:

r1	r2	r3	…	…	r5176
…	…	…	…	…	…
…	…	…	…	…	…
…	…	…	…	…	…
…	…	…	…	…	…
…	…	…	…	…	…
…	…	…	…	…	…
…	…	…	…	…	…
…	…	…	…	…	…

Since the first footprint matrix indicates the topological relationship between the operators of the first computation graph and other operators in the first computation graph, the second footprint matrix indicates the operators of the second computation graph and other operators in the second computation graph The topological relationship between the operators, therefore, the eigenvector of the operator obtained by the method of extracting the eigenvector of the operator provided by this application can well reflect the topological relationship between an operator and other operators in the calculation graph, That is, the "feature" of the operator in the calculation graph where it is located.

The value of the cosine similarity of any two non-zero eigenvectors is a real number between 0 and 1. The closer to 1, the more similar the two eigenvectors are, and the more similar the two operators corresponding to the eigenvectors are. The similarity between a zero eigenvector and an arbitrary eigenvector is defined as 0, i.e. not at all similar.

According to the eigenvectors of the operators determined above, the similarity of any two operators can be expressed as:

Among them, k and l are both integers between 1 and 5176,

represent the eigenvectors of operators qk and rl, respectively.

In the embodiment of the present application, the positions of the rows and columns of the footprint matrix are equivalent to the mapping relationship of the operators. Therefore, when calculating the similarity matrix, the processor calculates all the operators of the calculation graph Q and the calculation R The similarity between the obtained similarity matrix is as follows:

The objective function can be calculated according to the similarity matrix

value of .

After that, the processor can update the footprint matrix according to the process of Figure 5b. Since the corresponding row and column positions of the operators in the footprint matrix are equivalent to the mapping relationship between the operators, the footprint can be updated after finding the mapping relationship of the optimal operator in the stage. matrix, and then continue to calculate the similarity matrix. When updating the footprint matrix, you can fix the row and column position of one of the two footprint matrices unchanged, and adjust the row and column position of the other footprint matrix. The adjusted footprint matrix can be either the footprint matrix of the computational graph Q or the footprint matrix of the computational graph R. In a possible implementation manner, the footprint matrix of the computation graph with fewer real operators can be selected for adjustment, so as to reduce the number of adjustments. For example, in this example, the footprint matrix of the computation graph Q can be selected for adjustment.

FIG. 13 shows a schematic diagram of adjusting the footprint matrix according to an embodiment of the present application. As shown in Figure 13, the processor can adjust row 7 to the position of row 1, row 9 to the position of row 2, row 2 to the position of the last row... At the same time, the 7th column Adjust to the position of column 1, adjust the position of column 9 to the position of column 2, and adjust the position of column 2 to the position of the last column.

According to the footprint matrix of the adjusted calculation graph Q and the footprint matrix of the calculation graph R, the similarity matrix can be continuously calculated, and the value of the objective function can be calculated.

Then, the processor can determine whether a stop condition is reached, and the stop condition may refer to a condition satisfied by the similarity matrix (objective function value) obtained by multiple calculations, or the stop condition may also be the number of iterations, which is not limited in this application .

When the stop condition is not reached, the processor may continue the process of updating the footprint matrix, calculating the similarity according to the updated footprint matrix, and judging whether the stop condition is reached, until the stop condition is reached. The processor may determine the mapping relationship between the operators according to the footprint matrix when the stopping condition is reached.

For a homogeneous partition, one of the two operators corresponding to the same row (or column) in the two footprint matrices can be directly determined as the mapping operator of the other operator. For example, when the stopping condition is reached, the footprint matrix of the adjusted computational graph Q is:

	q3	q2	q1	…	…	q5176
q3	0	…	…	…	…	0
q2	…	…	…	…	…	…
q1	…	…	…	…	…	…
…	…	…	…	…	…	…
…	…	…	…	…	…	…
q5176	0	…	…	…	…	0

The mapping operator of operator q3 is r1, the mapping operator of operator q2 is r2, the mapping operator of operator q1 is r3... . That is to say, the two operators corresponding to the same row (or column) of the two footprint matrices are each other's mapping operators.

The processor can output a one-to-one mapping relationship between operators in the homogeneous partition according to the two footprint matrices.

For the mixed partition operator, there is no mapping operator. The processor may output a mapping suggestion of the operators in the mixed partition according to the similarity matrix calculated by the previous iterations. Specifically, the processor may sort according to the size of the cosine similarity between the mixed region operator and another operator in the similarity matrix. , determine the top-ranked one or more (for example, 5) operators as the mapping suggestion of the mixed area operator, and output the corresponding mapping operator and the value of the cosine similarity as the mapping suggestion.

Taking the Identity class operator of TF as an example, in the process of adjusting the footprint matrix of the computational graph Q and calculating the similarity matrix, the cosine similarity between the Identity class operator and other operators in the mixed region is calculated. Sort the calculated cosine similarities in descending order, and take the first five operators in the sorting and the corresponding cosine similarity values as the mapping suggestions for the Identity class operators for output.

In a possible implementation manner, the present application also provides a parameter for measuring whether the mapping result is better: L1 norm. The footprint difference matrix D can be calculated according to the footprint matrix q and the footprint matrix r, and the L1 norm of the footprint difference matrix can be calculated to measure the similarity between the operators in the two calculation graphs.

The footprint difference matrix can be calculated according to the following formula:

D=q-r, where D represents the footprint difference matrix;

L1 norm=∑ _i ∑ _j |d _ij |, the smaller the L1 norm, the closer the two footprint matrices are, and the more similar the two computation graphs are in topology.

Adjust the first footprint matrix according to the order in which the rows and columns in the first footprint matrix are adjusted in the foregoing example, and continue to calculate the L1 norm according to the adjusted first footprint matrix and the second footprint matrix. By calculating the change of the L1 norm in the process of adjusting the footprint matrix, it can be judged whether the obtained mapping relationship can reflect the similarity of the two calculation graphs.

FIG. 14 shows a schematic diagram of the effect of iterative calculation according to an embodiment of the present application. As shown in Figure 14, as the number of iterations increases, the value of the objective function and the value of the L1 norm gradually converge, and the value of the objective function gradually becomes larger. After reaching a certain value, the change is no longer obvious. Similarly, The value of the L1 norm gradually becomes smaller, and after reaching a certain value, the change is no longer obvious.

Therefore, the stopping condition can be set by setting the threshold of the objective function or L1 norm, or the number of iterations. In the above application example of the present application, the number of iterations can be set to 5 or 6, which is enough to obtain a better mapping relationship.

The following table shows some parameters of the mapping effect in the example of migrating the ResNet50 model from TensorFlow to MindSpore. In this example, the operator mapping method provided by this application can complete the calculation within 90s, complete the one-to-one mapping of 99% of the backbone operators, and provide many-to-many mapping suggestions, which can give all operators One-to-one mapping of 40% of the operators.

The total number of MS v1.0.0 operators	2513
1 to 1 mapping number	994 (40%)
Many-to-many mapping number	All MS operators have Top-K mapping suggestions
* number of backbone operators	895
* Backbone operator 1-to-1 mapping number	888 (99%)
Map operation time	~90s

The operator mapping apparatus of the embodiments provided by the present application constructs footprint matrices of two computation graphs, where the footprint matrix is a matrix representing the topological relationship between operators in the computation graph. A first matrix indicating the topological relationship between the two computational graphs can be iteratively calculated according to the footprint matrix, and after the iteration is stopped, one or more preferred mapping relationships can be determined according to the footprint matrix. When constructing the first footprint matrix and the second footprint matrix, and adjusting the first footprint matrix, ensure the type of the operator corresponding to the row at the same position and the type of the operator corresponding to the column at the same position of the first footprint matrix and the second footprint matrix The same, the dimensions of the tensors output by the operators are the same, that is, the operators corresponding to the rows in the same position of the first footprint matrix and the second footprint matrix and the operators corresponding to the columns in the same position are functionally the same. Combine the similarity matrix to measure the similarity of the topological relationship between operators. According to the operator mapping device provided by the present application, the mapping relationship between two operators can be reflected in terms of functions and topological relationships, thereby ensuring the correctness of the mapping result.

For operators without homogeneous operators, the operator mapping method provided in this application can output mapping suggestions, and supports one-to-one, one-to-many and many-to-many mappings, and the operator mapping method provided in this application can support similar The operator mapping between the computational graphs of the structure can be applied to scenarios where the deep learning framework performs model migration.

Application scenario and example description: Migrate the ResNet50 model from MindSpore v0.5.0 to MindSpore v1.0.0. The number of operators in the calculation graph of the ResNet50 model on the MindSpore v0.5.0 platform is 2514, and the number of operators in the calculation graph on the MindSpore v1.0.0 platform is 2513. In this example, the calculation graph of MindSpore v0.5.0 is denoted as R, and the calculation graph of MindSpore v1.0.0 is denoted as Q.

For the process of calculating the homogeneous depth, refer to the above example and will not be repeated here.

Perform homogeneous partitioning. In this example, since the model is transferred between different versions of the same deep learning framework, all operators have homogeneous operators, and there is no mixed partition after partitioning. After the homogeneous partition is completed, the total number of operators in both the computational graph Q and the computational graph R is 2514.

The processor can construct two 2514×2514 footprint matrices in the order of operator numbers, and initialize the values of all elements of the two footprint matrices to 0. For the process of determining the footprint matrix, reference may also be made to the above example, and details are not repeated here.

The process of calculating the similarity and outputting the mapping relationship is also the same as the above example, and will not be repeated.

FIG. 15 shows a schematic diagram of the effect of iterative calculation according to an embodiment of the present application. As shown in Figure 15, with the increase of the number of iterations, the value of the objective function gradually converges, and the value of the objective function gradually increases. After reaching a certain value, the change is no longer obvious. The value of the L1 norm also no longer decreases after the 3rd iteration.

The following table shows some parameters of the mapping effect in the example of migrating the ResNet50 model from MindSpore v0.5.0 to MindSpore v1.0.0. In this example, the operator mapping method provided by the present application can complete the calculation within 25s, complete the one-to-one mapping of 100% of the operators, with fast mapping speed and high accuracy.

The total number of MS v1.0.0 operators	2513
1 to 1 mapping number	2513 (100%)
1 to 1 mapping accuracy	83%
* number of backbone operators	895
* Backbone operator 1-to-1 mapping number	895 (100%)
* 1-to-1 mapping accuracy rate of backbone operators	100%
Map operation time	~25s

The operator mapping method of the embodiment provided by the present application can quickly find the operator mapping relationship of two calculation graphs one-to-one, one-to-many and many-to-many without modifying the model script, thereby improving the comparison effect of the calculation graphs , so as to improve the test efficiency of operator developers and the speed of model network development and migration. It can be determined from the mapping statistics table that the one-to-one mapping rate and its correctness of the backbone operators that directly play a role in the problem location of the model are very high whether it is a comparison of the calculation graphs of different versions of the framework or the same framework. Reasonable time. With accurate backbone operator mappings, developers can use it as a reference to quickly compare operator numerical files or analyze model scripts to locate problems.

According to the above example, the operator mapping method provided by the embodiments of the present application can quickly realize the mapping of operators in two calculation graphs, and can obtain one or more preferred mapping relationships through iterative calculation, so it can support Operator mapping between computational graphs with similar structures, and can also support one-to-many or many-to-many mapping, which can be applied to scenarios where deep learning frameworks perform model migration.

An embodiment of the present application provides an operator mapping apparatus for a computation graph, which is used to perform operator mapping on operators in a first computation graph and a second computation graph. FIG. 16 shows the computation according to an embodiment of the present application. The block diagram of the operator mapping apparatus 90 shown in FIG. 16 , the operator mapping apparatus 90 may include:

The iteration module 91 is configured to, based on the first footprint matrix and the second footprint matrix, repeat the following process until a stopping condition is reached: calculate the first matrix according to the first footprint matrix and the second footprint matrix, and calculate the first matrix according to the The first matrix adjusts the order of rows and columns in the first footprint matrix, and recalculates the first matrix according to the adjusted first footprint matrix and the second footprint matrix; wherein the first footprint matrix indicates The topological relationship between the operators of the first computational graph, the second footprint matrix indicates the topological relationship between the operators of the second computational graph, the first matrix indicates the first computational graph and the similarity of the topological relationship of the second calculation graph;

The corresponding module 92 is configured to obtain the operators corresponding to each of the multiple operators in the first calculation graph in the second calculation graph according to the first footprint matrix and the second footprint matrix when the stopping condition is reached .

In a possible implementation manner, the first calculation that belongs to the same first partition and has the same row or column position in the first footprint matrix and the second footprint matrix when the stopping condition is reached is the same There is a mapping relationship between the operators of the graph and the operators of the second computation graph.

The operator mapping apparatus of the embodiments provided by the present application constructs footprint matrices of two computation graphs, where the footprint matrix is a matrix representing the topological relationship between operators in the computation graph. A first matrix indicating the similarity of the topological relationship of the two computational graphs can be iteratively calculated according to the footprint matrix, and after the iteration is stopped, one or more preferred mapping relationships can be determined according to the footprint matrix. Therefore, the operator mapping method provided by the present application can support operator mapping between calculation graphs with similar structures, and can be applied to a scenario where a deep learning framework performs model migration.

In a possible implementation, the iteration module 91 includes:

The calculation unit is configured to obtain the first matrix according to a plurality of first eigenvectors and a plurality of second eigenvectors, wherein each of the first eigenvectors corresponds to an operator in the first calculation graph, and each Each of the first eigenvectors and the operators corresponding to the first eigenvectors are related to the row vectors and column vectors in the first footprint matrix, and each of the second eigenvectors corresponds to the second calculation graph The operators in , each of the second eigenvectors and the operators corresponding to the second eigenvectors are related to the row vectors and column vectors in the second footprint matrix, and the first eigenvectors and the The second feature vector is used to calculate the similarity between the operator corresponding to the first feature vector and the operator corresponding to the second feature vector.

In a possible implementation manner, the value of the element of the first matrix represents the similarity between the operator corresponding to the row where the element is located and the operator corresponding to the column where the element is located, wherein the row corresponding to the element is located. The operator and the operator corresponding to the column where the element is located are respectively the operator in the first calculation graph and the operator in the second calculation graph.

The eigenvectors of the operator are constructed by using the corresponding row and column vectors in the footprint matrix corresponding to the operator in the calculation graph, and the topological relationship characteristics of the operator in the calculation graph are extracted. The similarity of the topological relationship of the operator is calculated according to the eigenvector of the operator, and the similarity of the topological relationship of the operator of the first calculation graph and the operator of the second calculation graph forms a similarity matrix, and the similarity matrix is used to measure the two calculations. The similarity of the topological relationships of the graphs.

In a possible implementation manner, the type of the first footprint matrix, the adjusted first footprint matrix, and the operator corresponding to the row at the same position and the type of the operator corresponding to the column at the same position of the second footprint matrix The same, the dimensions of the tensors output by the operators are the same, that is, the operators corresponding to the rows at the same position of the first footprint matrix and the second footprint matrix and the operators corresponding to the columns at the same position are of the same type , the dimensions of the tensors output by the operators are the same, the operators corresponding to the rows at the same position and the columns at the same positions of the adjusted first footprint matrix and the second footprint matrix are of the same type, and the operators output The dimensions of the tensors are the same.

When constructing the first footprint matrix and the second footprint matrix, and adjusting the first footprint matrix, ensure the type of the operator corresponding to the row at the same position and the type of the operator corresponding to the column at the same position of the first footprint matrix and the second footprint matrix The same, the dimensions of the tensors output by the operators are the same, that is, the operators corresponding to the rows in the same position of the first footprint matrix and the second footprint matrix and the operators corresponding to the columns in the same position are functionally the same. Combine the similarity matrix to measure the similarity of the topological relationship between operators. According to the operator mapping method provided by the present application, the mapping relationship between two operators can be reflected in terms of functions and topological relationships, thereby ensuring the correctness of the mapping result.

In a possible implementation manner, the operators in the first calculation graph and the second calculation graph belong to multiple first partitions, the operators in the same first partition are of the same type, and the operator outputs The dimensions of the quantities are the same; some operators in the first calculation graph and the second calculation graph belong to the second partition, the types of operators in the second partition are different and/or the dimensions of the tensors output by the operators are different different. The operators corresponding to the rows in the same position of the first footprint matrix and the second footprint matrix are in the same first partition, and the operators corresponding to the columns in the same position are in the same first partition; and, belong to the same first partition. The operators of the first computation graph are arranged in consecutive rows or columns in the first footprint matrix, and the operators of the second computation graph belonging to the same first partition are arranged in consecutive rows or columns in the second footprint matrix arrangement.

Because it is more likely that there is a mapping relationship between homogeneous operators, and in the embodiment of the present application, there is a mapping relationship between the operators corresponding to the rows or columns in the same position of the first footprint matrix and the second footprint matrix . Therefore, according to the above-mentioned embodiment provided by the present application, the operators in the first calculation graph and the second calculation graph are homogeneously partitioned, and when adjusting the order of the rows and columns in the first footprint matrix, adjust the The order of the rows and columns corresponding to the operators of the homogeneous partition can make the iterative process converge faster and speed up the mapping process.

For operators without homogeneous operators, the corresponding operator similarity can be obtained by setting mixed partitions for classification and participating in the iterative calculation process. When determining the mapping operator, mapping suggestions can be output according to the calculated similarity. Therefore, it can support operator mapping between computational graphs with similar structures, and can also support one-to-many or many-to-many mapping, which can be applied to scenarios where deep learning frameworks perform model migration.

In a possible implementation manner, the homogeneous depths of the operators corresponding to the rows in the same position of the first footprint matrix and the second footprint matrix are the same, and the homogeneous depths of the operators corresponding to the columns of the same position are the same, And the corresponding rows or columns of the operators of the first computation graph belonging to the same first partition in the first footprint matrix are sorted according to the homogeneous depth of the operators, and the second computation graph of the same first partition The rows or columns corresponding to the operators in the second footprint matrix are sorted according to the homogeneity depth of the operators; wherein, the homogeneity depth is: in all branches to which the operator belongs in the calculation graph, in The maximum number of operators that have the same type and output tensor dimensions as the operator before the operator.

The operators in the homogeneous partition are sorted according to the homogeneous depth, and the first footprint matrix of the first computation graph and the second footprint matrix of the second computation graph are respectively determined according to the homogeneous partition. Since it is more likely that there is a mapping relationship between operators with the same homogeneous depth, the process of calculating the similarity matrix according to the above-mentioned embodiments of the present application can quickly converge, and the mapping relationship between operators can be determined more quickly. the process of mapping. In addition, by distinguishing operators with the same attributes by the homogeneous depth, the unreliability of distinguishing operators with the same attributes by the names of the operators can be avoided.

In a possible implementation manner, if the number of operators of the first computation graph in the first partition is different from the number of operators of the second computation graph, the first partition further includes Pseudo-operator; wherein, the pseudo-operator is an operator without operator type and operand; if the number of operators in the first computation graph in the first partition is less than the number of operators in the first partition The number of operators in the second calculation graph, the row or column corresponding to the pseudo-operator in the first footprint matrix is located in the row corresponding to the operator belonging to the first partition in the first calculation graph or column; if the number of operators in the second computation graph in the first partition is less than the number of operators in the first computation graph in the first partition, the pseudo operator is The corresponding row or column in the second footprint matrix is located after the row or column corresponding to the operator belonging to the first partition in the second calculation graph.

In a possible implementation manner, if the number of operators of the first computation graph in the second partition is different from the number of operators of the second computation graph, the second partition further includes the pseudo operator; if the number of operators of the first computation graph in the second partition is less than the number of operators of the second computation graph in the second partition, the pseudo operator The rows and columns corresponding to the first footprint matrix and the rows and columns corresponding to the operators belonging to the second partition in the first computation graph are randomly ordered, and the rows and columns are randomly ordered in the same manner; if the The number of operators of the second computation graph in the second partition is less than the number of operators of the second computation graph in the second partition, and the pseudo-operators correspond to the second footprint matrix The rows and columns of , and the rows and columns corresponding to the operators belonging to the second partition in the second calculation graph are randomly ordered, and the random ordering of the rows and columns is the same.

When the number of operators in the first computation graph and the second computation graph is different, the total number of operators in the first computation graph and the second computation graph is equal by adding pseudo-operators, thereby constructing footprint matrices of the same size, so that Subsequent matrix operations proceed normally. In this way, the operator mapping method provided by the present application can be applied to the scenarios of frameworks or migration models between systems using different computational graph representations.

In a possible implementation manner, the iteration module 91 further includes:

An adjustment unit, configured to adjust the order of the rows and columns corresponding to the operators located in the same first partition in the first footprint matrix according to the first matrix.

In a possible implementation manner, the first calculation that belongs to the same first partition and has the same row or column position in the first footprint matrix and the second footprint matrix when the stopping condition is reached is the same There is a mapping relationship between the operators of the graph and the operators of the second computation graph; there is a mapping relationship between the operators of the first computation graph belonging to the second partition and multiple operators of the second computation graph belonging to the second partition A mapping relationship, wherein the multiple operators of the second computation graph belonging to the second partition are: in the first matrix when the stopping condition is reached, the number of operators in the first computation graph belonging to the second partition is Among the values of the elements corresponding to the operator, the operator of the second calculation graph corresponding to the largest multiple values.

According to the mapping method of the above-mentioned embodiments of the present application, it is possible to support operator mapping between computational graphs with similar structures, support one-to-one, one-to-many, and many-to-many mappings, and can be applied to scenarios where deep learning frameworks perform model migration middle.

In a possible implementation manner, the stopping condition is that the number of times of repeatedly calculating the first matrix is greater than a first threshold.

By performing homogeneous partitioning on the operators in the first computational graph and the second computational graph, and when adjusting the order of the rows and columns in the first footprint matrix, adjust the rows and columns corresponding to the operators located in the same homogeneous partition order can make the iterative process converge faster and speed up the mapping process. This is because the similarity of homogenous operators is higher, and there is a greater possibility of a mapping relationship between homogenous operators. Therefore, the process of homogenous partitioning can accelerate the speed of iterative convergence and the process of mapping.

An embodiment of the present application provides an operator mapping apparatus for a computational graph, including: a processor and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions .

FIG. 17 shows a block diagram of an operator mapping apparatus according to an embodiment of the present application. As shown in FIG. 17 , the operator mapping apparatus 1700 may vary greatly due to different configurations or performances, and may include one or more central processing units (CPU) 1701 (eg, one or more processors) ) and a memory 1705 in which one or more applications or data are stored.

Among them, the memory 1705 may be volatile storage or persistent storage. The programs stored in memory 1705 may include one or more modules, each of which may include a series of instructions to operate on a blockchain node. Further, the central processing unit 1701 may be configured to communicate with the memory 1705 to execute a series of instruction operations in the memory 1705 on the operator mapping apparatus 1700.

The operator mapping device 1700 may also include one or more power supplies 1702, one or more wired or wireless network interfaces 1703, one or more input and output interfaces 1704, and/or, one or more operating systems, such as Windows Server™, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

The process performed by the central processing unit 1701 in the operator mapping apparatus 1700 in this embodiment is similar to the method process described in the embodiment shown in FIG. 3 , FIG. 5 a , FIG. 5 b , FIG. 6 or FIG. Repeat.

Embodiments of the present application provide a non-volatile computer-readable storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, implement the above method.

Embodiments of the present application provide a computer program product, including computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are stored in a processor of an electronic device When running in the electronic device, the processor in the electronic device executes the above method.

A computer-readable storage medium may be a tangible device that can hold and store instructions for use by the instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read-only memory (Electrically Programmable Read-Only-Memory, EPROM or flash memory), static random access memory (Static Random-Access Memory, SRAM), portable compact disk read-only memory (Compact Disc Read-Only Memory, CD - ROM), Digital Video Disc (DVD), memory sticks, floppy disks, mechanically encoded devices, such as punch cards or raised structures in grooves on which instructions are stored, and any suitable combination of the foregoing .

The computer readable program instructions or code described herein may be downloaded to various computing/processing devices from a computer readable storage medium, or to an external computer or external storage device over a network such as the Internet, a local area network, a wide area network and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer-readable program instructions from a network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .

The computer program instructions used to perform the operations of the present application may be assembly instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or in one or more source or object code written in any combination of programming languages, including object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or, can be connected to an external computer (e.g. use an internet service provider to connect via the internet). In some embodiments, electronic circuits, such as programmable logic circuits, Field-Programmable Gate Arrays (FPGA), or Programmable Logic Arrays (Programmable Logic Arrays), are personalized by utilizing state information of computer-readable program instructions. Logic Array, PLA), the electronic circuit can execute computer readable program instructions to implement various aspects of the present application.

Aspects of the present application are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present application. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer or other programmable data processing apparatus to produce a machine that causes the instructions when executed by the processor of the computer or other programmable data processing apparatus , resulting in means for implementing the functions/acts specified in one or more blocks of the flowchart and/or block diagrams. These computer readable program instructions can also be stored in a computer readable storage medium, these instructions cause a computer, programmable data processing apparatus and/or other equipment to operate in a specific manner, so that the computer readable medium storing the instructions includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.

Computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other equipment to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process , thereby causing instructions executing on a computer, other programmable data processing apparatus, or other device to implement the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of apparatuses, systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more functions for implementing the specified logical function(s) executable instructions. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.

It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in hardware (eg, circuits or ASICs (Application) that perform the corresponding functions or actions. Specific Integrated Circuit, application-specific integrated circuit)), or can be implemented by a combination of hardware and software, such as firmware.

While the invention has been described herein in connection with various embodiments, those skilled in the art will understand and understand from a review of the drawings, the disclosure, and the appended claims in practicing the claimed invention. Other variations of the disclosed embodiments are implemented. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that these measures cannot be combined to advantage.

Various embodiments of the present application have been described above, and the foregoing descriptions are exemplary, not exhaustive, and not limiting of the disclosed embodiments. Numerous modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or improvement over the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims (25)

1. An operator mapping method for a computational graph, characterized in that it is used for operator mapping of a first computational graph and a second computational graph, wherein the first computational graph and the second computational graph are of the same neural network model. Computational graph, the operator mapping method includes:

   Based on the first footprint matrix and the second footprint matrix, the following process is repeated until a stop condition is reached: calculating a first matrix from the first footprint matrix and the second footprint matrix, and adjusting the the order of rows and columns in the first footprint matrix, the first matrix is recalculated according to the adjusted first footprint matrix and the second footprint matrix; wherein the first footprint matrix indicates the first calculation graph The topological relationship between the operators of the second footprint matrix indicates the topological relationship between the operators of the second computational graph, and the first matrix indicates the first computational graph and the second computational graph The similarity of the topological relationship;

   According to the first footprint matrix and the second footprint matrix when the stopping condition is reached, the operators corresponding to each of the multiple operators in the first calculation graph in the second calculation graph are obtained.
2. The method according to claim 1, wherein calculating the first matrix according to the first footprint matrix and the second footprint matrix comprises:

   The first matrix is obtained according to a plurality of first eigenvectors and a plurality of second eigenvectors, wherein each of the first eigenvectors corresponds to an operator in the first calculation graph, and each of the first eigenvectors corresponds to an operator in the first calculation graph. The row vector corresponding to the operator corresponding to the eigenvector and the first eigenvector in the first footprint matrix is related to the column vector, and each of the second eigenvectors corresponds to the operator in the second calculation graph, Each of the second eigenvectors and the operator corresponding to the second eigenvectors are related to row vectors and column vectors in the second footprint matrix, and the first eigenvectors and the second eigenvectors are represented by for calculating the similarity of the topological relationship between the operator corresponding to the first eigenvector and the operator corresponding to the second eigenvector.
3. The method according to claim 2, wherein the value of the element of the first matrix represents the similarity of the topological relationship between the operator corresponding to the row where the element is located and the operator corresponding to the column where the element is located, wherein, The operator corresponding to the row where the element is located and the operator corresponding to the column where the element is located are the operator in the first calculation graph and the operator in the second calculation graph, respectively.
4. The method according to any one of claims 1-3, wherein the operators in the first calculation graph and the second calculation graph belong to multiple first partitions, and the operators in the same first partition The types are the same, and the dimensions of the tensors output by the operators are the same;

   The operators corresponding to the rows in the same position of the first footprint matrix and the second footprint matrix are in the same first partition, and the operators corresponding to the columns in the same position are in the same first partition;

   In addition, the operators of the first computation graph belonging to the same first partition are arranged consecutively in the corresponding rows or columns of the first footprint matrix, and the operators of the second computation graph belonging to the same first partition are arranged in the second footprint matrix. Corresponding rows or columns in the matrix are arranged consecutively.
5. The method according to claim 4, wherein some operators in the first computation graph and the second computation graph belong to a second partition, and the types of operators in the second partition are different and/or The dimensions of the tensors output by the operators are different.
6. The method according to claim 4, wherein the operators corresponding to the rows at the same position of the first footprint matrix and the second footprint matrix have the same homogeneous depth, and the operators corresponding to the columns at the same position have the same homogeneous depth. The rows or columns corresponding to the operators of the first computation graph belonging to the same first partition in the first footprint matrix are sorted according to the homogenous depth of the operators, and the first partition of the same first partition The corresponding rows or columns of the operators in the second footprint matrix are sorted according to the homogeneous depth of the operators;

   Wherein, the homogeneity depth is: in all the branches to which the operator belongs in the calculation graph, the maximum number of operators before the operator that have the same type as the operator and the same output tensor dimension.
7. The method of claim 5, wherein:

   If the number of operators of the first computation graph in the first partition is different from the number of operators of the second computation graph, the first partition also includes pseudo-operators; wherein the pseudo-operators are It is an operator without operator type and operand;

   If the number of operators of the first computation graph in the first partition is less than the number of operators of the second computation graph in the first partition, the pseudo-operator is in the first footprint The corresponding row or column in the matrix is located after the row or column corresponding to the operator belonging to the first partition in the first calculation graph,

   If the number of operators of the second computation graph in the first partition is less than the number of operators of the first computation graph in the first partition, the pseudo-operator is in the second footprint The corresponding row or column in the matrix is located after the row or column corresponding to the operator belonging to the first partition in the second calculation graph.
8. The method according to claim 5 or 7, wherein if the number of operators of the first computation graph in the second partition is different from the number of operators of the second computation graph, the The second partition also includes the pseudo operator;

   If the number of operators of the first computation graph in the second partition is less than the number of operators of the second computation graph in the second partition, the pseudo-operator is in the first footprint Corresponding rows and columns in the matrix are randomly sorted with the rows and columns corresponding to the operators belonging to the second partition in the first calculation graph, and the rows and columns are randomly sorted in the same manner;

   If the number of operators of the second computation graph in the second partition is less than the number of operators of the second computation graph in the second partition, the pseudo operator is located in the second footprint The corresponding rows and columns in the matrix and the rows and columns corresponding to the operators belonging to the second partition in the second computation graph are randomly ordered, and the random ordering of the rows and columns is the same.
9. The method according to any one of claims 5-8, wherein adjusting the order of rows and columns in the first footprint matrix according to the first matrix comprises:

   The order of the rows and columns corresponding to the operators located in the same first partition in the first footprint matrix is adjusted according to the first matrix.
10. The method according to any one of claims 5-8, wherein,

    an operator and a second computation graph of the first computation graph belonging to the same first partition and having the same row or column position in the first footprint matrix and the second footprint matrix when the stopping condition is reached There is a mapping relationship between the operators of ;

    There is a mapping relationship between the operators of the first computation graph belonging to the second partition and multiple operators of the second computation graph belonging to the second partition,

    Wherein, the multiple operators of the second computation graph belonging to the second partition are: in the first matrix when the stopping condition is reached, the operators corresponding to the operators of the first computation graph belonging to the second partition Among the values of the elements of , the operator of the second calculation graph corresponding to the largest multiple values.
11. The method according to any one of claims 1-10, wherein the stopping condition is that the number of times of repeatedly calculating the first matrix is greater than a first threshold.
12. A computational graph operator mapping device, characterized in that it is used for operator mapping to a first computational graph and a second computational graph, wherein the first computational graph and the second computational graph are of the same neural network model. Computational graph, the operator mapping device includes:

    an iterative module for repeating the following process based on the first footprint matrix and the second footprint matrix until a stopping condition is reached: calculating a first matrix according to the first footprint matrix and the second footprint matrix, and calculating a first matrix according to the first footprint matrix and the second footprint matrix A matrix adjusts the order of rows and columns in the first footprint matrix, and the first matrix is recalculated according to the adjusted first footprint matrix and the second footprint matrix; wherein the first footprint matrix indicates the the topological relationship between the operators of the first computational graph, the second footprint matrix indicates the topological relationship between the operators of the second computational graph, the first matrix indicates the first computational graph and all the similarity of the topological relationship of the second calculation graph;

    A corresponding module, configured to obtain operators corresponding to each of the multiple operators in the first calculation graph in the second calculation graph according to the first footprint matrix and the second footprint matrix when the stopping condition is reached.
13. The apparatus according to claim 12, wherein the iterative module comprises:

    The calculation unit is configured to obtain the first matrix according to a plurality of first eigenvectors and a plurality of second eigenvectors, wherein each of the first eigenvectors corresponds to an operator in the first calculation graph, and each Each of the first eigenvectors and the operators corresponding to the first eigenvectors are related to the row vectors and column vectors in the first footprint matrix, and each of the second eigenvectors corresponds to the second calculation graph The operators in , each of the second eigenvectors and the operators corresponding to the second eigenvectors are related to the row vectors and column vectors in the second footprint matrix, and the first eigenvectors and the The second eigenvector is used to calculate the similarity of the topological relationship between the operator corresponding to the first eigenvector and the operator corresponding to the second eigenvector.
14. The device according to claim 13, wherein the value of the element of the first matrix represents the similarity of the topological relationship between the operator corresponding to the row where the element is located and the operator corresponding to the column where the element is located, wherein, The operator corresponding to the row where the element is located and the operator corresponding to the column where the element is located are the operator in the first calculation graph and the operator in the second calculation graph, respectively.
15. The device according to any one of claims 12-14, wherein the operators in the first calculation graph and the second calculation graph belong to multiple first partitions, and operators in the same first partition The types are the same, and the dimensions of the tensors output by the operators are the same;

    The operators corresponding to the rows in the same position of the first footprint matrix and the second footprint matrix are in the same first partition, and the operators corresponding to the columns in the same position are in the same first partition;

    In addition, the operators of the first computation graph belonging to the same first partition are arranged consecutively in the corresponding rows or columns of the first footprint matrix, and the operators of the second computation graph belonging to the same first partition are arranged in the second footprint matrix. Corresponding rows or columns in the matrix are arranged consecutively.
16. The apparatus according to claim 15, wherein some operators in the first computation graph and the second computation graph belong to a second partition, and the types of operators in the second partition are different and/or The dimensions of the tensors output by the operators are different.
17. The device according to claim 15, wherein the operators corresponding to the rows at the same position of the first footprint matrix and the second footprint matrix have the same homogeneous depth, and the operators corresponding to the columns at the same position have the same homogeneous depth. The rows or columns corresponding to the operators of the first computation graph belonging to the same first partition in the first footprint matrix are sorted according to the homogenous depth of the operators, and the first partition of the same first partition The corresponding rows or columns of the operators in the second footprint matrix are sorted according to the homogeneous depth of the operators;

    Wherein, the homogeneity depth is: in all the branches to which the operator belongs in the calculation graph, the maximum number of operators before the operator that have the same type as the operator and the same output tensor dimension.
18. The apparatus of claim 16, wherein:

    If the number of operators of the first computation graph in the first partition is different from the number of operators of the second computation graph, the first partition also includes pseudo-operators; wherein the pseudo-operators are It is an operator without operator type and operand;

    If the number of operators of the first computation graph in the first partition is less than the number of operators of the second computation graph in the first partition, the pseudo-operator is in the first footprint The corresponding row or column in the matrix is located after the row or column corresponding to the operator belonging to the first partition in the first calculation graph,

    If the number of operators of the second computation graph in the first partition is less than the number of operators of the first computation graph in the first partition, the pseudo-operator is in the second footprint The corresponding row or column in the matrix is located after the row or column corresponding to the operator belonging to the first partition in the second calculation graph.
19. The device according to claim 16 or 18, wherein if the number of operators of the first computation graph in the second partition is different from the number of operators of the second computation graph, the The second partition also includes the pseudo operator;

    If the number of operators of the first computation graph in the second partition is less than the number of operators of the second computation graph in the second partition, the pseudo-operator is in the first footprint Corresponding rows and columns in the matrix are randomly sorted with the rows and columns corresponding to the operators belonging to the second partition in the first calculation graph, and the rows and columns are randomly sorted in the same manner;

    If the number of operators of the second computation graph in the second partition is less than the number of operators of the second computation graph in the second partition, the pseudo operator is located in the second footprint The corresponding rows and columns in the matrix and the rows and columns corresponding to the operators belonging to the second partition in the second computation graph are randomly ordered, and the random ordering of the rows and columns is the same.
20. The device according to any one of claims 16-19, wherein the iterative module further comprises:

    An adjustment unit, configured to adjust the order of the rows and columns corresponding to the operators located in the same first partition in the first footprint matrix according to the first matrix.
21. The device according to any one of claims 16-19, characterized in that,

    an operator and a second computation graph of the first computation graph belonging to the same first partition and having the same row or column position in the first footprint matrix and the second footprint matrix when the stopping condition is reached There is a mapping relationship between the operators of ;

    There is a mapping relationship between the operators of the first computation graph belonging to the second partition and multiple operators of the second computation graph belonging to the second partition,

    Wherein, the multiple operators of the second computation graph belonging to the second partition are: in the first matrix when the stopping condition is reached, the operators corresponding to the operators of the first computation graph belonging to the second partition Among the values of the elements of , the operator of the second calculation graph corresponding to the largest multiple values.
22. The apparatus according to any one of claims 12-21, wherein the stopping condition is that the number of times of repeatedly calculating the first matrix is greater than a first threshold.
23. An operator mapping device for computing a graph, comprising:

    processor;

    memory for storing processor-executable instructions;

    Wherein, the processor is configured to implement the method of any one of claims 1-11 when executing the instructions.
24. A non-volatile computer-readable storage medium on which computer program instructions are stored, characterized in that, when the computer program instructions are executed by a processor, the method described in any one of claims 1-11 is implemented.
25. A computer program product comprising computer readable code which, when run in an electronic device, implements the method of any one of claims 1-11.

Images