CN117933334A - Model processing method, device, electronic equipment and storage medium - Google Patents

Abstract

The disclosure provides a model processing method, relates to the technical field of artificial intelligence, and particularly relates to the technical fields of chips, deep learning platforms and large models. The specific implementation scheme is as follows: displaying a calculation diagram of the model to be processed on a visual interface according to model data to be edited of the model to be processed, wherein the calculation diagram comprises M nodes, M is an integer greater than or equal to 1, and the model to be processed corresponds to a first deep learning frame; acquiring at least one target operation data through a visual interface, wherein the target operation data are related to N nodes in a calculation graph, and N is an integer greater than or equal to 1; and adjusting the model data to be edited according to at least one target operation data to obtain target model data. The disclosure also provides a model processing device, an electronic device and a storage medium.

Description

Model processing method, device, electronic equipment and storage medium

Technical Field

The disclosure relates to the technical field of artificial intelligence, in particular to the technical fields of chips, deep learning platforms, large models and the like. More particularly, the present disclosure provides a model processing method, apparatus, electronic device, and storage medium.

Background

With the development of artificial intelligence technology, the deep learning model can be applied to various application scenes. In different application scenarios, the model can be adjusted differently so that efficient reasoning can be performed using the model in different application scenarios.

Disclosure of Invention

The present disclosure provides a model processing method, apparatus, device, and storage medium.

According to an aspect of the present disclosure, there is provided a model processing method including: displaying a calculation diagram of the model to be processed on a visual interface according to model data to be edited of the model to be processed, wherein the calculation diagram comprises M nodes, M is an integer greater than or equal to 1, and the model to be processed corresponds to a first deep learning frame; acquiring at least one target operation data through a visual interface, wherein the target operation data are related to N nodes in a calculation graph, and N is an integer greater than or equal to 1; and adjusting the model data to be edited according to at least one target operation data to obtain target model data.

According to another aspect of the present disclosure, there is provided a model processing apparatus including: the display module is used for displaying a calculation diagram of the to-be-processed model on a visual interface according to-be-edited model data of the to-be-processed model, wherein the calculation diagram comprises M nodes, M is an integer greater than or equal to 1, and the to-be-processed model corresponds to the first deep learning frame; the acquisition module is used for acquiring at least one target operation data through a visual interface, wherein the target operation data are related to N nodes in the calculation graph, and N is an integer greater than or equal to 1; and the adjusting module is used for adjusting the model data to be edited according to at least one target operation data to obtain target model data.

According to another aspect of the present disclosure, there is provided an electronic device including the model processing apparatus provided by the present disclosure.

According to another aspect of the present disclosure, there is provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method provided in accordance with the present disclosure.

According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform a method provided according to the present disclosure.

According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements a method provided according to the present disclosure.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.

Drawings

The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

FIG. 1 is a schematic diagram of an exemplary system architecture to which model processing methods and apparatus may be applied, according to one embodiment of the present disclosure;

FIG. 2 is a flow chart of a model processing method according to one embodiment of the present disclosure;

FIG. 3 is a schematic illustration of a computational graph of a model to be processed according to one embodiment of the present disclosure;

FIG. 4A is a schematic flow chart diagram of node deletion according to one embodiment of the present disclosure;

FIG. 4B is a schematic diagram of a visual interface according to one embodiment of the present disclosure;

FIG. 5A is a schematic flow chart diagram of subgraph extraction in accordance with one embodiment of the present disclosure;

FIG. 5B is a schematic diagram of a visual interface according to another embodiment of the present disclosure;

FIG. 6A is a schematic flow chart diagram of node fusion according to one embodiment of the present disclosure;

FIG. 6B is a schematic diagram of a visual interface according to another embodiment of the present disclosure;

FIG. 7 is a schematic diagram of generating object code according to one embodiment of the present disclosure;

FIG. 8 is a block diagram of a model processing device according to one embodiment of the present disclosure;

FIG. 9 is a schematic block diagram of an electronic device according to one embodiment of the present disclosure; and

Fig. 10 is a block diagram of an electronic device to which a model processing method may be applied according to one embodiment of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

In some embodiments, the graph structure of the model may be adjusted to improve the reasoning capabilities of the model in relevant application scenarios. For example, less computationally efficient nodes or fused portions of nodes may be replaced to improve the reasoning capabilities of the model. The graph structure of the model may be adjusted based on various deep learning frameworks.

However, different deep learning frameworks differ from one another in the definition of the model structure, and users need to learn and edit the model structure using interfaces specific to each deep learning framework while processing models in different formats. For example, the plurality of deep learning frames may include a flyer (PADDLEPADDLE) frame, a tensor flow (TensorFlow) frame, and so on. The model storage formats differ between these frameworks, resulting in the need to use different interfaces for editing the model structure. When editing using these interfaces, the user is faced with complex operational procedures and learning difficulties. In addition, these interfaces are generally limited to a specific framework, and have poor versatility, which results in a significant increase in complexity of model structure editing, and a high editing cost for users.

In some embodiments, some visual deep learning modeling tools may provide a graphical interface to create and expose a deep learning model. However, these visual deep learning modeling tools only involve the creation and presentation of models, which are difficult to directly edit the model structure and have poor flexibility.

In some embodiments, the model conversion tool may convert the model of the first deep learning framework to a model of the second deep learning framework. However, model conversion tools are mainly concerned with conversion between models. The user needs to edit based on the converted model, and also needs to edit based on the editing interface of the specific framework. Furthermore, operators between different frameworks may be difficult to correspond to each other, and model transformation tools may be difficult to successfully implement transformation of a partial model between different frameworks.

Thus, the present disclosure provides a model processing method, which will be described below.

FIG. 1 is a schematic diagram of an exemplary system architecture to which the model processing methods and apparatus may be applied, according to one embodiment of the present disclosure. It should be noted that fig. 1 is only an example of a system architecture to which embodiments of the present disclosure may be applied to assist those skilled in the art in understanding the technical content of the present disclosure, but does not mean that embodiments of the present disclosure may not be used in other devices, systems, environments, or scenarios.

As shown in fig. 1, a system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and/or wireless communication links, and the like.

The user may interact with the server 105 via the network 104 using the terminal devices 101, 102, 103 to receive or send messages or the like. The terminal devices 101, 102, 103 may be a variety of electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.

The server 105 may be a server providing various services, such as a background management server (by way of example only) providing support for websites browsed by users using the terminal devices 101, 102, 103. The background management server may analyze and process the received data such as the user request, and feed back the processing result (e.g., the web page, information, or data obtained or generated according to the user request) to the terminal device.

It should be noted that, the model processing method provided in the embodiment of the present disclosure may be generally executed by the server 105. Accordingly, the model processing apparatus provided by the embodiments of the present disclosure may be generally provided in the server 105. The model processing method provided by the embodiments of the present disclosure may also be performed by a server or a server cluster that is different from the server 105 and is capable of communicating with the terminal devices 101, 102, 103 and/or the server 105. Accordingly, the model processing apparatus provided by the embodiments of the present disclosure may also be provided in a server or a server cluster that is different from the server 105 and is capable of communicating with the terminal devices 101, 102, 103 and/or the server 105.

Fig. 2 is a flow chart of a model processing method according to one embodiment of the present disclosure.

As shown in fig. 2, the method 200 may include operations S210 to S230.

In operation S210, a calculation map of the model to be processed is displayed on the visual interface according to model data to be edited of the model to be processed.

In the embodiment of the disclosure, the model to be processed corresponds to the first deep learning frame. The first deep learning framework may be one of the above-described flyer framework and tensor flow framework.

In the disclosed embodiment, the computation graph may include M nodes. M may be an integer greater than or equal to 1. For example, the computational graph may be a directed graph. The computational graph may also include edges. Edges may indicate associations between nodes.

At operation S220, at least one target operation data is acquired via the visual interface.

In embodiments of the present disclosure, the visual interface may also present at least one control. Each control may correspond to one or more operations. After a node is selected, a user may trigger one or more controls to perform various operations on the node. And according to the control triggered by the user for the node, acquiring target operation data of the user for the target node.

In the disclosed embodiment, the target operation data is associated with N nodes in the computational graph. The target operational data may be operational data of a target node. The target node may be a node selected by the user. For example, the target node may be a node in the computation graph, or may be a node outside the computation graph, which is not limited by the present disclosure.

In the embodiments of the present disclosure, N may be an integer greater than or equal to 1. For example, N may be an integer less than or equal to M.

In operation S230, the model data to be edited is adjusted according to the target operation data of at least one target node, so as to obtain target model data.

In the embodiment of the disclosure, according to the target operation data, the data of the model data to be edited and the data related to the target node can be adjusted. After the model data to be edited is adjusted using the at least one target operation data, the target model data may be obtained.

According to the embodiment of the disclosure, based on the visual interface, a user can quickly adjust the graph structure of the model by simple point selection, so that the time cost of editing codes is greatly reduced. Editing the target node based on the visual interface is beneficial to improving the accuracy of the back-end mapping.

It will be appreciated that while the model processing method of the present disclosure is described above, the object model data of the present disclosure will be further described below.

In some embodiments, the method 200 may further include: and determining a target model corresponding to the target deep learning framework according to the conversion protocol and the target model data.

In an embodiment of the present disclosure, the conversion protocol may include a protocol buffer (protocol).

In the embodiments of the present disclosure, the target deep learning framework may be any deep learning framework. For example, the target deep learning frame may be the first deep learning frame or the second deep learning frame.

It will be appreciated that the foregoing describes the object model data of the present disclosure, and the following describes the computational graph of the present disclosure.

FIG. 3 is a schematic illustration of a computational graph of a model to be processed according to one embodiment of the present disclosure.

As shown in fig. 3, the computation graph 30 may include M nodes. The M nodes include an initial input node In30, a first computing node N31, a second computing node N32, a third computing node N33, a fourth computing node N34, a fifth computing node N35, a sixth computing node N36, a seventh computing node N37, and an output node Out30.

In some embodiments, presenting the computational graph of the model to be processed at the visual interface may include: and determining model data to be edited of the model to be processed according to the conversion protocol and the model to be processed corresponding to the first deep learning frame. For example, an interface provided by the first deep learning framework may be invoked to load initial model data for the model to be processed corresponding to the first deep learning framework. Then, the buffer protocol can be utilized to convert the initial model data to obtain the model data to be edited. The model data to be edited may include message structure class data of a cache protocol.

In the disclosed embodiment, the model data to be edited may include graph structure (GraphDef) data, node structure (NodeDef) data, and value structure (ValueDef) data. For example, the graph structure data corresponds to a calculation graph. The node structure data corresponds to nodes in the computational graph. The value structure data corresponds to constant data required for calculating nodes in the graph. The constant data may include data such as weight parameters. According to the embodiment of the invention, the data of the model to be edited is determined by utilizing the conversion protocol, so that the multi-frame model editing interface can be unified, interfaces of various frames are not required to be additionally learned in the process of adapting and optimizing the model, the model optimizing efficiency is improved, and the model application cost is reduced.

In an embodiment of the present disclosure, nodes in a computation graph include a plurality of node information. The plurality of node information includes node type information, node name information, node input information, and node output information.

For example, for the first calculation node N31, the node type information may include a node type "exp". The node type "exp" may indicate: the nodes may be calculated using an exponential function based on a natural constant e. The node name information may include a node name "exp_01". The node input information may include a node input "which may be an output of the initial input node In 30. The node output information may include a node output "exp_01_out".

For example, for the second computing node N32, the node type information may include a node type "abs". The node type "abs" may indicate: nodes may be calculated using absolute value functions. The node name information may include a node name "abs_01". The node input information may include a node input "exp_01_out", which may be a node output of the first computation node N31. The node output information may include node output "abs_01_out".

For example, for the third computing node N33, the node type information may include a node type "log". The node type "log" may indicate: the nodes may be calculated using a base 10 logarithmic function. The node name information may include a node name "log_01". The node input information may include a node input "abs_01_out", which may be an output of the second computing node N32. The node output information may include a node output "log_01_out".

For example, for the fourth calculation node N34, the node type information may include a node type "exp". The node name information may include a node name "exp_02". The node input information may include a node input "log_01_out", which may be an output of the third computing node N33. The node output information may include a node output "exp_02_out".

For example, for the fifth calculation node N35, the node type information may include a node type "exp". The node name information may include a node name "exp_03". The node input information may include a node input "exp_02_out", which may be an output of the fourth computation node N34. The node output information may include a node output "exp_03_out".

For example, for the sixth computing node N36, the node type information may include a node type "add". The node type "add" may indicate: nodes may be calculated using an addition function. The node name information may include a node name "add_01". The node input information may include a node input "exp_03_out", which may be an output of the fifth calculation node N35. The node output information may include node output "add_01_out".

For example, for the seventh calculation node N37, the node type information may include the node type "log". The node name information may include a node name "log_02". The node input information may include a node input "add_01_out", which may be an output of the sixth computing node N36. The node output information may include a node output "log_02_out".

In the embodiment of the present disclosure, the node information may further include node attribute information. For example, the node attribute information includes a node-related data type (e.g., 32-bit floating point number FP32 or 16-bit floating point number FP 16).

In some embodiments, presenting the computational graph of the model to be processed at the visual interface may include: and according to the model data to be edited, displaying the calculation graph on the visual interface. For example, the file format of the model data to be edited may be "pbtxt". The model data to be edited may be loaded with a program having a visual interface to expose a computational graph. When loading model data to be edited, a node name-to-node mapping (name_to_node) set, a node name-to-input node mapping (name_to_input_node) set, and a node name-to-output node mapping (name_to_output_node) set of the computation graph may be determined. The set of mappings may be implemented as a dictionary (direct). By the embodiment of the disclosure, the accuracy of mapping between the calculation map and the back-end code is improved.

It will be appreciated that while the computational graph of the present disclosure is described above, some embodiments of acquiring at least one target operational data via a visual interface will be described below.

In some embodiments, obtaining, via the visual interface, the at least one target operational data comprises: and in response to determining that the control is triggered, determining first operation data of the target node according to at least one of user input information and preset operation corresponding to the control. And obtaining target operation data of the target node according to at least one first operation data of the target node.

In embodiments of the present disclosure, the visual interface may be presented with at least one control. Each control corresponds to one or more preset operations.

In the embodiment of the present disclosure, the target node may be a node selected by the user. It will be appreciated that a user may trigger a control and then select a node. In addition, the user may also select a node first and then trigger a control, which is not limited by the present disclosure. It will also be appreciated that, for a target node, a user may trigger one or more controls to obtain one or more target operational data for the target node.

Some ways of obtaining the target operation data will be further described below in connection with different controls, respectively.

In embodiments of the present disclosure, the at least one control may include a node addition control. The node adding control corresponds to a plurality of preset nodes. The preset operation corresponding to the node adding control comprises an adding operation. For example, a plurality of preset nodes may be determined from a plurality of operators supported by the first deep learning framework and the second deep learning framework. For another example, the aggregate classification may be performed according to a plurality of operators supported by a plurality of deep learning frameworks including a first deep learning framework and a second deep learning framework to determine a plurality of preset nodes. Then, a plurality of preset operators corresponding to the node adding control can be determined according to the operators supported by the target deep learning framework.

In an embodiment of the present disclosure, in response to determining that the control is triggered, determining the first operation data of the target node according to at least one of the user input information and the preset operation corresponding to the control may include: and responding to the fact that the node adding control is triggered, and acquiring at least one node to be added selected by a user from a plurality of preset nodes as at least one target node. For example, after the node addition control is triggered, a plurality of preset nodes may be presented at the visual interface. At least one node selected by the user can be obtained and used as at least one node to be added.

In an embodiment of the present disclosure, in response to determining that the control is triggered, determining the first operation data of the target node according to at least one of the user input information and the preset operation corresponding to the control may include: and obtaining first operation data of the target node according to the user input information and preset operation corresponding to the node adding control. The user input information may include at least one node to be associated with the target node. At least one node to be associated is located in the computational graph. For example, in the case where the node to be added is a target node, the node type information and the node name information of the node to be added may be determined according to a preset node corresponding to the node to be added. For the node to be added, the node type information may include a node type "exp". The node name information may include a node name "node_01". The user input information may indicate node input information and node output information of the node to be added. The node input information may include a node input "exp_01_out", which may be an output of the above-described first calculation node N31. The node output information includes node output "node_01_out". In addition, according to the user input information, the node input of the second computing node may be modified to "node_01_out", or the first computing node N31 and the second computing node N32 may be respectively used as the nodes to be associated. And then, according to the plurality of node information of the nodes to be added and the preset operation corresponding to the node adding control, determining the first operation data.

In an embodiment of the present disclosure, obtaining, according to at least one first operation data of a target node, the target operation data of the target node may include: and obtaining target operation data according to the first operation data. The target operational data may at least indicate: at least one target node is added to the computational graph. And establishing an association relation between the target node and at least one node to be associated. For example, according to the target operation data, at least one node to be added may be added in the calculation graph, and an edge connecting the node to be added and the node to be associated may be established. Through the embodiment of the disclosure, various nodes can be added efficiently, and operators supported by the second deep learning framework can be added efficiently when the model of the first deep learning framework is converted into the model of the second deep learning framework.

It will be appreciated that the node addition control of the present disclosure is described above and the node deletion control will be described below.

In an embodiment of the present disclosure, the at least one control includes at least one of a first node deletion control and a second node deletion control. The preset operation corresponding to the first node deleting control comprises deleting operation. The preset operation corresponding to the second node deleting control comprises deleting operation. For example, a first node delete control may be used to delete a target node. For another example, the second node delete control can be used to delete the target node and at least one of a preceding node and a following node of the target node. The following will explain in conjunction with fig. 4A and 4B.

Fig. 4A is a schematic flow diagram of node deletion according to one embodiment of the present disclosure. Fig. 4B is a schematic diagram of a visual interface according to one embodiment of the present disclosure.

As shown in fig. 4A, the method 421 may determine, in response to determining that the control is triggered, first operation data of the target node according to at least one of user input information and a preset operation corresponding to the control. The method 421 may include operations S4211 through S4219.

As shown in fig. 4B, the visual interface may present a computational graph 40, a first node delete control 41, a preceding node delete control 42, and a following node delete control 43. The calculation map 40 may include an initial input node In40, first to seventh calculation nodes N41 to N47, and an output node Out40. The above description of the calculation map 30 is equally applicable to the calculation map 40, and the disclosure is not repeated here.

Some embodiments of node deletion in the present disclosure will be described below in connection with operations S4211 through S4215 and the first node deletion control 41.

In operation S4211, a target node is determined.

In the embodiment of the disclosure, the target node may be a node in the computation graph. For example, a node in the computation graph selected by the user may be taken as the target node.

In operation S4212, determining whether the first node delete control is triggered?

In the disclosed embodiment, in response to determining that the first node delete control is triggered, operation S4213 may be performed.

In the disclosed embodiment, in response to determining that the first node delete control is not triggered, operation S4219 may be performed waiting for the control to be triggered.

In operation S4213, an association relationship between the following node of the target node and the target node is acquired.

In the embodiment of the disclosure, the following nodes of the target node can be determined according to the mapping set from the node to the output node. For example, the node name to output node mapping combination may indicate: a node and a subsequent node associated with the node. For another example, the mapping combination of node names to output nodes may include mapping information "first computing node: a second computation node. The mapping information may indicate that a subsequent node associated with the first computing node N41 is the second computing node N42. It will be appreciated that the output of the destination node may be the input of a plurality of subsequent nodes or may be the input of one subsequent node, which is not limiting in this disclosure.

In the embodiment of the disclosure, first operation data is determined according to a preset operation corresponding to a first node deletion control and at least one following node associated with a target node. The first operational data may indicate: deleting at least one association relationship between the latter node and the target node and deleting the target node. The node input of the following node may be deleted to delete the association relationship between the following node and the target node. For example, if the first computing node N41 is the target node, the corresponding first operation data may indicate: the node input of the second computing node N42 is deleted and the first computing node N41 is deleted.

In operation S4214, user input information is acquired.

In the embodiment of the disclosure, information related to the following nodes input by a user can be acquired.

In operation S4215, second operation data corresponding to the following node is determined.

In an embodiment of the present disclosure, at least one second operation data corresponding to at least one subsequent node is determined according to user input information. The second operation data may indicate an input of a following node. For example, if the first computing node N41 is a target node, the second operation data of the second computing node N42 may be determined according to the user input information. The user may select the second computing node N42. In response to the second computing node N42 being selected, the node type, node name, node input, and node output of the second computing node 42 may be presented at the visual interface. As described above, the node input of the second computing node N42 has been deleted. The user may input the node input "node_01out" of the second computing node N42 at the visual interface. The second operational data may include node inputs of a second computing node N42. Through the embodiment of the disclosure, a single node can be deleted efficiently, and the association relationship between other nodes and the previous node or the following node of the deleted node can be established efficiently and intuitively according to the input of a user.

Next, obtaining the target operation data of the target node according to the at least one first operation data of the target node may include: and obtaining target operation data according to the first operation data and the at least one second operation data. For example, the target operation data may be obtained from the first operation data of the first calculation node N41 and the second operation data of the second calculation node N42.

It is to be appreciated that the present disclosure is described above in connection with the first node deletion control 41, and that the following will be described in connection with the preceding node deletion control 42, the following node deletion control 42, and operations S4211, S4216 to S4218.

In operation S4211, a target node is determined.

For example, a node selected by the user may be a target node.

In operation S4216, determining whether the second node delete control is triggered?

In the disclosed embodiment, in response to determining that the second node delete control is triggered, operation S4217 may be performed. The preceding node deletion control 42 and the following node deletion control 43 may function as second node deletion controls. It will be appreciated that the preceding node delete control 42 and the following node delete control 43 may be triggered separately or sequentially in various orders, which is not limiting of the present disclosure. For example, as shown in fig. 4B, if the post node delete control 43 is triggered, operation S4217 may be performed.

In the disclosed embodiment, in response to determining that the second node delete control is not triggered, operation S4219 may be performed waiting for the control to be triggered.

In operation S4217, at least one of the preceding node set and the following node set is determined.

In the disclosed embodiments, at least one of the preceding node set and the following node set may be determined from the node-to-output node map set. For example, as shown in fig. 4B, taking the fourth computing node N44 as an example of the target node, the subsequent node set of the fourth computing node N44 includes a fifth computing node N45, a sixth computing node N46, and a seventh computing node N47. For another example, as shown in fig. 4B, if the fourth computing node N44 is a target node, a plurality of node information of the fourth computing node N44 may be displayed. The plurality of node information includes node type information, node name information, node input information, and node output information. The node type information 441 includes a node type "exp". The node name information 442 may include a node name "exp_02". The node input information 443 may include a node input "log_01_out". The node output information 444 may include a node output "exp_02_out".

In operation S4218, first operation data is determined.

In the embodiment of the disclosure, the first operation data is determined according to preset operations corresponding to at least one of the following node set and the preceding node set, the target node and the second node deletion control. The first operational data may indicate: at least one of the set of following nodes and the set of preceding nodes and the target node are deleted. For example, taking the example that the post node deletion control 43 is triggered, the first operation data may be determined according to the post node set of the fourth computing node N44, and the preset operation corresponding to the post node deletion control 43. The first operational data may indicate: the fourth calculation node N44, the fifth calculation node N45, the sixth calculation node N46, and the seventh calculation node N47 are deleted.

Then, according to at least one first operation data of the target node, target operation data of the target node can be obtained. For example, the first operation data of the fourth computing node N44 may be the target operation data. According to the embodiment of the invention, the nodes associated with the target node can be deleted efficiently, so that the floating nodes (nodes without input or output) in the calculation graph are avoided, the user operation is reduced, and the processing efficiency is improved.

It will be appreciated that the method of the present disclosure is described above in connection with a plurality of node deletion controls, and that the present disclosure will be described below in connection with node modification controls.

In some embodiments, the at least one control may include a node modification control. The node modification control may be used to modify node information.

In some embodiments, in response to determining that the control is triggered, determining the first operation data of the target node according to at least one of the user input information and the preset operation corresponding to the control may include: in response to determining that the node modification control is triggered, a plurality of node information for the target node is presented. For example, if the node modification control is triggered, in the case that the target node is the fourth computing node N44, node type information 441, node name information 442, node input information 443, and node output information 444 of the fourth computing node N44 may be displayed.

In some embodiments, in response to determining that the control is triggered, determining the first operation data of the target node according to at least one of the user input information and the preset operation corresponding to the control may include: and taking at least one node information selected by a user in the plurality of node information as at least one node information to be modified. For example, if the user selects the node name information 442, the node name information may be used as the node information to be modified.

In some embodiments, in response to determining that the control is triggered, determining the first operation data of the target node according to at least one of the user input information and the preset operation corresponding to the control may include: and determining first operation data according to the user input information and the at least one node information to be modified. The first operation data includes at least one target information value of node information to be modified. For example, the user may input a target information value of the node name information 442. The first operation data may be determined based on the target information value and the node name information of the fourth calculation node N44. Through the embodiment of the disclosure, node information can be efficiently and accurately modified.

It will be appreciated that while the disclosure has been described above, a sub-graph extraction control of the disclosure will be described below.

In some embodiments, the at least one control includes a computation subgraph extraction control, and the preset operation corresponding to the subgraph extraction control includes a computation graph generation operation. The following will explain in conjunction with fig. 5A and 5B.

Fig. 5A is a schematic flow chart diagram of subgraph extraction according to one embodiment of the present disclosure. Fig. 5B is a schematic diagram of a visual interface according to another embodiment of the present disclosure.

As shown in fig. 5A, the method 521 may determine, in response to determining that the computation subgraph extraction control is triggered, first operation data according to at least one target node, an initial input node of the computation graph, and a preset operation corresponding to the computation subgraph extraction control. In the embodiment of the disclosure, at least one continuous association relationship can be determined according to at least one target node and an initial input node of a calculation graph. The first operation data may be determined according to at least one continuous association relation.

As shown in fig. 5B, the visual interface may present a computational graph 50. The subgraph extraction control may include a first node selection control 54 and a first validation control 55. The calculation map 50 may include an initial input node In50, first to seventh calculation nodes N51 to N57, and an output node Out50. The above description of the calculation map 30 is equally applicable to the calculation map 50, and the disclosure is not repeated here. The following will explain in conjunction with operations S5211 to S5217.

In operation S5211, it is determined whether the first node selection control is triggered.

In the disclosed embodiment, in response to determining that the first node selection control 54 of the subgraph extraction control is triggered, operation S5212 may be performed.

In the disclosed embodiment, in response to determining that the first node selection control 54 of the subgraph extraction control is not triggered, operation S5217 may be performed waiting for the control to be triggered.

In operation S5212, a target node is determined.

In the embodiment of the disclosure, the target node may be a node in the computation graph. For example, a node selected by the user may be a target node. As shown in fig. 5B, the fifth computing node may be referred to as a target node. If the fifth computing node N55 is a target node, a plurality of node information of the fifth computing node N55 may be displayed. The plurality of node information includes node type information, node name information, node input information, and node output information. The node type information 551 may include a node type "exp". The node name information 552 may include a node name "exp_03". The node input information 553 may include a node input "exp_02_out". The node output information 554 may include node output "exp_03_out".

In operation S5213, it is determined whether the first confirmation control is triggered.

In the presently disclosed embodiments, in response to determining that the first confirmation control is triggered, operation S5214 may be performed. For example, taking the example of the user selecting a target node, after selecting the fifth computing node, the user may trigger the first confirmation control. Next, operation S5214 may be performed.

In response to determining that the first confirmation control is not triggered, in embodiments of the present disclosure, a return may be made to operation S5212, either to continue waiting for the user to select a node or for the first confirmation control to be triggered. For another example, in other embodiments, the user may also select multiple target nodes. After selecting the plurality of target nodes, the user may trigger a first confirmation control.

In operation S5214, a continuous association relationship between the current target node and the initial input node is acquired.

In an embodiment of the present disclosure, the continuous association relationship includes at least one association relationship between the target node and the initial input node. For example, the continuous association relationship between the fifth calculation node N55 and the initial input node In50 includes: an association between the fifth calculation node N55 and the fourth calculation node N54, an association between the fourth calculation node N54 and the third calculation node N53, an association between the third calculation node N53 and the second calculation node N52, an association between the second calculation node N52 and the first calculation node N51, and an association between the first calculation node N51 and the initial input node In 50.

In the embodiment of the present disclosure, the continuous association relationship may be determined using a preset algorithm. For example, the preset algorithm may include a depth-first search algorithm (DFS). The algorithm may search for associations between the target nodes and the initial input nodes to determine successive associations.

In operation S5215, it is determined whether or not the continuous association relationship of all the target nodes has been determined.

In the embodiment of the present disclosure, in response to having determined the continuous association relationship of all the target nodes, operation S5216 may be performed.

In operation S5216, first operation data is determined.

For example, the first operation data may be determined according to the continuous association relationship between the fifth calculation node N55 and the initial input node In 50.

In an embodiment of the present disclosure, the first operation data is used to indicate: and generating a calculation subgraph according to at least one target node and the initial input node. The first operation data of the fifth calculation node may correspond to a calculation subgraph including an initial input node In50, a first calculation node N51, a second calculation node N52, a third calculation node N53, a fourth calculation node N54, a fifth calculation node N55, and an output node. The input of the output node may be the output of the fifth computing node N55.

In another embodiment of the present disclosure, in response to not determining the continuous association relationship of all the target nodes, the current target node may be re-determined from the at least one target node and returned to operation S5214. For example, if the sixth computing node N56 is also the target node, the operation may return to operation S5214 to determine the continuous association relationship between the sixth computing node N56 and the initial input node In50 to determine the first operation data of the sixth computing node N56.

It will be appreciated that the description above is in connection with sub-graph extraction controls and will be described below in connection with fusion controls.

In some embodiments, the at least one control may comprise a fusion control. A fusion control can be used to fuse multiple nodes. The following will explain in conjunction with fig. 6A and 6B.

Fig. 6A is a schematic flow chart diagram of node fusion according to one embodiment of the present disclosure. Fig. 6B is a schematic diagram of a visual interface according to another embodiment of the present disclosure.

As shown in fig. 6A, the method 621 may determine the first operation data according to the user input information, the plurality of target nodes, and the preset operation corresponding to the fusion control in response to determining that the computation subgraph extraction control is triggered, in response to determining that the fusion control is triggered. In the embodiment of the disclosure, multiple node information of the fusion node can be determined according to the user input information and the multiple target nodes. And determining the first operation data according to the information of the multiple nodes of the fusion node and the preset operation corresponding to the fusion control.

As shown in fig. 6B, the visual interface may present a computational graph 60. The fusion control may include a second node selection control 66 and a second validation control 67. The calculation map 60 may include an initial input node In60, first to seventh calculation nodes N61 to N67, and an output node Out60. The above description of the calculation map 30 is equally applicable to the calculation map 60, and the disclosure is not repeated here. The following will explain in conjunction with operations S6211 to S6217.

In operation S6211, it is determined whether the second node selection control is triggered.

In an embodiment of the present disclosure, in response to determining that the second node selection control 66 of the fusion control is triggered, operation S6212 may be performed.

In an embodiment of the present disclosure, in response to determining that the second node selection control 66 of the fusion control is not triggered, operation S6217 may be performed waiting for the control to be triggered.

In operation S6212, a fusion node is generated.

For example, a fusion node may be generated. According to the user input information, node name information, node type information and node attribute information of the fusion node can be determined. The node type information 601 may include a node type "FusedOP". Node name information 602 may include a node name "custom_op". The node attribute information 605 may include node attributes entered by a user. After the plurality of target nodes are determined, node input information and node output information may be determined based on the user input information and the plurality of target nodes.

In operation S6213, a target node is determined.

For example, a node selected by the user may be a target node. As shown in fig. 6B, the first computing node N61 may be regarded as a target node.

In operation S6214, the node set to be deleted is updated.

In the embodiment of the present disclosure, the target node may be regarded as the node to be deleted. The first computing node N61 is added to the set of nodes to be deleted.

In operation S6215, it is determined whether the second confirmation control is triggered.

In response to determining that the first confirmation control is not triggered, in embodiments of the present disclosure, a return may be made to operation S6213, either to continue waiting for the user to select a node or to wait for the first confirmation control to be triggered. For example, in the case where the first confirmation control is not triggered, operations S6213 to S6215 may be performed a plurality of times in response to the user selecting the node. Taking the example of the user selecting the first computing node N61 to the seventh computing node N67, the user may trigger the second confirmation control after selecting the seventh computing node N67.

In the presently disclosed embodiments, in response to determining that the second confirmation control is triggered, operation S6216 may be performed.

In operation S6216, first operation data is determined.

For example, the first operation data of the first to seventh computing nodes N61 to N67 may indicate: deleting the first to seventh computing nodes N61 to N67 from the computing map 60; and adding the fusion nodes to the computational graph 60. The output of the initial input node In60 may be the input of the fusion node. The input of the output node Out60 may be the output of the fusion node. Next, target operation data of a plurality of target nodes may be determined based on the first operation data.

It will be appreciated that while the present disclosure has been described above in connection with at least one control, some ways of obtaining object model data will be described below.

In some embodiments, the model data to be edited is adjusted according to at least one target operation data, and the target model data can be obtained.

In the embodiment of the disclosure, according to target operation data of a target node, a target code corresponding to the target node is generated. And adjusting a model file (with a file format of pbtxt) of the model to be edited by using the target code to obtain a target model file (with a file format of pbtxt) serving as target model data. The following will describe the nodes to be added in conjunction with the above.

FIG. 7 is a schematic diagram of generating object code according to one embodiment of the present disclosure.

In the embodiment of the disclosure, node name information of the target node and the identification value of the corresponding preset operation may be determined according to the target operation data of the target node. The node name may be added to the node queue, and the corresponding identification value of the preset operation may be added to the operation queue. The node queue includes a plurality of node names and corresponding ending characters. The operation queue may include identification values and corresponding ending symbols for a plurality of preset operations. For example, the plurality of node names may respectively correspond to identification values of a plurality of preset operations. For another example, the plurality of node names may correspond to an identification value of one preset operation (a preset operation corresponding to the second node deletion control described above). The plurality of node names corresponding to the identification values of the preset operations may be indicated based on the ending symbol between the identification values of the different preset operations.

As shown in fig. 7, a plurality of node information of the node to be added may be determined according to the target operation data D78 of the node to be added. The plurality of node information includes node name information, node type information, node attribute information, node input information, and node output information. The node name information includes a node name "node_01". The node type information includes a node type "exp". The first node attribute information includes a first node attribute "int64" which may indicate that the data type supported by the node is a 64-bit integer. The second node attribute information includes a second node attribute "0.00001", and may indicate that the calculated deviation of the node is 0.00001. The node name "node_01" may be added to the node queue. According to the target operation data D78, and adding an identification value 'ADD' of the corresponding preset operation to the operation queue.

In the embodiment of the disclosure, the target operation data is utilized to generate the target code corresponding to the target node according to the node queue and the operation queue. As shown in fig. 7, code generation may be performed sequentially according to the node queue and the operation queue. When generating the object code of the node to be added, the object code C78 is generated using the object operation data D78. Object code C78 may include a plurality of code lines. The code line "new_node= NodeDef ()" may indicate: object code C78 corresponds to a node. The code line "new_node.op= 'Exp'" may correspond to the node type information of the node to be added. The code line "new_node.name= 'node_01'" may correspond to the node name information of the node to be added. The code line "new_node.attr [ 'T' ]. CopyFrom (attrDef (" int64 "))" may correspond to the above-described first node attribute information of the node to be added. The code line "new_node.attr [ 'bias' ]. F=0.00001" may correspond to the above-described second node attribute information of the node to be added. The code line "new_node.input.extension (name_to_node [" exp_01 ")" may correspond to the node input information of the node to be added described above. The code line "name_to_node [" abs_01 "]. Input=" node_01_out ". The code line" new_node.output.extension (' node_01_out ") may correspond to the node output information of the node to be added, and the code line" name_to_node [ "abs_01" ]. Input= "node_01_out" may represent that the input of the node with the node name "abs_01" is modified to the output of the node to be added, so as to establish the association relationship between the node to be added and the node with the node name "abs_01".

It will be appreciated that some ways of obtaining the target model data of the present disclosure are described above in connection with node addition controls, and some ways of obtaining the target model data will be described below in connection with first node deletion controls.

In the embodiment of the disclosure, the target model data can be obtained according to the target operation data of the target node and the model data to be edited. Taking the first computing node N41 as an example, in the case where the first computing node N41 is used as the target node and the first node deletion control is triggered, the target operation data of the first computing node N41 may be obtained according to the first operation data of the first computing node N41 and the second operation data of the second computing node N42. When the target operation data is used for adjusting the model data to be edited, the first computing node N41 can be deleted, and the node input of the second computing node N42 can be adjusted according to the user input information so as to obtain the target model data.

It will be appreciated that some of the ways in which the present disclosure obtains the target model data are described above in connection with the first node deletion control, and some of the ways in which the target model data are obtained are described below in connection with the second node deletion control.

In the embodiment of the disclosure, the target model data can be obtained according to the target operation data of the target node and the model data to be edited. Taking the fourth computing node N44 as an example, in the case where the fourth computing node N44 is the target node and the subsequent node deletion control 43 is triggered, the target operation data of the fourth computing node N44 may be obtained according to the first operation data of the fourth computing node N44. When the model data to be edited is adjusted by using the target operation data, the fourth calculation node N44, the fifth calculation node N45, the sixth calculation node N46, the seventh calculation node N47, and the output node Out40 may be deleted, and the third calculation node N43 may be used as a new output node to obtain the target model data.

It will be appreciated that some of the ways in which the present disclosure obtains the target model data are described above in connection with the second node delete control, and some of the ways in which the target model data are obtained are described below in connection with the node modify control.

In another embodiment of the present disclosure, the target model data may be obtained according to the target operation data of the target node and the model data to be edited. Taking the fourth computing node N44 as an example, in the case where the fourth computing node N44 is the target node and the node modification control is triggered, the target operation data of the fourth computing node N44 may be obtained according to the first operation data of the fourth computing node N44. When the target operation data is used for adjusting the model data to be edited, the node name of the fourth computing node N44 can be modified to be a target information value input by a user, so that the target model data can be obtained.

It will be appreciated that some of the ways in which the present disclosure obtains the target model data are described above in connection with node modification controls, and some of the ways in which the target model data are obtained will be described below in connection with computing subgraph extraction controls.

In another embodiment of the present disclosure, the target model data may be obtained according to the target operation data of the target node and the model data to be edited. Taking the fifth computing node N55 as an example, in the case where the fifth computing node N55 is used as a target node and the computation sub-graph extraction control is triggered, the target operation data of the fifth computing node N55 may be obtained according to the first operation data of the fifth computing node N55. When the target operation data is utilized to adjust the model data to be edited, a calculation subgraph can be generated according to the continuous association relationship among the initial input node In50, the first calculation node N51, the second calculation node N52, the third calculation node N53, the fourth calculation node N54, the fifth calculation node N55, the output node, the fifth calculation node N55 and the initial input node In 50. For example, model data corresponding to the calculation subgraph is added to the model data to be edited to obtain target model data. For another example, model data corresponding to the computation subgraph may be set as the target model data.

It will be appreciated that some ways of obtaining the target model data of the present disclosure are described above in connection with computing the subgraph extraction control, and some ways of obtaining the target model data are described below in connection with the fusion control.

In another embodiment of the present disclosure, the target model data may be obtained according to the target operation data of the target node and the model data to be edited. Taking the first to seventh computing nodes N61 to N67 as an example, in the case where the first to seventh computing nodes N61 to N67 are the target nodes and the fusion control is triggered, the target operation data of the first to seventh computing nodes N61 to N67 may be obtained according to the first operation data of the first to seventh computing nodes N61 to N67. When the model data to be edited is adjusted by using the target operation data, the first to seventh calculation nodes N61 to N67 may be deleted from the calculation map 60; and adding the fusion nodes to the computational graph 60 to obtain the target model data.

It will be appreciated that while the method of the present disclosure is described above, the apparatus of the present disclosure will be described below.

Fig. 8 is a block diagram of a model processing apparatus according to one embodiment of the present disclosure.

As shown in fig. 8, the apparatus 800 may include a presentation module 810, an acquisition module 820, and an adjustment module 830.

The display module 810 is configured to display a calculation map of the model to be processed on the visual interface according to the model data to be edited of the model to be processed. The computational graph includes M nodes. M is an integer greater than or equal to 1. The model to be processed corresponds to the first deep learning frame.

An acquisition module 820 for acquiring at least one target operation data via the visual interface. The target operational data is associated with N nodes in the computational graph. N is an integer greater than or equal to 1;

the adjusting module 830 is configured to adjust the model data to be edited according to at least one target operation data, so as to obtain target model data.

In some embodiments, the apparatus 800 may further include: and the determining module is used for determining a target model corresponding to the target deep learning framework according to the conversion protocol and the target model data.

In some embodiments, the presentation module comprises: and the first determining submodule is used for determining model data to be edited of the model to be processed according to the conversion protocol and the model to be processed corresponding to the first deep learning frame. And the display sub-module is used for displaying the calculation graph on the visual interface according to the model data to be edited.

In some embodiments, the visual interface also presents at least one control. The acquisition module comprises: and the second determining sub-module is used for determining the first operation data of the target node according to at least one of the user input information and the preset operation corresponding to the control in response to determining that the control is triggered. The target node is the node selected by the user. The first obtaining module is used for obtaining target operation data of the target node according to at least one first operation data of the target node.

In some embodiments, the at least one control comprises a node addition control. The node adding control corresponds to a plurality of preset nodes, the preset operation corresponding to the node adding control comprises adding operation, the user input information comprises at least one node to be associated related to the target node, and the at least one node to be associated is located in the calculation graph. The second determination submodule includes: the first acquisition unit is used for responding to the fact that the node adding control is triggered, and acquiring at least one node to be added selected by a user from a plurality of preset nodes as at least one target node. The first determining unit is used for determining first operation data of the target node according to the user input information and preset operation corresponding to the node adding control.

In some embodiments, the first obtaining submodule includes: the first obtaining unit is used for obtaining target operation data according to the first operation data. The target operational data is at least for indicating: adding at least one target node in the calculation graph; and establishing an association relationship between the target node and at least one node to be associated.

In some embodiments, the at least one control includes a first node deletion control, and the preset operation corresponding to the first node deletion control includes a deletion operation, and the target node is a node in the calculation graph. The second determination submodule includes: and the second determining unit is used for determining the first operation data according to the preset operation corresponding to the first node deleting control and at least one subsequent node associated with the target node in response to the fact that the first node deleting control is triggered. The first operation data is used for indicating: deleting at least one association relationship between the latter node and the target node and deleting the target node. And a third determining unit configured to determine at least one second operation data corresponding to the at least one subsequent node according to the user input information, wherein the second operation data is used to indicate an input of the subsequent node.

In some embodiments, the first obtaining submodule includes: and the second obtaining unit is used for obtaining target operation data according to the first operation data and at least one second operation data.

In some embodiments, the at least one control includes a second node deletion control, and the preset operation corresponding to the second node deletion control includes a deletion operation, where the target node is a node in the computation graph. The second determination submodule includes: and a fourth determining unit configured to determine at least one of a subsequent node set and a preceding node set associated with the target node in response to determining that the second node deletion control is triggered. And the fifth determining unit is used for determining the first operation data according to preset operations corresponding to at least one of the rear node set and the front node set, the target node and the second node deleting control. The first operation data is used for indicating: at least one of the set of following nodes and the set of preceding nodes and the target node are deleted.

In some embodiments, the at least one control comprises a node modification control, the target node being a node in the computational graph. The second determination submodule includes: and the display unit is used for displaying the node information of the target node in response to the fact that the node modification control is triggered. And a sixth determining unit, configured to use at least one node information selected by the user from the plurality of node information as at least one node information to be modified. And a seventh determining unit, configured to determine the first operation data according to the user input information and the at least one node information to be modified. The first operation data includes at least one target information value of node information to be modified.

In some embodiments, the at least one control includes a computation subgraph extraction control, the preset operation corresponding to the subgraph extraction control includes a computation graph generation operation, and the target node is a node in the computation graph. The second determination submodule includes: and the eighth determining unit is used for determining the first operation data according to at least one target node, the initial input node of the calculation graph and the preset operation corresponding to the calculation subgraph extraction control in response to the fact that the calculation subgraph extraction control is triggered. The first operation data is used for indicating: and generating a calculation subgraph according to at least one target node and the initial input node.

In some embodiments, the at least one control comprises a fusion control, and the target node is a node in the computational graph. The second determination submodule includes: and the ninth determining unit is used for determining the first operation data according to the user input information, the plurality of target nodes and the preset operation corresponding to the fusion control in response to determining that the fusion control is triggered. The first operation data is used for indicating: and fusing the plurality of target nodes into a fusion node.

It will be appreciated that while the apparatus of the present disclosure has been described above, an electronic device of the present disclosure will be described below.

Fig. 9 is a schematic diagram of an electronic device according to one embodiment of the present disclosure.

As shown in fig. 9, the apparatus 9000 may comprise a model processing means 900. The model processing device 900 may be the device 800 described above.

In the technical scheme of the disclosure, the related processes of collecting, storing, using, processing, transmitting, providing, disclosing and the like of the personal information of the user accord with the regulations of related laws and regulations, and the public order colloquial is not violated.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

Fig. 10 shows a schematic block diagram of an example electronic device 1000 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 10, the apparatus 1000 includes a computing unit 1001 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a Random Access Memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the device 1000 can also be stored. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other by a bus 1004. An input/output (I/O) interface 1005 is also connected to bus 1004.

Various components in device 1000 are connected to I/O interface 1005, including: an input unit 1006 such as a keyboard, a mouse, and the like; an output unit 1007 such as various types of displays, speakers, and the like; a storage unit 1008 such as a magnetic disk, an optical disk, or the like; and communication unit 1009 such as a network card, modem, wireless communication transceiver, etc. Communication unit 1009 allows device 1000 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunications networks.

The computing unit 1001 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 1001 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs the respective methods and processes described above, for example, the model processing method. For example, in some embodiments, the model processing method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 1008. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 1000 via ROM 1002 and/or communication unit 1009. When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the model processing method described above may be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to perform the model processing method in any other suitable way (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) display or an LCD (liquid crystal display)) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel, sequentially, or in a different order, provided that the desired results of the disclosed aspects are achieved, and are not limited herein.

The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims (28)

1. A model processing method, comprising:

displaying a calculation map of the model to be processed on a visual interface according to the model data to be edited of the model to be processed, wherein the calculation map comprises M nodes, M is an integer greater than or equal to 1, and the model to be processed corresponds to a first deep learning frame;

Acquiring at least one target operation data through the visual interface, wherein the target operation data are related to N nodes in the calculation graph, and N is an integer greater than or equal to 1;

and adjusting the model data to be edited according to the at least one target operation data to obtain target model data.

2. The method of claim 1, further comprising:

and determining a target model corresponding to the target deep learning framework according to the conversion protocol and the target model data.

3. The method of claim 2, wherein the presenting the computational graph of the model to be processed at the visual interface according to the model to be edited data of the model to be processed comprises:

Determining the model data to be edited of the model to be processed according to a conversion protocol and the model to be processed corresponding to a first deep learning frame;

And displaying the calculation graph on the visual interface according to the model data to be edited.

4. The method of claim 1, wherein the visual interface further presents at least one control,

The obtaining, via the visual interface, at least one target operation data includes:

Responding to the fact that the control is triggered, and determining first operation data of a target node according to at least one of user input information and preset operation corresponding to the control, wherein the target node is a node selected by a user;

and obtaining the target operation data of the target node according to at least one first operation data of the target node.

5. The method of claim 4, wherein the at least one control comprises a node addition control, the node addition control corresponding to a plurality of preset nodes, the preset operation corresponding to the node addition control comprising an addition operation, the user input information comprising at least one node to be associated with the target node, the at least one node to be associated being located in the computational graph;

wherein, in response to determining that the control is triggered, determining, according to at least one of user input information and a preset operation corresponding to the control, first operation data of the target node includes:

responding to the fact that a node adding control is triggered, and acquiring at least one node to be added selected by the user from the plurality of preset nodes as the at least one target node;

And determining first operation data of the target node according to the user input information and preset operation corresponding to the node adding control.

6. The method of claim 5, wherein the obtaining the target operation data of the target node from at least one first operation data of the target node comprises:

Obtaining the target operation data according to the first operation data, wherein the target operation data is at least used for indicating: adding the at least one target node in the computational graph; and establishing an association relationship between the target node and the at least one node to be associated.

7. The method of claim 4, wherein the at least one control comprises a first node deletion control, the preset operation corresponding to the first node deletion control comprises a deletion operation, the target node is a node in the computational graph,

Wherein, in response to determining that the control is triggered, determining, according to at least one of user input information and a preset operation corresponding to the control, first operation data of a target node includes:

In response to determining that the first node deletion control is triggered, determining the first operation data according to a preset operation corresponding to the first node deletion control and at least one following node associated with the target node, wherein the first operation data is used for indicating: deleting the association relationship between the at least one subsequent node and the target node, and deleting the target node;

and determining at least one second operation data corresponding to the at least one subsequent node according to the user input information, wherein the second operation data is used for indicating the input of the subsequent node.

8. The method of claim 7, wherein the obtaining the target operation data of the target node from at least one first operation data of the target node comprises:

And obtaining the target operation data according to the first operation data and the at least one second operation data.

9. The method of claim 4, wherein the at least one control comprises a second node deletion control, the preset operation corresponding to the second node deletion control comprises a deletion operation, the target node is a node in the computational graph,

Wherein, in response to determining that the control is triggered, determining, according to at least one of user input information and a preset operation corresponding to the control, first operation data of the target node includes:

Responsive to determining that the second node delete control is triggered, determining at least one of a set of subsequent nodes and a set of preceding nodes associated with the target node;

Determining the first operation data according to preset operations corresponding to at least one of the following node set and the preceding node set, the target node and the second node deletion control, wherein the first operation data is used for indicating: and deleting at least one of the following node set and the preceding node set and the target node.

10. The method of claim 4, wherein the at least one control comprises a node modification control, the target node is a node in the computational graph,

The determining, in response to determining that the control is triggered, first operation data of the target node according to at least one of user input information and a preset operation corresponding to the control includes:

Responsive to determining that the node modification control is triggered, displaying a plurality of node information for the target node;

at least one node information selected by the user in the plurality of node information is used as at least one node information to be modified;

And determining the first operation data according to the user input information and the at least one node information to be modified, wherein the first operation data comprises a target information value of the at least one node information to be modified.

11. The method of claim 4, wherein the at least one control comprises a computational subgraph extraction control, the preset operations corresponding to the subgraph extraction control comprise a computational graph generation operation, the target node is a node in the computational graph,

The determining, in response to determining that the control is triggered, first operation data of the target node according to at least one of user input information and a preset operation corresponding to the control includes:

In response to determining that the computational subgraph extraction control is triggered, determining the first operation data according to at least one target node, an initial input node of the computational graph and a preset operation corresponding to the computational subgraph extraction control, wherein the first operation data is used for indicating: and generating a calculation subgraph according to at least one target node and the initial input node.

12. The method of claim 4, wherein the at least one control comprises a fusion control, the target node is a node in the computational graph,

The determining, in response to determining that the control is triggered, first operation data of the target node according to at least one of user input information and a preset operation corresponding to the control includes:

in response to determining that the fusion control is triggered, determining the first operation data according to the user input information, a plurality of target nodes and preset operations corresponding to the fusion control, wherein the first operation data is used for indicating: and fusing a plurality of target nodes into a fusion node.

13. A model processing apparatus comprising:

The display module is used for displaying a calculation graph of the to-be-processed model on a visual interface according to the to-be-edited model data of the to-be-processed model, wherein the calculation graph comprises M nodes, M is an integer greater than or equal to 1, and the to-be-processed model corresponds to a first deep learning frame;

The acquisition module is used for acquiring at least one target operation data through the visual interface, wherein the target operation data are related to N nodes in the calculation graph, and N is an integer greater than or equal to 1;

And the adjusting module is used for adjusting the model data to be edited according to the at least one target operation data to obtain target model data.

14. The apparatus of claim 13, further comprising:

and the determining module is used for determining a target model corresponding to the target deep learning frame according to the conversion protocol and the target model data.

15. The method of claim 2, wherein the presentation module comprises:

The first determining submodule is used for determining the model data to be edited of the model to be processed according to a conversion protocol and the model to be processed corresponding to the first deep learning frame;

And the display sub-module is used for displaying the calculation graph on the visual interface according to the model data to be edited.

16. The apparatus of claim 13, wherein the visual interface further presents at least one control,

The acquisition module comprises:

The second determining submodule is used for responding to the fact that the control is triggered, and determining first operation data of a target node according to at least one of user input information and preset operation corresponding to the control, wherein the target node is a node selected by a user;

The first obtaining module is used for obtaining the target operation data of the target node according to at least one first operation data of the target node.

17. The apparatus of claim 16, wherein the at least one control comprises a node addition control, the node addition control corresponding to a plurality of preset nodes, the preset operation corresponding to the node addition control comprising an addition operation, the user input information comprising at least one node to be associated with the target node, the at least one node to be associated being located in the computational graph;

wherein the second determining submodule includes:

the first acquisition unit is used for responding to the fact that the node adding control is triggered, and acquiring at least one node to be added selected by the user from the plurality of preset nodes as the at least one target node;

and the first determining unit is used for determining first operation data of the target node according to the user input information and preset operation corresponding to the node adding control.

18. The apparatus of claim 17, wherein the first obtaining submodule comprises:

The first obtaining unit is configured to obtain the target operation data according to the first operation data, where the target operation data is at least used to indicate: adding the at least one target node in the computational graph; and establishing an association relationship between the target node and the at least one node to be associated.

19. The apparatus of claim 16, wherein the at least one control comprises a first node deletion control, the preset operation corresponding to the first node deletion control comprises a deletion operation, the target node is a node in the computational graph,

Wherein the second determining submodule includes:

The second determining unit is used for determining the first operation data according to preset operation corresponding to the first node deleting control and at least one following node associated with the target node in response to the fact that the first node deleting control is triggered, wherein the first operation data are used for indicating: deleting the association relationship between the at least one subsequent node and the target node, and deleting the target node;

And a third determining unit configured to determine at least one second operation data corresponding to the at least one subsequent node according to the user input information, where the second operation data is used to indicate the input of the subsequent node.

20. The apparatus of claim 19, wherein the first obtaining submodule comprises:

and a second obtaining unit, configured to obtain the target operation data according to the first operation data and the at least one second operation data.

21. The apparatus of claim 16, wherein the at least one control comprises a second node deletion control, the preset operation corresponding to the second node deletion control comprises a deletion operation, the target node is a node in the computational graph,

Wherein the second determining submodule includes:

a fourth determining unit configured to determine at least one of a subsequent node set and a preceding node set associated with the target node in response to determining that the second node deletion control is triggered;

a fifth determining unit, configured to determine the first operation data according to a preset operation corresponding to at least one of the following node set and the preceding node set, the target node, and the second node deletion control, where the first operation data is used to indicate: and deleting at least one of the following node set and the preceding node set and the target node.

22. The apparatus of claim 16, wherein the at least one control comprises a node modification control, the target node being a node in the computational graph,

The second determination submodule includes:

the display unit is used for displaying a plurality of node information of the target node in response to determining that the node modification control is triggered;

A sixth determining unit, configured to use at least one node information selected by the user from the plurality of node information as at least one node information to be modified;

And a seventh determining unit, configured to determine the first operation data according to the user input information and at least one node information to be modified, where the first operation data includes a target information value of the at least one node information to be modified.

23. The apparatus of claim 16, wherein the at least one control comprises a computational subgraph extraction control, the preset operation corresponding to the subgraph extraction control comprises a computational graph generation operation, the target node is a node in the computational graph,

The second determination submodule includes:

an eighth determining unit, configured to determine, in response to determining that the computation subgraph extraction control is triggered, the first operation data according to at least one of the target node, an initial input node of the computation graph, and a preset operation corresponding to the computation subgraph extraction control, where the first operation data is used to indicate: and generating a calculation subgraph according to at least one target node and the initial input node.

24. The apparatus of claim 16, wherein the at least one control comprises a fusion control, the target node is a node in the computational graph,

The second determination submodule includes:

A ninth determining unit, configured to determine, in response to determining that the fusion control is triggered, the first operation data according to the user input information, the plurality of target nodes, and preset operations corresponding to the fusion control, where the first operation data is used to indicate: and fusing a plurality of target nodes into a fusion node.

25. An electronic device comprising the apparatus of any one of claims 13 to 24.

26. An electronic device, comprising:

At least one processor; and

A memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 12.

27. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1 to 12.

28. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 12.