CN112069456A - Model file generation method and device, electronic equipment and storage medium - Google Patents
Model file generation method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112069456A CN112069456A CN202010997489.XA CN202010997489A CN112069456A CN 112069456 A CN112069456 A CN 112069456A CN 202010997489 A CN202010997489 A CN 202010997489A CN 112069456 A CN112069456 A CN 112069456A
- Authority
- CN
- China
- Prior art keywords
- operator
- model
- operators
- generating
- general expression
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 62
- 230000014509 gene expression Effects 0.000 claims abstract description 87
- 230000006870 function Effects 0.000 claims description 55
- 230000015654 memory Effects 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000010801 machine learning Methods 0.000 description 14
- 230000008569 process Effects 0.000 description 9
- 238000011144 upstream manufacturing Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000007781 pre-processing Methods 0.000 description 3
- 238000012417 linear regression Methods 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/15—Correlation function computation including computation of convolution operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Mathematical Optimization (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computational Mathematics (AREA)
- Mathematical Analysis (AREA)
- Pure & Applied Mathematics (AREA)
- Medical Informatics (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Algebra (AREA)
- Databases & Information Systems (AREA)
- Stored Programmes (AREA)
Abstract
The application provides a method and a device for generating a model file, electronic equipment and a storage medium, wherein the method for generating the model file comprises the following steps: obtaining a model function corresponding to a target model; wherein the model function comprises a plurality of operators; converting each operator in the model function into a first generic expression; wherein the first generic expression comprises a plurality of operator attributes for the operator; determining an incidence relation between any two operators in the plurality of operators according to the input information and the output information corresponding to each operator; and generating a model file corresponding to the target model according to the first general expression corresponding to each operator and the incidence relation between any two operators in the plurality of operators. According to the method and the device, a plurality of algorithms in the model can be uniformly expressed, and the efficiency of model engineering is improved.
Description
Technical Field
The application relates to the technical field of computer information, in particular to a method and a device for generating a model file, electronic equipment and a storage medium.
Background
At present, both the conventional machine learning method and the emerging deep learning method involve a large number of heterogeneous frameworks and algorithms, such as a linear regression algorithm supported by a sklern tool, a tree regression algorithm supported by an XG boost, a tree regression algorithm supported by a light GBM, and the like.
In practice, each model may include multiple algorithms, and each algorithm in the model may correspond to a different machine learning framework, and this kind of multi-framework multi-algorithm model improves the complexity of model engineering and reduces the efficiency of model engineering, so unifying the expression ways of multiple algorithms in the model is a technical problem to be solved urgently at present.
Disclosure of Invention
In view of this, an object of the embodiments of the present application is to provide a method and an apparatus for generating a model file, an electronic device, and a storage medium, which can uniformly express a plurality of algorithms in a model and improve efficiency of model engineering.
In a first aspect, an embodiment of the present application provides a method for generating a model file, where the method includes:
obtaining a model function corresponding to a target model; wherein the model function comprises a plurality of operators;
converting each operator in the model function into a first generic expression; wherein the first generic expression comprises a plurality of operator attributes for the operator;
determining an incidence relation between any two operators in the plurality of operators according to the input information and the output information corresponding to each operator;
and generating a model file corresponding to the target model according to the first general expression corresponding to each operator and the incidence relation between any two operators in the plurality of operators.
In one possible embodiment, the converting each operator in the model function into a first general expression includes:
and aiming at each operator in the model function, determining a plurality of operator attributes corresponding to the operator according to the category of the operator, and converting the operator into a first general expression comprising the operator attributes.
In one possible embodiment, the class to which each operator belongs is determined by:
searching the storage address of each operator in the model function;
and determining the class matched with the search result as the class to which the operator belongs based on the search result of the storage address.
In a possible implementation manner, the generating a model file corresponding to the target model according to the first general expression corresponding to each operator and the association relationship between any two operators in the plurality of operators includes:
generating a second general expression which comprises a plurality of model attributes and corresponds to the model function based on a plurality of preset model attributes;
and generating a model file corresponding to the target model according to the first general expression corresponding to each operator, the second general expression corresponding to the model function and the incidence relation between any two operators in the plurality of operators.
In a possible implementation, the category to which the operator belongs includes: self-defining operators and framework semantic operators.
In a possible implementation manner, the generating a model file corresponding to the target model according to the first general expression corresponding to each operator and the association relationship between any two operators in the plurality of operators includes:
determining a combination mode corresponding to the operators according to the incidence relation between any two operators in the operators;
and generating a model file corresponding to the target model according to the first general expression corresponding to each operator and the combination mode corresponding to the operators.
In a possible implementation manner, the corresponding combination manner of the plurality of operators includes: sequential execution, one-to-many execution, many-to-many execution.
In a second aspect, an embodiment of the present application provides a device for generating a model file, where the device includes:
the acquisition module is used for acquiring a model function corresponding to the target model; wherein the model function comprises a plurality of operators;
the conversion module is used for converting each operator in the model function into a first general expression; wherein the first generic expression comprises a plurality of operator attributes for the operator;
the first determining module is used for determining the incidence relation between any two operators in the plurality of operators according to the input information and the output information corresponding to each operator;
and the generating module is used for generating a model file corresponding to the target model according to the first general expression corresponding to each operator and the incidence relation between any two operators in the plurality of operators.
In a third aspect, an embodiment of the present application provides an electronic device, including: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, when the electronic device runs, the processor and the memory communicate through the bus, and the processor executes the machine-readable instructions to execute the steps of the model file generation method according to any one of the first aspect.
In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to execute the steps of the model file generation method according to any one of the first aspect.
The method, the device, the electronic equipment and the storage medium for generating the model file are used for acquiring a model function corresponding to a target model; wherein the model function comprises a plurality of operators; converting each operator in the model function into a first generic expression; wherein the first generic expression comprises a plurality of operator attributes for the operator; determining an incidence relation between any two operators in the plurality of operators according to the input information and the output information corresponding to each operator; and generating a model file corresponding to the target model according to the first general expression corresponding to each operator and the incidence relation between any two operators in the plurality of operators. According to the embodiment of the application, a plurality of algorithms in the model can be uniformly expressed, and the efficiency of model engineering is improved.
In order to make the aforementioned objects, features and advantages of the present application more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.
FIG. 1 is a flowchart illustrating a method for generating a model file according to an embodiment of the present disclosure;
FIG. 2 is a flow chart illustrating another method for generating a model file according to an embodiment of the present disclosure;
FIG. 3 is a flow chart illustrating another method for generating a model file according to an embodiment of the present disclosure;
FIG. 4 is a flow chart illustrating another method for generating a model file according to an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram illustrating a device for generating a model file according to an embodiment of the present application;
fig. 6 shows a schematic diagram of an electronic device provided in an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.
At present, both the conventional machine learning method and the emerging deep learning method involve a large number of heterogeneous frameworks and algorithms, such as a linear regression algorithm supported by a sklern tool, a tree regression algorithm supported by an XG boost, a tree regression algorithm supported by a light GBM, and the like. In practice, each model may include multiple algorithms, and each algorithm in the model may correspond to a different machine learning framework, and this kind of multi-framework multi-algorithm model improves the complexity of model engineering and reduces the efficiency of model engineering, so unifying the expression ways of multiple algorithms in the model is a technical problem to be solved urgently at present.
Based on the above problems, the method, the device, the electronic device and the storage medium for generating the model file provided by the embodiment of the application obtain the model function corresponding to the target model; wherein the model function comprises a plurality of operators; converting each operator in the model function into a first generic expression; wherein the first generic expression comprises a plurality of operator attributes for the operator; determining an incidence relation between any two operators in the plurality of operators according to the input information and the output information corresponding to each operator; and generating a model file corresponding to the target model according to the first general expression corresponding to each operator and the incidence relation between any two operators in the plurality of operators. According to the embodiment of the application, a plurality of algorithms in the model can be uniformly expressed, and the efficiency of model engineering is improved.
The above-mentioned drawbacks are the results of the inventor after practical and careful study, and therefore, the discovery process of the above-mentioned problems and the solution proposed by the present application to the above-mentioned problems in the following should be the contribution of the inventor to the present application in the process of the present application.
The technical solutions in the present application will be described clearly and completely with reference to the drawings in the present application, and it should be understood that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the present application, as generally described and illustrated in the figures herein, may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
Referring to fig. 1, fig. 1 is a flowchart of a method for generating a model file according to an embodiment of the present application, where the method includes the following steps:
s101, obtaining a model function corresponding to a target model; wherein the model function comprises a plurality of operators.
In this embodiment of the application, the target model corresponds to a model function, and functions of the target model can be realized based on the model function, for example, classifying articles, recommending articles, and the like, where the model function includes a plurality of operators, where the plurality of operators is at least one operator, in practice, the operators are algorithms, the operators correspond to two categories, and the categories to which the operators belong include: the self-defined operator and the frame semantic operator are respectively an algorithm corresponding to the organic machine learning frame, the self-defined operator is a user-defined algorithm, the self-defined operator does not depend on the machine learning frame, and each machine learning frame can correspond to a plurality of operators.
For example, the operators included in the target model are: the method comprises a first operator, a second operator, a third operator and a fourth operator, wherein the first operator is a preprocessing module preprocessing.scale in a machine learning frame Sklean, original data R1 are changed into a vector T1 based on the first operator, the second operator is a user-defined algorithm A1, the second operator processes the T1 to adapt to subsequent operations, T2 is obtained, the third operator is a PyTorch deep learning model, the third operator fits the T2 to obtain output data S1 of a first layer, the fourth operator is an LGBMClassifier in a machine learning frame lightGBM, the fourth operator processes the S1 to obtain a final output U1, and the target model performs correlation processing on the original data R1 through a plurality of algorithms to obtain a final processing result U1.
S102, converting each operator in the model function into a first general expression; wherein the first generic expression comprises a plurality of operator attributes for the operator.
In this embodiment of the present application, an operator in a target model may correspond to an organic device learning frame, or may not depend on a machine learning frame, and machine learning frames corresponding to different operators may be different, and in order to uniformly express each operator in the target model, each operator is converted into a first general expression, where the first general expression of each operator is composed of a plurality of operator attributes of the operator, where the operator attributes include: the method comprises the following steps of obtaining an algorithm name, an algorithm version, an algorithm implementation language, a version of the implementation language, an algorithm implementation frame, a version of the algorithm implementation frame, a name corresponding to the algorithm in the frame, parameters needed by the algorithm during initialization, parameter names needed by the algorithm during calling, source codes corresponding to the language (the algorithm implementation language), a source file name and the like.
S103, determining the incidence relation between any two operators in the plurality of operators according to the input information and the output information corresponding to each operator.
In the embodiment of the application, each operator corresponds to input information and output information, wherein the input information is a source of data to be processed of the operator, the output information is a destination of the data obtained by processing of the operator, and for each operator, according to the input information and the output information of the operator, an association relationship between the operator and other operators except the operator in a target model can be determined.
For example, the target model includes a first operator, a second operator, a third operator, a fourth operator, and a fifth operator, where it may be determined that data to be processed of the second operator comes from the first operator according to input information and output information of the second operator, the data obtained by processing by the second operator is sent to the third operator and the fourth operator, and then it may be determined that first data obtained by processing by the first operator is sent to the second operator, the second operator processes the first data to obtain second data, and then the second data is sent to the third operator and the fourth operator, respectively, and the second operator is unrelated to the fifth operator, where the first operator is determined as an upstream operator of the second operator, and the third operator and the fourth operator are determined as downstream operators of the second operator, respectively.
S104, generating a model file corresponding to the target model according to the first general expression corresponding to each operator and the incidence relation between any two operators in the operators.
In the embodiment of the application, a first general expression corresponding to each operator in a target model is determined, an incidence relation between any two operators in a plurality of operators included in the target model is determined, and then the first general expressions corresponding to each operator are combined based on the incidence relation to obtain a model file corresponding to the target model.
It should be noted that the model file corresponding to the target model is an S-expression, and the S-expression is used as both code and data in Lisp. The S-expression, also known as the S-operator, or, sexp, where "S" stands for "symbolic", refers to a convention for expressing semi-structured data in human-readable text form. The S-expression is known for its use in the Lisp family of programming languages. Other applications are found in languages derived from Lisp, such as DSSSL, and CBCL as a marker presentation and john mecanitin in communication protocols such as IMAP. Although the details of the syntax and the types of data supported vary from language to language, the most common property between these languages is the use of S-expressions as bracketed prefix representations (and sometimes Cambridge Poland representations).
The method for generating the model file can uniformly express a plurality of algorithms in the model, and further improves the efficiency of model engineering.
Further, in the method for generating a model file provided in the embodiment of the present application, converting each operator in the model function into a first general expression includes:
and aiming at each operator in the model function, determining a plurality of operator attributes corresponding to the operator according to the category of the operator, and converting the operator into a first general expression comprising the operator attributes.
In the embodiment of the present application, the attributes of operators corresponding to different classes of operators are different, and the classes to which the operators belong include: the user-defined operator and the frame semantic operator specifically include the operator attributes corresponding to the frame semantic operator: the method comprises the following steps of (1) obtaining an algorithm name (name), an algorithm version (version), an algorithm implementation language (language), an implementation language version (language-version), an algorithm implementation frame (frame), an algorithm implementation frame version (frame-version), an algorithm-inner-name corresponding to the algorithm in the frame, (parameters) needed by the algorithm during initialization, and parameter names (args) needed by the algorithm during calling; the operator attribute corresponding to the self-defined operator comprises: the method comprises the following steps of algorithm name (name), algorithm version (version), algorithm implementation language (language), implementation language version (language-version), parameter name (args) required to be transmitted when the algorithm is called, source code (code) corresponding to the language, and source-file-name.
Wherein the algorithm version defaults to the v1 version; the algorithm implementation language is defaulted to Python implementation and can also comprise options such as java, c + + and the like so as to facilitate access to different algorithm frameworks; the implementation framework of the algorithm comprises machine learning frameworks such as sklern, tensorflow, pyrrch and the like; the parameter name required to be transmitted in the calling process of the algorithm is expressed by list, and is output corresponding to the list generated by the upstream operator, for example, the first operator is the upstream operator of the second operator, and the data obtained by processing the first operator is the data to be processed by the second operator; the source code corresponding to the language is processed by the language, the compiling language needs to be compiled first, and the interpreting language can be directly interpreted and run; and if the source file name is a self-defined algorithm and the item is not nil, the content corresponding to the code is placed into the file name corresponding to the source-file-name for compiling, and some compiling languages have special requirements on the file name.
Specifically, when the computer server determines the first general expression corresponding to each operator, the corresponding processing flow is as follows: reading a model function corresponding to a target model, reading next operation in the model function, judging the category of an operator corresponding to the operation, if the operator is a user-defined operator, generating a first general expression based on a plurality of operator attributes corresponding to the user-defined operator, and writing the first general expression into a result set; and if the operator is a frame semantic operator, generating a first general expression based on a plurality of operator attributes corresponding to the frame semantic operator, and writing the first general expression into the result set until all the steps of operation in the model function are read in to obtain the first general expression corresponding to each operator in the target model.
The method for generating the model file provided by the embodiment of the application is based on the first general expression, the frame semantic operators of the heterogeneous frame are represented in a self-description mode, the self-defined operators are directly described by the codes of the self-defined operators, algorithms of various frames can be represented by means of the characteristic of the first general expression that data is codes, the self-defined algorithms are supported, various traditional machine learning frames and deep learning frames and different versions of the traditional machine learning frames and the deep learning frames are supported, a mixed frame is supported, the first general expression is represented by a pure text, the model file can be directly viewed and modified without special software, and the engineering efficiency of the target model is improved.
Further, referring to fig. 2, in the method for generating a model file provided in the embodiment of the present application, the category to which each operator belongs is determined as follows:
s201, searching the storage address of each operator in the model function.
In the embodiment of the application, because the position for storing the custom operator is different from the position for storing the frame semantic operator, the type of the operator can be distinguished based on the storage address corresponding to each operator, specifically, the frame semantic operator corresponds to the storage address, and the custom operator is only stored in the memory, so that the storage address corresponding to the custom operator is empty.
S202, based on the search result of the storage address, determining the category matched with the search result as the category to which the operator belongs.
In the embodiment of the application, the search result of the storage address comprises two conditions of 'having a storage address' and 'having a storage address being empty', wherein the 'having a storage address' is matched with the frame semantic operator, and the 'having a storage address being empty' is matched with the custom operator, for each operator, if the search result of the operator is 'having a storage address', the operator is determined as the frame semantic operator, and if the search result of the operator is 'having a storage address being empty', the operator is determined as the custom operator.
Further, referring to fig. 3, in the method for generating a model file provided in the embodiment of the present application, the generating a model file corresponding to the target model according to the first general expression corresponding to each operator and the association relationship between any two operators in the multiple operators includes:
s301, generating a second general expression which comprises a plurality of model attributes and corresponds to the model function based on a plurality of preset model attributes.
In the embodiment of the present application, the model file is a structure, and is composed of a model name (name0), a model version (version0), an author (author0), a brief description of the model (descriptor), an input parameter pattern (inputs) of the model, an output parameter pattern (outputs) of the model, and a first general expression (operator) corresponding to each operator.
Wherein, model version, default v1 version; the input parameter pattern of the model, which is represented in the form of list since there may be multiple input parameters; the output pattern of the model, again, is represented here in the form of list, since there may be multiple output parameters; the operator may be a single algorithm or a combination of multiple operators, that is, the target model may include one operator, or may include two or more operators.
Here, the model attributes of the object model include: the model name (name0), the model version (version0), the author (author0), the brief description (descriptor) of the model, the input parameter style (inputs) of the model, and the output parameter style (outputs) of the model, and based on the plurality of model attributes, a second general expression including the plurality of model attributes corresponding to the model function is generated.
S302, generating a model file corresponding to the target model according to the first general expression corresponding to each operator, the second general expression corresponding to the model function and the incidence relation between any two operators in the plurality of operators.
In the embodiment of the application, a first general expression corresponding to each operator in a target model, a second general expression corresponding to a model function, and an incidence relation between any two operators in a plurality of operators included in the target model are respectively determined, and a model file corresponding to the target model is obtained based on the second general expression, the incidence relation between the operators, and the first general expression corresponding to each operator.
Further, referring to fig. 4, in the method for generating a model file provided in the embodiment of the present application, the generating a model file corresponding to the target model according to the first general expression corresponding to each operator and the association relationship between any two operators in the multiple operators includes:
s401, determining a combination mode corresponding to the operators according to the incidence relation between any two operators in the operators.
In this embodiment of the present application, the combination manner corresponding to the multiple operators includes: the method comprises the following steps of sequential execution, one-to-many execution and many-to-many execution, wherein when the sequential execution is carried out, each operator in a target model corresponds to a unique upstream operator and a unique downstream operator, and each operator is sequentially executed according to the sequence of the operators; one-to-many execution, namely, the result of the first operator is copied to all the following operators; and performing many-to-many, namely uniformly inputting the result of each operator from the second operator to the last operator to the first operator.
According to the incidence relation between any two operators in the operators, the upstream operator and the downstream operator corresponding to each operator can be known, and if the operators correspond to a plurality of upstream operators or the operators do not correspond to a plurality of downstream operators, the combination mode corresponding to the operators is determined to be sequential execution; if the downstream operator of one operator is all operators except the operator in the target model, determining that the combination mode corresponding to the operators is 'one-to-many execution'; and if the upstream operator of one operator is all operators except the operator in the target model, determining that the combination mode corresponding to the operators is 'many-to-many execution'.
S402, generating a model file corresponding to the target model according to the first general expression corresponding to each operator and the combination mode corresponding to the operators.
In the embodiment of the application, the first general expressions corresponding to each operator in the target model are combined based on the combination mode corresponding to the plurality of operators in the target model, and the model file corresponding to the target model is obtained.
Corresponding to the target model exemplified in step 101, the model file corresponding to the target model is:
(make-s-modle:name’p1
:author’xiaoming
:vertion’v1
:inputs(list“[-1,15]”)
:outputs(list“[1]”)
:operator(pipeline
(make-operator:name’preprocessing
:language’python
:framework’sklearn
:framework-vertion“0.22”
:operator-inner-name’preproccessing.scale
:args(R1))
(make-operator:name’A1
:code“x+3”)
(make-operator:framework’pytorch
:framework-vertion“1.3.1”
:operator-inner-name’nn.Linear
:parameters(list 3 4))
(make-operator:framework’lightgbm
:framework-vertion“2.3.0”
:operator-inner-name’LGBMClassifier)))
the combination mode corresponding to a plurality of operators in the target model is "pipeline", namely, sequential execution, the first general expression corresponding to the first operator in the target model is the content before the name 'A1 (without the content) of the make-operator, the first general expression corresponding to the second operator is the content before the name' A1 of the make-operator, the content before the frame 'catalog of the make-operator, the first general expression corresponding to the third operator is the content after the frame' catalog of the make-operator, the content before the frame 'light bm of the make-operator, and the first general expression corresponding to the fourth operator is the content after the frame' light catalog of the make-operator.
The method for generating the model file provided by the embodiment of the application supports free combination of various user-defined operators and frame semantic operators, and can combine the first general expressions corresponding to each operator into a new general expression after determining the combination modes corresponding to the operators, which is a recursive process. By means of operator combination, functions of 'combination' and 'recursion' in programming language can be given to the target model, and therefore the target model with high complexity and high flexibility can be created.
Based on the same inventive concept, the embodiment of the present application further provides a device for generating a model file corresponding to the method for generating a model file, and since the principle of solving the problem of the device in the embodiment of the present application is similar to the method for generating a model file described above in the embodiment of the present application, the implementation of the device may refer to the implementation of the method, and repeated details are omitted.
Referring to fig. 5, fig. 5 is a schematic structural diagram of a device for generating a model file according to an embodiment of the present application, where the device includes:
an obtaining module 501, configured to obtain a model function corresponding to a target model; wherein the model function comprises a plurality of operators;
a conversion module 502, configured to convert each operator in the model function into a first general expression; wherein the first generic expression comprises a plurality of operator attributes for the operator;
a first determining module 503, configured to determine, according to the input information and the output information corresponding to each operator, an association relationship between any two operators in the multiple operators;
a generating module 504, configured to generate a model file corresponding to the target model according to the first general expression corresponding to each operator and the association relationship between any two operators in the multiple operators.
In one possible implementation, the converting module 502, when converting each operator in the model function into the first general expression, includes:
and aiming at each operator in the model function, determining a plurality of operator attributes corresponding to the operator according to the category of the operator, and converting the operator into a first general expression comprising the operator attributes.
In a possible implementation manner, the generating device of the model file further includes:
the searching module is used for searching the storage address of each operator in the model function;
and the second determining module is used for determining the category matched with the search result as the category to which the operator belongs based on the search result of the storage address.
In a possible implementation manner, when generating the model file corresponding to the target model according to the first general expression corresponding to each operator and the association relationship between any two operators in the plurality of operators, the generating module 504 includes:
generating a second general expression which comprises a plurality of model attributes and corresponds to the model function based on a plurality of preset model attributes;
and generating a model file corresponding to the target model according to the first general expression corresponding to each operator, the second general expression corresponding to the model function and the incidence relation between any two operators in the plurality of operators.
In a possible implementation, the category to which the operator belongs includes: self-defining operators and framework semantic operators.
In a possible implementation manner, when the generating module 504 generates the model file corresponding to the target model according to the first general expression corresponding to each operator and the association relationship between any two operators in the multiple operators, the generating module further includes:
determining a combination mode corresponding to the operators according to the incidence relation between any two operators in the operators;
and generating a model file corresponding to the target model according to the first general expression corresponding to each operator and the combination mode corresponding to the operators.
In a possible implementation manner, the corresponding combination manner of the plurality of operators includes: sequential execution, one-to-many execution, many-to-many execution.
The device for generating the model file provided by the embodiment of the application can uniformly express a plurality of algorithms in the model, and further improves the efficiency of model engineering.
Referring to fig. 6, fig. 6 is an electronic device 600 provided in an embodiment of the present application, where the electronic device 600 includes: a processor 601, a memory 602 and a bus, wherein the memory 602 stores machine-readable instructions executable by the processor 601, when the electronic device runs, the processor 601 and the memory 602 communicate with each other through the bus, and the processor 601 executes the machine-readable instructions to execute the steps of the method for generating the model file.
Specifically, the memory 602 and the processor 601 can be general-purpose memories and processors, which are not limited to the specific examples, and the model file generation method can be executed when the processor 601 runs a computer program stored in the memory 602.
Corresponding to the method for generating the model file, an embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and the computer program is executed by a processor to perform the steps of the method for generating the model file.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical division, and there may be other divisions in actual implementation, and for example, a plurality of modules or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or modules through some communication interfaces, and may be in an electrical, mechanical or other form.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present application may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present application, and are used for illustrating the technical solutions of the present application, but not limiting the same, and the scope of the present application is not limited thereto, and although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope disclosed in the present application; such modifications, changes or substitutions do not depart from the spirit and scope of the exemplary embodiments of the present application, and are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A method for generating a model file, the method comprising:
obtaining a model function corresponding to a target model; wherein the model function comprises a plurality of operators;
converting each operator in the model function into a first generic expression; wherein the first generic expression comprises a plurality of operator attributes for the operator;
determining an incidence relation between any two operators in the plurality of operators according to the input information and the output information corresponding to each operator;
and generating a model file corresponding to the target model according to the first general expression corresponding to each operator and the incidence relation between any two operators in the plurality of operators.
2. The method for generating a model file according to claim 1, wherein said converting each operator in said model function into a first general expression comprises:
and aiming at each operator in the model function, determining a plurality of operator attributes corresponding to the operator according to the category of the operator, and converting the operator into a first general expression comprising the operator attributes.
3. The method of generating a model file according to claim 2, wherein the category to which each operator belongs is determined by:
searching the storage address of each operator in the model function;
and determining the class matched with the search result as the class to which the operator belongs based on the search result of the storage address.
4. The method for generating the model file according to claim 1, wherein the generating the model file corresponding to the target model according to the first general expression corresponding to each operator and the incidence relation between any two operators in the plurality of operators comprises:
generating a second general expression which comprises a plurality of model attributes and corresponds to the model function based on a plurality of preset model attributes;
and generating a model file corresponding to the target model according to the first general expression corresponding to each operator, the second general expression corresponding to the model function and the incidence relation between any two operators in the plurality of operators.
5. The method of generating a model file according to claim 2, wherein the category to which the operator belongs includes: self-defining operators and framework semantic operators.
6. The method for generating the model file according to claim 1, wherein the generating the model file corresponding to the target model according to the first general expression corresponding to each operator and the incidence relation between any two operators in the plurality of operators comprises:
determining a combination mode corresponding to the operators according to the incidence relation between any two operators in the operators;
and generating a model file corresponding to the target model according to the first general expression corresponding to each operator and the combination mode corresponding to the operators.
7. The method of claim 6, wherein the combination of the operators comprises: sequential execution, one-to-many execution, many-to-many execution.
8. An apparatus for generating a model file, the apparatus comprising:
the acquisition module is used for acquiring a model function corresponding to the target model; wherein the model function comprises a plurality of operators;
the conversion module is used for converting each operator in the model function into a first general expression; wherein the first generic expression comprises a plurality of operator attributes for the operator;
the first determining module is used for determining the incidence relation between any two operators in the plurality of operators according to the input information and the output information corresponding to each operator;
and the generating module is used for generating a model file corresponding to the target model according to the first general expression corresponding to each operator and the incidence relation between any two operators in the plurality of operators.
9. An electronic device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating via the bus when the electronic device is operating, the processor executing the machine-readable instructions to perform the steps of the method of generating a model file according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, performs the steps of the method of generating a model file according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010997489.XA CN112069456A (en) | 2020-09-21 | 2020-09-21 | Model file generation method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010997489.XA CN112069456A (en) | 2020-09-21 | 2020-09-21 | Model file generation method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112069456A true CN112069456A (en) | 2020-12-11 |
Family
ID=73681381
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010997489.XA Pending CN112069456A (en) | 2020-09-21 | 2020-09-21 | Model file generation method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112069456A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113342346A (en) * | 2021-05-18 | 2021-09-03 | 北京百度网讯科技有限公司 | Operator registration method, device, equipment and storage medium of deep learning framework |
| CN113469284A (en) * | 2021-07-26 | 2021-10-01 | 浙江大华技术股份有限公司 | Data analysis method, device and storage medium |
| CN113591934A (en) * | 2021-07-07 | 2021-11-02 | 浙江大华技术股份有限公司 | Method, device and equipment for arranging business analysis model and storage medium |
| CN115145965A (en) * | 2022-09-01 | 2022-10-04 | 浙江大华技术股份有限公司 | Data stream generation method, electronic device and computer-readable storage medium |
| CN116362316A (en) * | 2023-05-29 | 2023-06-30 | 成都阿加犀智能科技有限公司 | Model conversion method and device, storage medium and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102521451A (en) * | 2011-12-13 | 2012-06-27 | 华东电网有限公司 | Power grid model file for supporting fast model splicing, generation method and system thereof |
| CN108279885A (en) * | 2017-01-03 | 2018-07-13 | 中国航发商用航空发动机有限责任公司 | A kind of method and device that multiple model codes are carried out with Integrated Simulation |
| CN109697500A (en) * | 2018-12-29 | 2019-04-30 | 北京中科寒武纪科技有限公司 | Data processing method, device, electronic equipment and storage medium |
-
2020
- 2020-09-21 CN CN202010997489.XA patent/CN112069456A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102521451A (en) * | 2011-12-13 | 2012-06-27 | 华东电网有限公司 | Power grid model file for supporting fast model splicing, generation method and system thereof |
| CN108279885A (en) * | 2017-01-03 | 2018-07-13 | 中国航发商用航空发动机有限责任公司 | A kind of method and device that multiple model codes are carried out with Integrated Simulation |
| CN109697500A (en) * | 2018-12-29 | 2019-04-30 | 北京中科寒武纪科技有限公司 | Data processing method, device, electronic equipment and storage medium |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113342346A (en) * | 2021-05-18 | 2021-09-03 | 北京百度网讯科技有限公司 | Operator registration method, device, equipment and storage medium of deep learning framework |
| CN113342346B (en) * | 2021-05-18 | 2022-03-25 | 北京百度网讯科技有限公司 | Operator registration method, device, equipment and storage medium of deep learning framework |
| US11625248B2 (en) | 2021-05-18 | 2023-04-11 | Beijing Baidu Netcom Science Technology Co., Ltd. | Operator registration method and apparatus for deep learning framework, device and storage medium |
| CN113591934A (en) * | 2021-07-07 | 2021-11-02 | 浙江大华技术股份有限公司 | Method, device and equipment for arranging business analysis model and storage medium |
| CN113469284A (en) * | 2021-07-26 | 2021-10-01 | 浙江大华技术股份有限公司 | Data analysis method, device and storage medium |
| CN115145965A (en) * | 2022-09-01 | 2022-10-04 | 浙江大华技术股份有限公司 | Data stream generation method, electronic device and computer-readable storage medium |
| CN116362316A (en) * | 2023-05-29 | 2023-06-30 | 成都阿加犀智能科技有限公司 | Model conversion method and device, storage medium and electronic equipment |
| CN116362316B (en) * | 2023-05-29 | 2023-12-12 | 成都阿加犀智能科技有限公司 | Model conversion method and device, storage medium and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112069456A (en) | Model file generation method and device, electronic equipment and storage medium | |
| CN110968325B (en) | Applet conversion method and device | |
| US8887135B2 (en) | Generating test cases for functional testing of a software application | |
| CN110704479A (en) | Task processing method and device, electronic equipment and storage medium | |
| CN106648662B (en) | Report generation device and method based on project cost calculation description language BCL | |
| CN110941427B (en) | Code generation method and code generator | |
| CN108984155B (en) | Data processing flow setting method and device | |
| US20140173559A1 (en) | Identifying differences between source codes of different versions of a software when each source code is organized using incorporated files | |
| CN111026634A (en) | Interface automation test system, method, device and storage medium | |
| CN111221842A (en) | Big data processing system and method | |
| CN111626028A (en) | Report generation method, device and equipment based on function analysis and computer medium | |
| CN111221852A (en) | Mixed query processing method and device based on big data | |
| CN116028028A (en) | Request function generation method, device, equipment and storage medium | |
| CN111580821B (en) | Script binding method and device, electronic equipment and computer readable storage medium | |
| US9274755B2 (en) | Infrastructure for generating code using annotation and template generators | |
| CN111221888A (en) | Big data analysis system and method | |
| US20110296373A1 (en) | Command line shell command generation based on schema | |
| WO2023287786A1 (en) | Collaborative industrial integrated development and execution environment | |
| CN112445468A (en) | Typescript type file generation method, device, equipment and computer readable storage medium | |
| Chiu et al. | Integrating Semantic Web and Object-Oriented Programming for Cooperative Design. | |
| JP2016051367A (en) | Data analysis device, data analysis method, and program | |
| Ding et al. | Enhanced enumeration techniques for syntax-guided synthesis of bit-vector manipulations | |
| CN111597205B (en) | Template configuration method, information extraction device, electronic equipment and medium | |
| CN117270838B (en) | Method, device, equipment and medium for generating universal formula script | |
| CN112988298B (en) | Visual interface configuration method, visual interface configuration device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |