CN114444232A

CN114444232A - Model management method and device

Info

Publication number: CN114444232A
Application number: CN202111617078.4A
Authority: CN
Inventors: 李文臣; 黄彦浩; 王申; 陈兴雷; 文晶; 郭宇; 邹卫美; 李勤新; 裘微江; 王甜婧; 王兵; 龙晓飞; 常文雅; 雷富强; 章姝俊
Original assignee: Beijing Zetyun Tech Co ltd; State Grid Zhejiang Electric Power Co Ltd; China Electric Power Research Institute Co Ltd CEPRI
Current assignee: Beijing Zetyun Tech Co ltd; State Grid Zhejiang Electric Power Co Ltd; China Electric Power Research Institute Co Ltd CEPRI
Priority date: 2021-12-27
Filing date: 2021-12-27
Publication date: 2022-05-06

Abstract

The invention discloses a model management method and a device, comprising the following steps: obtaining a model to be processed and a data source corresponding to the model; performing data preprocessing on the data source to acquire standard data conforming to a preset data source interface; and training the model through the standard data to generate a pipeline model corresponding to the model to be processed. The problems that the current model service calling efficiency is low, the management workload of multiple models is huge and relatively complex and the like are solved.

Description

Model management method and device

Technical Field

The present application relates to the field of data management technologies, and in particular, to a model management method and apparatus.

Background

The model is a substitute of a prototype which is obtained by condensing, abstracting and abstracting a part of objective objects for a certain purpose, and the model intensively reflects a part of characteristics required by people in the prototype. For example, an Artificial Neural Network (ANN) model, similar to a biological Neural Network, is formed by connecting a plurality of neuron models according to a certain rule, and can solve the problems related to classification and regression according to requirements.

While the model is only an abstraction of reality, it represents the current state of the business and the environment in which the business operates, some models are now widely used in strategic business planning. Financial models such as balance sheets, profit and cash flow sheets. Each is an abstraction of the current state of the service. While the importance of these models and their use in budgeting and financial planning is well known, it is not common to use models for decision making in all other aspects of an enterprise.

In summary, the model has the advantages of being able to clearly represent the current state of a business, reducing the risk of making decisions based on incorrect understanding of the business, being able to reflect different perspectives of the same situation, being able to help identify bottlenecks or problems in the business process, and ensuring the integrity of the solution to be implemented.

At present, in the face of diversification and personalized development of models, from different angles of problem solving and related theoretical bases, different models have one or more management methods, and unified management of the multiple models is difficult to realize.

The models cannot realize unified management, and the problems include low efficiency of model service calling, huge and relatively complex management workload of multiple models, incapability of managing multiple models on-line or off-line in batches, and the like.

Disclosure of Invention

In order to solve the above problem, the present application provides a model management method, including:

obtaining a model to be processed and a data source corresponding to the model;

performing data preprocessing on the data source to acquire standard data conforming to a preset data source interface;

and training the model through the standard data to generate a pipeline model corresponding to the model to be processed.

Preferably, after the step of generating the pipeline model corresponding to the model to be processed, the method further includes:

and receiving the calling of the pipeline model through a model calling interface.

Preferably, the data source corresponding to the model is any one of the following formats: csv, txt, queue, zip, tar.

Preferably, the data preprocessing is performed on the data source to obtain standard data meeting a preset data source interface, and the method includes:

and customizing the workflow of the data source to generate standard data which accords with a preset data source interface.

The present application also provides a model management device, comprising:

the data source acquisition unit is used for acquiring a model to be processed and a data source corresponding to the model;

the data source processing unit is used for carrying out data preprocessing on the data source to acquire standard data which accord with a preset data source interface;

and the model training unit is used for training the model according to the standard data to generate a pipeline model corresponding to the model to be processed.

Preferably, the method further comprises the following steps:

and the calling unit receives the calling of the pipeline model through a model calling interface.

The application also discloses a management method for the image recognition model, which comprises the following steps:

acquiring an image recognition model to be processed and a data source corresponding to the image recognition model;

and training the image recognition model through the standard data to generate a pipeline model corresponding to the image recognition model to be processed.

Preferably, after the step of generating the pipeline model corresponding to the image recognition model to be processed, the method further includes:

Preferably, the data source corresponding to the image recognition model is in a zip format.

decompressing the data source in the zip format;

and customizing the workflow of the decompressed data source to generate standard data which accords with a preset data source interface.

The application also provides a management device for the image recognition model, which comprises:

the data source acquisition unit is used for acquiring an image recognition model to be processed and the like and a data source corresponding to the image recognition model;

the preprocessing unit is used for preprocessing the data of the data source to acquire standard data which accord with a preset data source interface;

and the pipeline model generating unit is used for training the image recognition model through the standard data to generate a pipeline model corresponding to the image recognition model to be processed.

A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the method of any one of the preceding claims are implemented when the computer program is executed by the processor.

A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of the preceding claims.

Drawings

FIG. 1 is a schematic flow chart diagram of a model management method provided herein;

FIG. 2 is a schematic structural diagram of a model management apparatus provided in the present application;

FIG. 3 is a flow chart illustration of a method for managing an image recognition model provided herein;

FIG. 4 is a flow chart of model management as contemplated by the present application.

FIG. 5 is a model management configuration form to which the present application relates;

FIG. 6 is a model itemized form to which the present application relates;

FIG. 7 is a model management form to which the present application relates;

FIG. 8 is a model management form to which the present application relates;

FIG. 9 is a model management form to which the present application relates;

fig. 10 is a model modeling form according to the present application.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is capable of implementation in many different ways than those herein set forth and of similar import by those skilled in the art without departing from the spirit of this application and is therefore not limited to the specific implementations disclosed below.

Fig. 1 is a schematic flow chart of a model management method provided in the present application, and the method provided in the present application is described in detail below with reference to fig. 1.

Step S101, obtaining a model to be processed and a data source corresponding to the model.

The application provides a model management method, namely an idea of 'all models'. For example, the AI model file, the ETL script, and the rules are all considered as models in the present system, so as to perform unified staged management.

The model to be processed may be a neural network model, and the ETL script and the rules are both considered as models in this application. The model to be processed is one or more of a multi-class model. The multi-class model management relates to complex operations such as multi-system docking, multi-process configuration and the like, a fixed operation process obviously cannot meet the system flexibility, and in order to solve the problems, the flexibility is realized by adopting a 'plugging and unplugging' mode. The specific embodiments are the aspects of "flow definition", "system docking" and the like.

Firstly, for obtaining a model to be processed and a data source corresponding to the model, the data source is any one of the following formats: csv, txt, parquet, zip, tar. The model to be processed has a one-to-one relationship with the data source. The model receives input from the data source before processing via a defined interface, which may be any of: the File interface File and the Java database are connected with JDBC, the data warehouse Hive and the Hadoop distributed File system hdfs. The model can select the corresponding interface according to actual needs.

And step S102, carrying out data preprocessing on the data source to acquire standard data conforming to a preset data source interface.

By adopting a plurality of standardized data access modes, the data source can be accessed to a relevant model management platform. And carrying out standardization processing on the data source so that the processed data conforms to a preset data source interface, thereby realizing the unified standard for establishing the model. The preset data source interface can receive a plurality of standardized data access modes, including: the File interface File and the Java database are connected with JDBC, the data warehouse Hive and the Hadoop distributed File system hdfs.

Then, the workflow of the data source is customized, and standard data which are in line with a preset data source interface are generated. For example, the model can only receive a txt format data source before generating standard data, and can receive a zip format data source after normalization processing.

And S103, training the model through the standard data to generate a pipeline model corresponding to the model to be processed.

The model entering the modeling stage is modeled, the modeling stage comprises a multi-latitude modeling process, for example, an AI model comprises two stages of data development and model development, the model enters development platforms of different stages by clicking a jump link (in a plug-in mode), each development stage also has personalized operation, for example, the model development stage provides a model uploading function, and each operation can display the submission time of the operation, the type of the model and the version of the model. The submitted model file can be selected as an Area Under a Receiver Operating characteristic Curve (ROC) Curve defined by coordinate axes, and a model evaluation index AUC (AUC) is selected for automatic evaluation and displayed in a page. The state of the development phase model is updated with each commit of the operation. And after modeling is completed, training the model to generate a pipeline model, and then issuing the pipeline model to the model management platform.

After the model training is finished, the model is released on line on the model management platform, and the model on line can receive the calling of the pipeline model through the model calling interface. Based on the same inventive concept, the present application also provides a type management apparatus 200, as shown in fig. 2, including:

a data source obtaining unit 210, configured to obtain a model to be processed and a data source corresponding to the model;

the data source processing unit 220 is configured to perform data preprocessing on the data source to obtain standard data that conforms to a preset data source interface;

and the model training unit 230 is configured to train the model according to the standard data, and generate a pipeline model corresponding to the model to be processed.

And the calling unit 240 is configured to complete the calling of the different models in a uniform interface request manner.

Preferably, the method further comprises the following steps:

and the calling unit receives calling of the pipeline model through a model calling interface.

Based on the same inventive concept, the present application also provides a management method for an image recognition model, as shown in fig. 3, including:

step S301: acquiring an image recognition model to be processed and a data source corresponding to the image recognition model;

step S302: performing data preprocessing on the data source to acquire standard data which accord with a preset data source interface;

step S303: and training the image recognition model through the standard data to generate a pipeline model corresponding to the image recognition model to be processed.

And the data source corresponding to the image recognition model is in a zip format. Decompressing the data source in the zip format; and customizing the workflow of the decompressed data source to generate standard data which accords with a preset data source interface. And after the image recognition model is trained and a pipeline model corresponding to the image recognition model to be processed is generated, receiving the calling of the pipeline model through a model calling interface.

Based on the same inventive concept, the present application also provides a management apparatus for an image recognition model, comprising:

Example 1 the following:

the model management method provided by the application realizes the management of the model, and the flow is shown in fig. 4. The main management process comprises the following steps: establishing items, developing models, storing models, uploading/downloading models and monitoring model operation and maintenance. In the project establishment phase, manageable models and realizable functions are determined, wherein the manageable models comprise projects relating to an ETL model, AI model and RULE model. The functions that can be realized include training, storing and managing the model. A platform for model management is then developed, including data platform development, AI training platform development, and rules engine development. The model management platform can also store the model and the relevant files of the model, including SQL scripts, AI model files, rule files and the like. The flow control, task scheduling and model iteration of the model are realized through the SQL script, the AI model file and the rule file. After the model is on line through the model management platform, the platform runs model services, the model has a life cycle, and after the model is not adapted to the current application any more, the life cycle is finished, and offline processing is carried out. And monitoring and managing the model through operation and maintenance in the whole life cycle of the model, wherein the operation and maintenance comprises the model, the script of the model, the file management of the model and the running state monitoring.

Firstly, a process definable engine is provided, and each stage process in the model management can be customized through a page function so as to meet the management process requirements of different models.

As shown in fig. 5, the order, name, etc. of the flow are defined according to the serial number of the configuration, and the node name, node code, form name and form code of the configuration form can be edited by the edit button in the operation field to perform redefinition of the model management flow.

Meanwhile, a development API which can be integrated by a platform is provided, and different development tools or platforms of different models can be loaded and integrated. The model management relates to multi-system docking, the requirement on platform integration capacity is high, and the usability, maintainability, compatibility and applicability of system integration are improved by using a 'plug-pull' mode.

The model management platform supports various format data sources such as csv, txt, partial, zip, tar and the like, and adopts various standardized data access modes: the platform has data unified processing capability, which is mainly embodied in that data conforming to various interfaces are generated through a data processing process or a mode of self-defined workflow, a unified pipeline model is generated through model training, a unified standardized environment of model training is created, model service is released and is on-line on the basis, and model service calling can be carried out through API modes such as restful, batch and grp to complete a complete life cycle of model management.

Taking the conventional lifecycle of the AI model as an example, to achieve uniform lifecycle management for all category models, the following stages are abstracted,

1) setting up an item;

2) modeling;

3) getting on line;

4) operation and maintenance;

5) iteration is carried out;

6) and (5) offline.

Each stage is provided with a corresponding auditing process, and the next stage operation is carried out after the auditing is passed.

In the project establishment phase, a project name and a type of a selected project may be set, and after the project is created, the platform may display information such as the project name, the project type, the status, the owner, the creation date, and whether the project is in an operable state on a corresponding project establishment display interface, as shown in fig. 6. To better understand the project, after the project is established, the project name, project type, status, owner and creation date of the established project can be checked through the project management interface, and the project of the related established project can be managed through operation, as shown in fig. 7. The external system can be accessed through item setting configuration, so that the pluggable management can be realized, and the model platform corresponding to the model platform is accessed through the following graph. And accessing the celestial model system corresponding to the model platform as shown in figure 6.

And a modeling phase, after the project is set up, if the project is not limited and is in an operable state, entering the modeling phase. The modeling stage comprises a multi-latitude modeling process, for example, an AI model comprises two stages of data development and model development, a development platform of different stages is accessed by clicking a jump link (plug-in mode), each development stage also has personalized operation, for example, the model development stage provides a function of uploading a model, and each operation displays the submission time of the operation, the type of the model and the version of the model. The submitted model file can be selected as an Area Under a Receiver Operating characteristic Curve (ROC) Curve defined by coordinate axes, and a model evaluation index AUC (AUC) is selected for automatic evaluation and displayed in a page. The state of the model in the development phase is updated along with the submission of each operation, such as modeling, data uploading, modeling completion, model training and model testing. As shown in fig. 8, the status of the item is an item standing status, and information such as the related item profile and item standing material is displayed. As shown in fig. 9, the state of the project is a modeling state, and modeling and result information, submitted model information and the like are displayed. Aiming at different types of models, different modeling platforms and data platforms are butted in a plugging and unplugging mode in the modeling stage, and loosely-coupled and personalized modeling process management is achieved. If the function of 'jump celestial machine' in figure 10 is clicked, the user can jump to the celestial machine platform; and uploading the model script and the model file and the like.

The training process of the model can be selected as required, and may or may not be performed.

And in the online stage, the model which has completed the model test can be subjected to online operation, the model file is uploaded to the directory of the corresponding calling model of the server through the unified interface layer of the model management platform, the calling model service is realized, and the online of the model is completed.

In the operation and iteration stage, after the model is finished on line, the model is used by a user, and the maintenance and version updating of the model are related in the operation and iteration stage, namely the iteration of the model, so that the normal use of the model is ensured.

And in the offline stage, offline processing is carried out on the model which has a long service period or cannot continuously create value.

The operation, the maintenance, the iteration and the offline of the model are all processed through a unified interface of the model management platform, the same protocol specification is used, the workload of the model management is greatly simplified, and the calling efficiency of the model service is improved.

In addition, for submission of each operation in the life cycle of the model, the examination and approval is required to ensure the visualization and standardization of each operation, and specifically, the examination and approval date, examination and approval conclusion and examination and approval state can be displayed.

Further, the model management platform can at least realize the management of the AI model, the ETL model, the mathematical model and the rule model, and set different modeling stages according to the needs and document information required in the respective modeling processes for different types of models, taking the establishment of the AI model as an example, the corresponding modeling stages and the required document information can be referred to the following table:

modeling phase	Document data
		Standing article	Requirement document
Modeling	Characteristic broad table and labeled document
		Evaluation of	Model evaluation reports including AUC, ROC, F1, MAE, MSE, etc
Threading	Online result document
		Off-line	Model-related asset offline and archiving

As for the ETL model, the development phase is also involved in the modeling phase, and corresponding ETL script data is required.

The model management platform uses an interface style selection RESTFUL of an Application Program (API) to support 5 operation interfaces including GET, POST, PUT, PATCH and DELET.

Before the user performs the establishment modeling, the method further comprises an authorization authentication process of the user, and specifically comprises the following steps:

1) registering appKey and secCode with a model management platform;

2) obtaining an access token, used for limiting validity in time, accessoken;

3) the Http requests the Header to add a token accessToken;

for example: accessToken ═ xx.

4) And can be retrieved through refresh _ access _ token.

Reference is made to the following Table for accessToken

Obtaining an accessToken, namely get _ access _ token, the process of obtaining the access token includes:

the request URL is as follows:

http://localhost:10400/modelps/openapi/oauth/tokenappKey＝xx&secCode＝sssssss

the request mode comprises the following steps: get

Parameters are as follows:

parameter name	Type (B)	Whether or not to fill	Description of the invention	Remarks for note
					appKey	String	Is that	Third party application key
secCode	String	Is that	Third party application key

Returning to the example:

returning to the description of the parameters:

the process of refreshing the token includes (refresh _ access _ token):

request URL:

http://localhost:10400/modelps/openapi/oauth/token/refreshappKey＝xx&secCode＝ss

the request mode comprises the following steps: get

Request parameters:

Returning to the example:

returning to the description of the parameters:

name (R)	Type (B)	Description of the invention	Remarks for note
				returnCode	int	Return code	0 successful, not 0: error code

Example 2 is as follows:

the aspects of creating, jumping and deleting the model operation are explained in detail.

The method mainly comprises the following steps:

1) creating

The creator defaults to the owner and the owner identity may be transferred.

2) Jump to

Project names, profiles, summaries, project types, development status may be changed.

3) Deleting

An item can be deleted if and only if no other components are included in the item (analysis module, automatic modeling, etc.). Only the item owner can delete it.

4) Jump to

The main data of the model project display interface comprise:

creating a project, wherein according to different butted systems, { systemURL } is a domain name corresponding to the system, the system is required to realize corresponding api to realize modelproject creation, and the specific process comprises the following steps:

request URL:

·https://{system_modelproject_create_url}

the request mode comprises the following steps: POST (positive position transducer)

Request example:

request parameters:

returning to the example:

description of the Return parameters

The step of deleting comprises:

request URL:

·https://{system_modelproject_delete_url}/<projectId>

the request mode comprises the following steps:

·DELETE

unsolicited data

Return example

Description of the Return parameters

The step of jumping (to a specific modeling system by providing a corresponding link through the model management platform) includes:

jump URL:

·https://{system_direct_url}/<projectId>

the request mode comprises the following steps:

POST request example

·https://crm/modelproject/abs-6d3f6cc4-cd44-4c65-b050-3dfa1c0e6d5a

By the model management method and the model management device, the unified standard for establishing the models is realized, the calling of various models is realized, the pluggable characteristic of the interface is realized through online and offline of the model service, the full life cycle of the model management is further completed, the working efficiency is improved, and the management cost of multiple models is reduced.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

Finally, it should be noted that: although the present invention has been described in detail with reference to the above embodiments, it should be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from the spirit and scope of the invention.

Claims

1. A method of model management, comprising:

obtaining a model to be processed and a data source corresponding to the model;

2. The method according to claim 1, wherein after the step of generating the pipeline model corresponding to the model to be processed, the method further comprises:

and receiving the call of the pipeline model through a model call interface.

3. The method of claim 1, wherein the data source corresponding to the model is any one of the following formats: csv, txt, parquet, zip, tar.

4. The method of claim 1, wherein preprocessing the data source to obtain standard data that conforms to a preset data source interface comprises:

5. A model management apparatus, comprising:

6. The apparatus of claim 5, further comprising:

7. A method for managing an image recognition model, comprising:

8. The method according to claim 7, further comprising, after the step of generating a pipe model corresponding to the image recognition model to be processed:

9. The method according to claim 7, wherein the image recognition model corresponds to a data source, in particular in a zip format.

10. The method according to claim 7 or 9, wherein the data preprocessing is performed on the data source to obtain standard data meeting a preset data source interface, and the method comprises:

decompressing the data source in the zip format;

11. A management apparatus for an image recognition model, comprising:

the data source acquisition unit is used for acquiring an image recognition model for processing and the like and a data source corresponding to the image recognition model;

12. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method of any of claims 1 to 4 or 7 to 10 are implemented by the processor when executing the computer program.

13. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 4 or 7 to 10.