CN114610273A - AI model realization method, electronic device and storage medium - Google Patents

AI model realization method, electronic device and storage medium Download PDF

Info

Publication number
CN114610273A
CN114610273A CN202011346401.4A CN202011346401A CN114610273A CN 114610273 A CN114610273 A CN 114610273A CN 202011346401 A CN202011346401 A CN 202011346401A CN 114610273 A CN114610273 A CN 114610273A
Authority
CN
China
Prior art keywords
model
user
data
client
image file
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011346401.4A
Other languages
Chinese (zh)
Inventor
庞磊
潘绪洋
蒋阳
赵丛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Gongdadi Innovation Technology Shenzhen Co ltd
Original Assignee
Gongdadi Innovation Technology Shenzhen Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Gongdadi Innovation Technology Shenzhen Co ltd filed Critical Gongdadi Innovation Technology Shenzhen Co ltd
Priority to CN202011346401.4A priority Critical patent/CN114610273A/en
Publication of CN114610273A publication Critical patent/CN114610273A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00Arrangements for software engineering
    • G06F8/10Requirements analysis; Specification techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00Arrangements for software engineering
    • G06F8/30Creation or generation of source code
    • G06F8/35Creation or generation of source code model driven
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/451Execution arrangements for user interfaces
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00Machine learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Medical Informatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Human Computer Interaction (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The application relates to the technical field of artificial intelligence, and particularly discloses an AI model implementation method, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring demand information issued by a user on a client of an AI transaction platform; determining a corresponding AI model algorithm according to the demand information, packaging the AI model algorithm into an image file, and loading the image file into the AI transaction platform; carrying out hyper-parameter analysis on the mirror image file, and sending the hyper-parameters obtained by analysis to a client for displaying so that a user can confirm the hyper-parameters; obtaining model training data, and replacing a data path of the model training data with a specified path of an AI trading platform so as to map the data path to a client for display; and receiving and responding to a determination instruction of a user, loading the image file, performing automatic machine learning training according to model training data to obtain an AI model, and issuing the AI model, so that convenience and intellectualization of AI model training are realized, and the experience degree of the user is improved.

Description

AI model realization method, electronic device and storage medium
Technical Field
The present application relates to the field of artificial intelligence technologies, and in particular, to an AI model implementation method, an electronic device, and a storage medium.
Background
With the rapid development of science and technology, the world has gradually entered the Artificial Intelligence (AI) era. The AI is an application technology, and how to quickly convert an AI model into a product so as to realize business landing is an urgent problem to be solved in the field of artificial intelligence at present. The application of the AI model includes data processing, model design, model training, model publishing, etc., and the realization method of the AI model is usually handed to an AI model engineer to debug and train to finally obtain the AI model.
Because the training difficulty of the AI model is high, the requirements of different types of users on the model are different, for example, some engineers need to obtain some AI models with higher precision, while some common users do not have the higher precision on the AI model, but professional engineers still need to debug repeatedly, which wastes resources.
Disclosure of Invention
The application relates to the technical field of artificial intelligence, in particular to an implementation method of an AI model, electronic equipment and a storage medium, aiming at solving the problems that the difficulty of the current AI model training is high, a professional engineer is required to debug repeatedly in the training process, time and labor are consumed, the supply of the AI model is less, and supply and demand cannot be met.
In order to achieve the above object, the present application provides an implementation method based on an AI model, including:
acquiring demand information issued by a user on a client of an AI transaction platform, wherein the demand information comprises description information of the user on a required AI model;
determining a corresponding AI model algorithm according to the demand information, packaging the AI model algorithm into an image file, and loading the image file into the AI transaction platform;
carrying out hyper-parameter analysis on the mirror image file, and sending the hyper-parameters obtained by analysis to the client for display so that the user can confirm the hyper-parameters;
obtaining model training data, and replacing a data path of the model training data with a specified path of the AI trading platform so as to map the data path to the client for display;
and receiving and responding to the determination instruction of the user, loading the image file, performing automatic machine learning training according to the model training data to obtain an AI model, and issuing the AI model.
In addition, to achieve the above object, the present application also provides an electronic device, which includes a memory and a processor; the memory for storing a computer program; the processor is configured to execute the computer program and implement the AI model implementation method provided in any one of the embodiments of the present application when the computer program is executed.
In addition, to achieve the above object, the present application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the AI model implementation method according to any one of the embodiments of the present application.
According to the implementation method of the AI model, the electronic device and the storage medium, the server and the client of the AI transaction platform are built, the corresponding AI model algorithm can be determined according to the demand information of the AI model provided by the user, automatic machine learning training is carried out, and the AI model is obtained, so that the problems that the number of providers of the AI model is small, the supply and demand cannot be met are solved, and the user experience is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic view of an application scenario of an AI transaction platform according to an embodiment of the present application;
fig. 2 is a schematic flow chart of an implementation method of an AI model provided in an embodiment of the present application;
fig. 3 is a schematic flowchart of acquiring demand information published by a user on a client of an AI trading platform according to an embodiment of the present application;
FIG. 4 is a schematic flow chart diagram of an algorithm for determining whether a user provides an open source AI model according to an embodiment of the application;
FIG. 5(a) is a schematic diagram of a configuration hyper-parameter provided in an embodiment of the present application;
FIG. 5(b) is a schematic diagram of a client task type provided by an embodiment of the present application;
FIG. 5(c) is a schematic diagram of a configuration database provided in an embodiment of the present application;
fig. 6 is a schematic diagram of a K8S schedule provided in an embodiment of the present application;
FIG. 7 is a diagram illustrating defined model parameters provided by an embodiment of the present application;
fig. 8 is a schematic block diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The flow diagrams depicted in the figures are merely illustrative and do not necessarily include all of the elements and operations/steps, nor do they necessarily have to be performed in the order depicted. For example, some operations/steps may be decomposed, combined or partially combined, so that the actual execution order may be changed according to the actual situation. In addition, although the division of the functional blocks is made in the device diagram, in some cases, it may be divided in blocks different from those in the device diagram.
The term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
At present, applications of an AI (intellectual intelligence) model include data processing, model design, model training, model publishing and the like, demands of the AI model are increasing day by day in recent years, and as training difficulty of the AI model is high, a professional engineer needs to repeatedly debug the AI model in a training process, time and labor are consumed, so that a few providers of the AI model cannot meet supply and demand, and meanwhile, demands of users of different types on the model are not the same, for example, some engineers need to obtain some AI models with higher precision, while some ordinary users do not have the high precision on the AI model, but the professional engineer still needs to repeatedly debug the AI model, which wastes resources.
Therefore, the present application provides an AI model implementation method, an electronic device, and a storage medium to solve the above problems.
Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and the features of the embodiments can be combined with each other without conflict.
The implementation method of the AI model provided by the embodiment of the application is implemented based on the AI transaction platform, so for convenience of understanding, the AI transaction platform is introduced before the implementation method of the AI model is introduced.
The AI trading platform is a platform system realized based on a third-party code hosting platform and comprises a server and a client. The third-party code hosting platform is an open source code library such as github and gitlab, a plurality of AI model algorithms developed by algorithm experts are available in a third-party open source community corresponding to the third-party code hosting platform and are open source, model training is facilitated, however, complex environment configuration, code debugging and other work are still required, a required AI model cannot be easily obtained by a demand party, the AI trading platform is realized and improved based on the third-party code hosting platform, and a user can realize the AI trading platform without complex configuration. Meanwhile, the AI trading platform also realizes the low threshold of AI model training based on a third-party code hosting platform so as to solve the problem of unmatched supply and demand.
As shown in fig. 1, a client of the AI trading platform may be installed in a terminal device, and a server may be installed in a server. The terminal device may include a fixed terminal such as a mobile phone, a tablet computer, a notebook computer, a palm top computer, a Personal Digital Assistant (PDA), and a Digital TV, a desktop computer, and the like. The servers may be, for example, individual servers or clusters of servers.
The client can obtain the demand information issued by the user, the demand information comprises the description information of the user on the needed AI model, and the demand information is sent to the server, so that the server determines the corresponding AI model algorithm and the training data according to the demand information to carry out model training, and sends the trained model to the client for the user to use.
Specifically, the client may be an application program (APP), and when the APP is opened by a user, the APP displays a demand information interface so that the user fills in demand information on the demand information interface, acquires the demand information filled in by the user on the demand information interface, sends the demand information to the server, and the server executes the implementation method of the AI model provided in the embodiment of the present application according to the demand information to perform model training, and sends the trained model to the user so that the user can use the model.
The following describes in detail an implementation method of the AI model provided in the embodiment of the present application based on the application scenario in fig. 1.
Referring to fig. 2, fig. 2 is a schematic flowchart of an implementation method of an AI model according to an embodiment of the present disclosure. The implementation method of the AI model can be applied to a server of an AI transaction platform, realizes the intellectualization of AI model training, improves the convenience of acquiring the required AI model by a user, reduces the cost of artificially training the AI model, and improves the user experience.
As shown in fig. 2, the AI model implementation method includes steps S101 to S105.
S101, acquiring demand information issued by a user on a client of an AI transaction platform, wherein the demand information comprises description information of the user on a required AI model.
In an embodiment of the present application, the description information of the AI model includes a function, a type and an application range of the AI model, that is, a description of what a user wants to do with the AI model, and is used for determining a corresponding AI model algorithm.
Specifically, the client may display a requirement information interface, so that the user fills in the requirement information on the requirement information interface, and obtains the requirement information filled in by the user on the requirement information interface.
In some embodiments, the voice of the user may also be obtained from the client, for example, the client is provided with a voice button to prompt the user to issue the demand information through voice, and the demand information of the user is obtained by recognizing the voice of the user.
In some embodiments, as shown in fig. 3, the step of acquiring the requirement information published by the user on the client of the AI trading platform specifically includes the following steps:
and S1011, obtaining the model requirement of the user through the client of the AI trading platform.
In an embodiment of the present application, the model requirement of the user is used to determine description information of a corresponding AI model. The AI model requirements may be functional requirements or contextual requirements.
For example, a user may enter the AI trading platform on a client, such as a laptop, and send an AI model requirement for screening a specific picture to a server.
For example, a user may enter the AI trading platform on a client, such as a palm-top computer, and send an AI model requirement for a computing scenario in the construction industry to a server.
S1012, determining a target AI model and description information of the target AI model from a historical AI model according to the model requirements.
In the embodiment of the application, according to the model requirement, an AI model closest to the model requirement is searched from historical AI models, and the closest AI model is set as a target AI model.
The historical AI model comprises a high-star model collected on the network or an AI model stored after being trained by the AI trading platform, and the server can search the high-star models on the network and encapsulate the high-star models into corresponding mirror images. The high-star model is an AI model which is used most on the network, an AI model with high evaluation rate or an AI model with high practicability. The efficiency and the accuracy of determining the target AI model can be improved by searching the high-star model on the internet, and the time cost is greatly saved.
Illustratively, the server searches an AI model closest to the description information of the AI model provided by the user from historical AI models in the third-party code hosting platform as a target AI model through the description information of the AI model provided by the user, and outputs the description information of the target AI model.
In some embodiments, after publishing the AI model, the server may package the AI model into a corresponding image file to be stored on the third party code hosting platform as a historical AI model. The AI model is packaged into a corresponding image file which is stored on a third-party code hosting platform, so that the target AI model can be conveniently searched next time, the efficiency of determining the target AI model is improved, and the time cost is greatly saved.
And S1013, displaying the description information through the client so as to be confirmed by the user.
In the embodiment of the application, the description information of the target AI model is sent to the client, and the client displays the received description information of the target AI model so that a user can confirm whether the AI model is a required model.
In some embodiments, the server may further send description information of one or more target AI models to the client, and sort the description information according to the functional similarity and the size of the applicable range, so that the client displays the description information of the multiple target AI models to the user for the user to select. The most desirable model may be presented to the user more clearly by ranking the multiple target AI models.
Illustratively, the client displays 3 target AI models sorted according to priority, wherein the functional similarity of the 1 st target AI model is 95%, the application range is the construction industry, the functional similarity of the 2 nd target AI model is 90%, the application range is the industry, the functional similarity of the 3 rd target AI model is 80%, the application range is all industries, and at this time, the user can select according to the actual needs of the user.
It can be understood that the higher priority AI model can screen the server out the target AI model that best suits the needs of the user model.
In some embodiments, if the user is not satisfied with the target model result, the model requirement may be modified at the client, so as to continue searching for the target AI model. For example, if the user is not satisfied with the implementation function of the target model, the model requirement may be refined and then searched.
S1014, in response to receiving the confirmation instruction of the user to the description information, taking the description information as requirement information.
Illustratively, an instruction for confirming, modifying or quitting the description information of the target AI model by the user is detected, and if the instruction for confirming the description information of the target AI model by the user is received, the description information of the target AI model is used as the requirement information.
Of course, if a modification instruction of the description information of the target AI model by the user is received, the model requirement information modification page is displayed on the client, and if an exit instruction of the description information of the target AI model by the user is received, the current page is closed.
S102, determining a corresponding AI model algorithm according to the demand information, packaging the AI model algorithm into an image file, and loading the image file into the AI transaction platform.
And after an AI model algorithm is determined according to the demand information, packaging the AI model algorithm into an image file so as to carry out hyper-parameter analysis. The AI model algorithm comprises an open source AI model algorithm collected by the AI trading platform or an open source AI model algorithm collected by a user.
The image file is an executable software package which can be independently operated and directly loaded into the AI transaction platform, and the software package has low requirement on the operating environment and is basically not influenced by the operating environment, so that the loading of the image file can be stably operated.
Illustratively, the Image file may be a container Image (Image) that carries binary data encapsulating the code.
In some embodiments, if the image file is stored in a third-party code hosting platform, a tag may be added after the name of the image file, so that different versions in the same image sequence can be identified by the tag.
And loading different versions of image files in the same image sequence, wherein the different versions are the initial image file and the image file with parameters modified by a user.
In some embodiments, in order to quickly determine whether the user provides the open source AI model algorithm, as shown in fig. 4, the step of determining whether the user provides the open source AI model algorithm specifically includes the following steps:
s1021, detecting whether a user provides an open source AI model algorithm;
s1022, if the user provides an open source AI model algorithm, acquiring a URL corresponding to the open source AI model algorithm collected by the user, and transmitting the URL into a preset interface of the AI transaction platform;
and S1023, if the user does not provide the open source AI model algorithm, adopting the open source AI model algorithm collected by the AI trading platform.
Specifically, whether a user provides an open source AI model algorithm is detected; if the user provides an open source AI model algorithm, acquiring a URL corresponding to the open source AI model algorithm collected by the user, and transmitting the URL into a preset interface of the AI transaction platform; and if the user does not provide the open source AI model algorithm, adopting the open source AI model algorithm collected by the AI trading platform.
Specifically, whether the user provides the open-source AI model algorithm or not is detected can be determined according to the data address of the AI model algorithm, and if the URL corresponding to the open-source AI model algorithm is detected as a private address, the open-source AI model algorithm provided by the user is determined; and if the URL corresponding to the open-source AI model algorithm is detected to be the open-source address, determining that the user does not provide the open-source AI model algorithm.
If the open source AI model algorithm provided by the user is detected, acquiring a URL (uniform resource locator) corresponding to the open source AI model algorithm collected by the user, and transmitting the URL (uniform resource locator) corresponding to the open source AI model algorithm into a preset interface of the AI transaction platform, so that the AI transaction platform analyzes the code of the open source AI model algorithm collected by the user according to the URL (uniform resource locator) and maps the code to the client for display. The preset interface is used for acquiring codes of the open source AI model algorithm. By acquiring the URL corresponding to the open source AI model algorithm collected by the user, the corresponding AI model algorithm can be determined in a targeted manner according to the user requirements.
If the fact that the user does not provide the open source AI model algorithm is detected, the AI trading platform searches the corresponding open source AI model algorithm through a third party code hosting platform, and the AI trading platform meets algorithm requirements of most AI models through collecting mainstream open source AI model algorithms.
Illustratively, the server transmits a URL (uniform resource locator) provided by the user to the AI transaction platform, and the AI transaction platform parses the URL to obtain a code of the AI model algorithm, and maps the code to the client for the user to confirm.
In some embodiments, if the user needs to modify the code, the user can modify the code through the client and send a confirmation instruction to the server; and if the user does not need to modify the code, the user can confirm and then sends a confirmation instruction to the server.
S103, carrying out hyper-parameter analysis on the mirror image file, and sending the hyper-parameters obtained by analysis to the client for displaying so that the user can confirm the hyper-parameters.
And carrying out hyper-parameter analysis on the mirror image file, exposing the character string obtained by analyzing the mirror image file into character strings, and automatically presenting the character strings corresponding to the analyzed hyper-parameters on a client by utilizing a dynamic front-end technology so as to facilitate modification of a user.
In some embodiments, if the image file is searched by a third-party code hosting platform such as github, most of algorithm items in the image file are more standard writing methods, the image file is analyzed through an independent hyper-parameter analysis function, and the analyzed configuration parameters are sent to the client for display configuration. The analyzed configuration parameters are sent to the client side for display configuration, so that a user can check the configuration parameters in real time, and change can be modified in time.
In some embodiments, if the image file is provided by the user, the algorithm item in the image file may not be a canonical writing method, and therefore, the hyper-parameter related parsing function, such as a parser function, in the hyper-parameter parsing tool module is used to parse the hyper-parameter related parsing function into a character string, which is exposed.
Illustratively, by using a file analysis tool, the hyper-parameters in the open source project of the algorithm are automatically analyzed, and the analyzed hyper-parameters are automatically presented on the client by utilizing a dynamic front-end technology.
In some embodiments, different parsing functions correspond to different parsing formats, but are a limited number of parsing methods, and can be uniformly packaged into a uniform interface.
Illustratively, the parsing function of the configuration file in the json format is json, and the parsing function of the configuration file in the argparse format is pars _ args, which have different parsing formats, but the parsing methods are all based on python parsing, and thus can be packaged into a unified interface.
In some embodiments, when the client requests the parameter configuration page of the server, for example, an HTML (hypertext markup language) page is generated and then sent to the client, the user may modify the hyper-parameter configuration on the HTML (hypertext markup language) page, send the modified hyper-parameter configuration to the server after confirming the modification, and update the image file of the modified hyper-parameter configuration page.
In some embodiments, the client-displayed hyper-parameters allow the user to modify the changes; and/or the code displayed by the client allows the user to modify. The user can train out the required model more accurately by modifying and changing the hyper-parameters at the client.
Illustratively, a user may perform hyper-parameter configuration on an HTML page generated by the server, where the manner of configuring hyper-parameters is presented by clicking a button on a client, such as a web page, to select, and sending a confirmation instruction to the server and updating its image file.
For example, as shown in fig. 5(a), fig. 5(a) shows a process of selecting an AI training data set by a user, where date refers to a manually labeled data set, and the user may configure the AI training data set on an HTML page generated by a server, where a mode of configuring parameters is presented by filtering and selecting a part of data in the data set through category options on a client, such as a web page, so as to train a model with more suitable configuration parameters, and after configuration is completed, a confirmation instruction is sent to the server, and an image file of the server is updated.
As shown in fig. 5(b), which is a schematic diagram of configuration parameters. The configuration parameters may include a model backbone network (backbone), a head network (head), a neck network (nic), a learning rate, a weight decay (weight decay), and the like.
And S104, obtaining model training data, and replacing a data path of the model training data with a specified path of the AI trading platform so as to map the data path to the client side for display.
And obtaining model training data through a third-party code hosting platform, uniformly replacing the model training data with a specified data path, and mapping the specified data path to the client side for displaying. The model training data comprises open source data or self-contained data, the self-contained data is training data provided by the user, and the designated path of the AI trading platform is a data path of the open source data.
In some embodiments, in order to quickly confirm whether the model training data is normative, the step of determining the type of the model training data specifically includes the following steps:
detecting the data type of the model training data; if the test model training data is the owned data, replacing the data format of the owned data model training data with the data format of the open source data; if the detection model training data is open source data and the data format is a predefined open source data format, the data format does not need to be converted.
The data type of the detected model training data can be determined according to the data address of the model training data, if the data address of the model training data is an open source address, the model training data is determined to be open source data, and if the data address of the model training data is a private URL, the model training data is determined to be self-owned data.
And if the model training data is detected to be self data, converting the data format of the self data into the data format of the predefined open source data, namely replacing the data path of the self data model training data with the data path of the predefined open source data, and sending an instruction for determining the model training data.
In some embodiments, the trading platform only needs to predefine and well express the open source data formats of a plurality of tasks, and the user can convert the own data format according to the open source data format, namely, the own data format is changed into the open source data format, so that the trading platform can automatically help the user to train by using the own data loading open source algorithm.
As shown in fig. 5(c), for example, when the requirement information of the AI model is a detection task, the COCO format or the VOC format may be used in the model training data format of the user so as not to replace the data format, and the model training data can be determined more quickly.
Illustratively, when the AI model's requirement information is a classification task, the VOC format may be used in the agreed-upon user's model training data format to avoid replacing the data format.
Illustratively, if the model training data is detected to be open source data and the data format is a predefined open source data format, the open source data does not need to replace a data path, the data path of the open source data is a specified path of the AI transaction platform, and an instruction for determining the model training data is sent.
For example, if it is detected that the model training data is open source data but the data format is not in the predefined good open source data format, the data format of the open source data still needs to be converted into the predefined good open source data format, that is, the data path of the open source data model training data is also replaced with the predefined good open source data path, and an instruction for determining the model training data is sent.
And S105, receiving and responding to the determination instruction of the user, loading the image file, performing automatic machine learning training according to the model training data to obtain an AI model, and issuing the AI model.
And when the determining instruction is received and responded, the image file is loaded, automatic machine learning training (AutoML) is carried out according to the model training data to obtain an AI model, and the AI model is issued on a trading platform. Therefore, automatic training of AI model training can be realized, the efficiency of AI model training is improved, and the time cost is greatly saved.
The AI model is published on a trading platform, and specifically, an access interface of the AI model can be generated and sent to a user through a client, so that the user can use the AI model through the access interface.
In some embodiments, the AI transaction platform includes a container orchestration scheduling tool including any of a K8S (kubernets) tool, a Swarm tool, or a mess tool.
For example, taking a K8S (kubernets) tool as an example, the image file is loaded and automatic machine learning training is performed according to the model training data, and specifically, the image file is loaded and corresponding computing resources are scheduled based on the K8S tool, automatic machine learning training is performed according to the model training data by using the image file and the computing resources, an AI model is obtained, and the AI model is published on an AI transaction platform.
As shown in fig. 6, that is, fig. 6 shows that the scheduling method for scheduling the K8S platform specifically includes the following steps:
s1051, K8S dispatcher gets the cluster information from the application program interface server;
s1052, placing the unscheduled algorithm pod into a queue to be scheduled, and sequentially selecting deployment nodes;
s1053, the K8S dispatcher sends the deployment node to the application program interface server, and the application program interface server modifies the deployment node of the algorithm pod;
s1054, a daemon Kubelet on the deployment node creates an image of the pod.
Specifically, as shown in fig. 6, the Node3 and the Node2 correspond to a daemon process before scheduling, as shown in the block A, B in the figure, when scheduling is completed, a pod which is not scheduled before may appear in the daemon process, as shown in the block C in fig. 6, in the Node 1.
In some embodiments, as shown in FIG. 7, the automatic machine learning training may also allow the user to define model parameters including minimum model width, maximum model depth, minimum model depth, operator type, chip class, whether compressed or jumped topology form, etc., and send to the client for display and update its image file.
Specifically, a model parameter interface may be displayed at the client, specifically as shown in fig. 7, where the model parameter interface at least includes a minimum model width, a maximum model depth, a minimum model depth, an operator type, and a chip type, so that a user may select or determine a corresponding model parameter, and then perform training according to the model parameter determined by the user.
In some embodiments, the user may modify the parameter on the client, and send a confirmation modification instruction to the server after the modification is completed. And after the server defines the parameters according to the modification instruction, starting automatic training of the AI model.
In an embodiment of the application, a server receives transaction information input by a user through the client, and provides an access interface corresponding to the AI model for the user according to the transaction information. Therefore, convenience of model transaction can be improved, and user experience is improved.
Illustratively, a user inputs an order of an AI model with a classification function at a client, a server trains the AI model according to requirements, and an access interface corresponding to the AI model is provided for the user.
The methods of the present application are operational with numerous general purpose or special purpose computing system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet-type devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
Illustratively, the above-described method may be implemented in the form of a computer program that is executable on an electronic device as shown in fig. 8.
Referring to fig. 8, fig. 8 is a schematic view of an electronic device 200 according to an embodiment of the present disclosure. The electronic device may be a server or a terminal.
As shown in fig. 8, the electronic device 200 includes a processor 202 and a memory 201 connected by a system bus, wherein the memory may include a nonvolatile storage medium and an internal memory.
The non-volatile storage medium may store an operating system and a computer program. The computer program includes program instructions that, when executed, cause a processor to perform any one of the methods for implementing the AI model.
The processor is used for providing calculation and control capability and supporting the operation of the whole electronic equipment.
The internal memory provides an environment for running a computer program in a nonvolatile storage medium, and the computer program, when executed by the processor, causes the processor to execute any one of the implementation methods of the AI model.
Those skilled in the art will appreciate that the electronic device is merely a block diagram of a portion of the structure associated with the embodiments of the present application and does not constitute a limitation on the electronic device to which the embodiments of the present application may be applied, and that a particular electronic device may include more or less components than those shown, or combine certain components, or have a different arrangement of components.
It should be understood that the Processor may be a Central Processing Unit (CPU), and the Processor may be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, etc. Wherein a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
Wherein, in some embodiments, the processor is configured to execute a computer program stored in the memory to implement the steps of:
acquiring demand information issued by a user on a client of an AI transaction platform, wherein the demand information comprises description information of the user on a required AI model; determining a corresponding AI model algorithm according to the demand information, packaging the AI model algorithm into an image file, and loading the image file into the AI transaction platform;
carrying out hyper-parameter analysis on the mirror image file, and sending the hyper-parameters obtained by analysis to the client for display so that the user can confirm the hyper-parameters; obtaining model training data, and replacing a data path of the model training data with a specified path of the AI trading platform so as to map the data path to the client for display; and receiving and responding to the determination instruction of the user, loading the image file, performing automatic machine learning training according to the model training data to obtain an AI model, and issuing the AI model.
In some embodiments, a URL corresponding to the open source AI model algorithm collected by the user is obtained; and transmitting the URL into a preset interface of the AI trading platform so that the AI trading platform analyzes the codes of the open source AI model algorithm collected by the user according to the URL and maps the codes to the client for display.
In some embodiments, the client-displayed hyper-parameters allow a user to modify changes; and/or the code displayed by the client allows the client to modify.
In some embodiments, the processor, when implementing acquiring the requirement information published by the user on the client of the AI trading platform, is specifically configured to:
obtaining the model requirement of the user through the client of the AI trading platform; determining a target AI model and description information of the target AI model from a historical AI model according to the model requirements; displaying the description information through the client to facilitate the confirmation of the user; and when a confirmation instruction of the user on the description information of the target AI model is received, taking the description information of the target AI model as requirement information.
In some embodiments, the processor in implementing the determine whether the user provides an open source AI model algorithm is specifically configured to:
and if detecting that the user provides an open source AI model algorithm, transmitting the URL (uniform resource locator) into a preset interface of the AI trading platform so that the AI trading platform analyzes the code of the open source AI model algorithm collected by the user according to the URL (uniform resource locator) and maps the code to the client for display. If the fact that the user does not provide the open source AI model algorithm is detected, the AI trading platform searches the corresponding open source AI model algorithm through a third party code hosting platform, and the AI trading platform meets algorithm requirements of most AI models through collecting mainstream open source AI model algorithms.
In some embodiments, the processor is implemented to determine the model training data type, and is specifically configured to:
whether the user provides the open-source AI model algorithm or not is detected, and the determination can be carried out according to the data address of the AI model algorithm, if the URL corresponding to the open-source AI model algorithm is detected not to be recorded on the third-party code hosting platform, the determination is carried out on the open-source AI model algorithm provided by the user; and if the URL corresponding to the open-source AI model algorithm is recorded on the third-party code hosting platform, determining that the open-source AI model algorithm is not provided by the user.
In some embodiments, a container-based orchestration scheduling tool loads the image file and schedules the corresponding computing resources; and performing automatic machine learning training according to the model training data by using the mirror image file and the computing resources.
In some embodiments, a user inputs transaction information through the client, and the access interface corresponding to the AI model is provided to the user according to the transaction information.
The embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, where the computer program includes program instructions, and the program instructions, when executed, implement any one of the methods for implementing the AI model provided in the embodiment of the present application.
The computer-readable storage medium may be an internal storage unit of the electronic device according to the foregoing embodiment, for example, a hard disk or a memory of the electronic device. The computer readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, provided on the electronic device.
Further, the computer-readable storage medium may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function, and the like; the storage data area may store data created according to the use of the blockchain node, and the like.
The application refers to a novel application mode of computer technologies such as storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like of a block chain language model. A block chain (Blockchain), which is essentially a decentralized database, is a string of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, which is used for verifying the validity (anti-counterfeiting) of the information and generating a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
While the invention has been described with reference to specific embodiments, the scope of the invention is not limited thereto, and those skilled in the art can easily conceive various equivalent modifications or substitutions within the technical scope of the invention. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (10)

1. An AI model implementation method, comprising:
acquiring demand information issued by a user on a client of an AI transaction platform, wherein the demand information comprises description information of the user on a required AI model;
determining a corresponding AI model algorithm according to the demand information, packaging the AI model algorithm into an image file, and loading the image file into the AI transaction platform;
carrying out hyper-parameter analysis on the mirror image file, and sending the hyper-parameters obtained by analysis to the client for display so that the user can confirm the hyper-parameters;
obtaining model training data, and replacing a data path of the model training data with a specified path of the AI trading platform so as to map the data path to the client for display;
and receiving and responding to the determination instruction of the user, loading the image file, performing automatic machine learning training according to the model training data to obtain an AI model, and issuing the AI model.
2. The method of claim 1, wherein the AI model algorithms comprise open source AI model algorithms collected by the AI trading platform or open source AI model algorithms collected by a user;
if the AI model algorithm is an open source AI model algorithm collected by the user, the method further comprises:
acquiring a URL corresponding to the open source AI model algorithm collected by the user;
and transmitting the URL into a preset interface of the AI trading platform so that the AI trading platform analyzes the codes of the open source AI model algorithm collected by the user according to the URL and maps the codes to the client for display.
3. The method of claim 2, wherein the client-displayed hyper-parameters allow user modification; and/or the code displayed by the client allows the user to modify.
4. The method of claim 1, wherein the model training data comprises open source data or proprietary data, the proprietary data providing training data for the user;
and if the model training data is the owned data, converting the data format of the owned data into the data format of open source data.
5. The method of claim 4, further comprising:
and displaying the data formats of the open source data of a plurality of predefined different tasks at the client of the AI trading platform so that a user can convert the data format of the own data according to the data format of the open source data.
6. The method of claim 1, wherein the AI trading platform includes a container orchestration scheduling tool, and wherein the loading the image file and performing automatic machine learning training according to the model training data comprises:
loading the mirror image file and scheduling corresponding computing resources based on the container scheduling tool;
and performing automatic machine learning training according to the model training data by using the mirror image file and the computing resources.
7. The method of claim 1, wherein the publishing the AI model comprises:
and receiving the transaction information input by the user through the client, and providing the access interface corresponding to the AI model for the user according to the transaction information.
8. The method according to claim 1, wherein the acquiring the demand information published by the user on the client of the AI trading platform comprises:
obtaining the model requirement of the user through the client of the AI trading platform;
determining a target AI model and description information of the target AI model from a historical AI model according to the model requirements;
displaying the description information through the client to facilitate the confirmation of the user;
and in response to receiving a confirmation instruction of the user on the description information, taking the description information as requirement information.
9. An electronic device, comprising a memory and a processor;
the memory for storing a computer program;
the processor is used for executing the computer program and realizing the following when the computer program is executed:
implementation of the AI model according to any of claims 1 to 8.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which, when executed by a processor, causes the processor to implement the implementing method of the AI model according to any one of claims 1 to 8.
CN202011346401.4A 2020-11-25 2020-11-25 AI model realization method, electronic device and storage medium Pending CN114610273A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011346401.4A CN114610273A (en) 2020-11-25 2020-11-25 AI model realization method, electronic device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011346401.4A CN114610273A (en) 2020-11-25 2020-11-25 AI model realization method, electronic device and storage medium

Publications (1)

Publication Number Publication Date
CN114610273A true CN114610273A (en) 2022-06-10

Family

ID=81857094

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011346401.4A Pending CN114610273A (en) 2020-11-25 2020-11-25 AI model realization method, electronic device and storage medium

Country Status (1)

Country Link
CN (1) CN114610273A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114780110A (en) * 2022-06-21 2022-07-22 山东极视角科技有限公司 Optimization method and optimization system of algorithm link
CN115525298A (en) * 2022-11-28 2022-12-27 长沙海信智能系统研究院有限公司 AI scene algorithm access deployment method and device and electronic equipment

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114780110A (en) * 2022-06-21 2022-07-22 山东极视角科技有限公司 Optimization method and optimization system of algorithm link
CN114780110B (en) * 2022-06-21 2022-09-09 山东极视角科技有限公司 Optimization method and optimization system of algorithm link
CN115525298A (en) * 2022-11-28 2022-12-27 长沙海信智能系统研究院有限公司 AI scene algorithm access deployment method and device and electronic equipment

Similar Documents

Publication Publication Date Title
US11327675B2 (en) Data migration
EP2808790B1 (en) Migration assessment for cloud computing platforms
US20060190280A1 (en) Method and apparatus for management for use in fleet service and logistics
US11667033B2 (en) Systems and methods for robotic process automation
CN111181773B (en) Delay prediction method for multi-component application of heterogeneous border cloud collaborative intelligent system
EP3686750A1 (en) Data migration
Raj Engineering MLOps: Rapidly build, test, and manage production-ready machine learning life cycles at scale
CN114610273A (en) AI model realization method, electronic device and storage medium
US20160239275A1 (en) Generating an integrated service
CA3170083C (en) Systems and methods for project and program management using artificial intelligence
US20230351308A1 (en) System and method for universal mapping of structured, semi-structured, and unstructured data for application migration in integration processes
CN114035793A (en) Page generation method, page generation device, equipment and storage medium
CN117223016A (en) Industry specific machine learning application
CN111813739A (en) Data migration method and device, computer equipment and storage medium
Rosales-Morales et al. An analysis of tools for automatic software development and automatic code generation
CN114691122A (en) Service operation method and device, electronic equipment and storage medium
CN117114909A (en) Method, device, equipment and storage medium for constructing accounting rule engine
Deshmukh et al. Transforming Next Generation-Based Artificial Intelligence for Software Development: Current Status, Issues, Challenges, and Future Opportunities
US20230215061A1 (en) Project visualization system
CN113867700B (en) Model construction method, display platform, server and storage medium
WO2022140650A2 (en) Systems and methods for building and deploying machine learning applications
US20210342737A1 (en) Ai/ml based proactive system to improve sales productivity by categorizing and determining relevant news
Grzech et al. Methodology and platform for business process optimization
US20220398129A1 (en) Application prioritization system
US20240086188A1 (en) Automatic navigation between reference architecture and code repository

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination