CN112988165A - Kubernetes-based interactive modeling method and device, electronic equipment and storage medium - Google Patents
Kubernetes-based interactive modeling method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112988165A CN112988165A CN202110407172.0A CN202110407172A CN112988165A CN 112988165 A CN112988165 A CN 112988165A CN 202110407172 A CN202110407172 A CN 202110407172A CN 112988165 A CN112988165 A CN 112988165A
- Authority
- CN
- China
- Prior art keywords
- code
- interactive modeling
- environment
- interactive
- development environment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/40—Transformation of program code
- G06F8/41—Compilation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/30—Creation or generation of source code
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
- G06F8/61—Installation
- G06F8/63—Image based installation; Cloning; Build to order
Abstract
The application provides an interactive modeling method and device based on Kubernetes, electronic equipment and a storage medium, and relates to the technical field of neural network models. The method comprises the following steps: configuring an instance at a front-end page; building a Kubernetes cluster, and connecting the Kubernetes cluster with a back-end server; starting an interactive modeling environment based on Jupiter notewood or an online integrated development environment in a cluster; and calling a computing framework integrated in the Notebook, or creating a model based on an example by using real-time code completion and error prompt of an online integrated development environment. The method is based on Kubernets and an interactive modeling environment comprising Notebook and an online integrated development environment, so that the common complex steps of data processing, algorithm development and the like are shielded, the corresponding development environment can be simply and quickly accessed, and the model development efficiency is improved.
Description
Technical Field
The application relates to the technical field of neural network models, in particular to an interactive modeling method and device based on Kubernetes, electronic equipment and a storage medium.
Background
Due to the rapid development of artificial intelligence and big data, people who work in data analysis, big data development, machine learning algorithm, deep learning algorithm, data visualization, business intelligence and the like are more and more. At present, the development languages used by the users are various, but the mainstream programming language is Python, the Python is an interpretation type script language, the operation is easy, a lot of data analysis, data visualization, algorithm packages and the like exist, and the development languages become the preferred languages in the fields of big data and artificial intelligence. However, Python is a development language software, is not a Web program, cannot be used after being opened, needs to be installed, cannot be remotely accessed, and cannot be cooperatively developed. Jupyter notewood is an open source Web application based on Python, allowing users to create and share file formats including code, functions, visualizations, and text documents. Its uses include: data cleaning and conversion, numerical simulation, statistical modeling, data visualization, machine learning, and the like. The Jupyter notewood is an interactive Python development environment, and a user can write Python codes at a webpage end and execute the codes one line by one line. However, jupitter notewood is a general interactive environment, and there are some problems in performing algorithm modeling, such as: the deep learning and machine learning algorithm framework is endless, the development environment and the dependency package installation process are complicated, and the problems of complicated model development steps and low efficiency exist in the model development.
Disclosure of Invention
In view of this, an object of the embodiments of the present application is to provide an interactive modeling method and apparatus based on Kubernetes, an electronic device, and a storage medium, so as to solve the problems of complex model development steps and low efficiency in the prior art.
The embodiment of the application provides an interactive modeling method based on Kubernetes, which comprises the following steps: configuring an instance at a front-end page; building a Kubernetes cluster, and connecting the Kubernetes cluster with a back-end server; starting an interactive modeling environment based on Jupiter notewood or an online integrated development environment in the Kubernets cluster; and calling a calculation framework creation model integrated in the Jupiter Notebook in the interactive modeling environment based on the example, or calling a real-time code completion and error prompt function creation model of the online integrated development environment in the interactive modeling environment based on the example.
In the implementation mode, based on Kubernets and an interactive modeling environment comprising Jupitter notewood or an online integrated development environment, the complex installation of commonly used frames such as data processing, algorithm development and the like is shielded, so that a user can simply and quickly enter a corresponding development ring, and the efficiency of developing a neural network model is improved.
Optionally, before the interactive modeling environment based on jupiter notewood or an online integrated development environment is started in the kubernets cluster, the method further includes: integrating the computing framework into the Jupyter notewood; and carrying out sharing setting on the Jupyter notewood.
In the implementation mode, file sharing and computing frame integration of different accounts are realized in Jupyter notewood, so that a basic condition is provided for different users in an interactive environment to perform online interactive modeling by utilizing computing frame resources in respective files.
Optionally, the integrating the computing framework into the Jupyter notewood includes: integrating the computing framework into JupyterLab using ipkerrel; making a mirror image file based on the computing framework; and uploading the image file to an image warehouse of the JupyterLab to complete the integration of the computing framework in the JupyterNotebook.
In the implementation mode, the calculation frame is integrated in the Jupiter Notebook, so that a user can directly perform modeling in the mode of the calculation frame in the modeling process, the modeling complexity is reduced, and the modeling efficiency is improved.
Optionally, the performing sharing setting on the Jupyter notewood includes: adopting logic of an interactive system to obtain an account password to replace the login code of the JupyterLab; and creating a shared file under a Jupyterhub/handles directory, and performing shared setting on an account in the JupyterLab based on object storage.
In the implementation mode, the user can automatically log in the account and share modeling resources under other accounts through the replacement of the login code and the account sharing setting based on the object storage, and the modeling efficiency is improved.
Optionally, before the interactive modeling environment based on jupiter notewood or an online integrated development environment is started in the kubernets cluster, the method further includes: modifying the source code of the code editor; integrating the modified code editor as the online integrated development environment into the interactive modeling environment; in the interactive modeling environment, one instance is created for each user and the resources between each instance are not shared.
In the implementation mode, on the basis that the account can share the resources, the data, the files and the resources are isolated based on Jupyterhub, and the environmental isolation among interactive modeling personnel can be realized, so that the safety of modeling data is improved.
Optionally, the Code editor is a Visual Studio Code, and the performing source Code modification on the Code editor includes: downloading a source Code of the Visual Studio Code; and modifying front-end page display options of the source code to form the online integrated development environment.
In the implementation mode, the Visual Studio Code editor is improved and integrated into an online integrated development environment, and an online IDE (integrated development environment) which is an interactive modeling capability is provided, so that the problems of Code error real-time prompt and automatic completion during development are solved.
Optionally, the integrating the modified code editor into the interactive modeling environment as the online integrated development environment includes: compiling and packaging the source code with the modified display options, and making the source code into a Docker mirror image; deploying the Docker mirror image, and configuring an access domain name and a link of the Docker mirror image; integrating the online integrated development environment into the interactive modeling environment at the front-end page based on the access domain name and the link.
In the implementation mode, the source code is compiled, packaged and then deployed and configured as a mirror image, so that each user can perform real-time prompt and automatic completion functions of code errors of an online integrated development environment through the front end during instance execution, and the modeling efficiency is improved.
An embodiment of the present application further provides an interactive modeling apparatus based on kubernets, where the apparatus includes: the instance configuration module is used for configuring instances on the front-end page; the cluster building module is used for building a Kubernetes cluster and connecting the Kubernetes cluster with a back-end server; the starting module is used for starting an interactive modeling environment based on Jupiter notewood or an online integrated development environment in the Kubernets cluster; and the calling module is used for calling a calculation frame creation model integrated in the Jupiter Notebook in the interactive modeling environment based on the example, or calling a real-time code completion and error prompt function creation model of the online integrated development environment in the interactive modeling environment based on the example.
In the implementation mode, based on Kubernets and an interactive modeling environment comprising Jupitter notewood or an online integrated development environment, the complex installation of commonly used frames such as data processing, algorithm development and the like is shielded, so that a user can simply and quickly enter the corresponding development environment, the efficiency of neural network model development is improved, meanwhile, the complexity of model development is further simplified and the development efficiency is improved by calling a calculation frame.
Optionally, the Kubernetes-based interactive modeling apparatus further includes: a computing frame integration module, configured to integrate the computing frame into the Jupyter notewood; and carrying out sharing setting on the Jupyter notewood.
In the implementation mode, file sharing and computing frame integration of different accounts are realized in Jupyter notewood, so that a basic condition is provided for different users in an interactive environment to perform online interactive modeling by utilizing computing frame resources in respective files.
Optionally, the computing framework integration module is specifically configured to: integrating the computing framework into JupyterLab using ipkerrel; making a mirror image file based on the computing framework; and uploading the image file to an image warehouse of the JupyterLab to complete the integration of the computing framework in the JupyterNotebook.
In the implementation mode, the calculation frame is integrated in the Jupiter Notebook, so that a user can directly perform modeling in the mode of the calculation frame in the modeling process, the modeling complexity is reduced, and the modeling efficiency is improved.
Optionally, the computing framework integration module is specifically configured to: adopting logic of an interactive system to obtain an account password to replace the login code of the JupyterLab; and creating a shared file under a Jupyterhub/handles directory, and performing shared setting on an account in the JupyterLab based on object storage.
In the implementation mode, the user can automatically log in the account and share modeling resources under other accounts through the replacement of the login code and the account sharing setting based on the object storage, and the modeling efficiency is improved.
Optionally, the Kubernetes-based interactive modeling apparatus further includes: the online integrated development environment integration module is used for modifying the source code of the code editor; integrating the modified code editor as the online integrated development environment into the interactive modeling environment; in the interactive modeling environment, one instance is created for each user and the resources between each instance are not shared.
In the implementation mode, on the basis that the account can share the resources, the data, the files and the resources are isolated based on Jupyterhub, and the environmental isolation among interactive modeling personnel can be realized, so that the safety of modeling data is improved.
Optionally, the online integrated development environment integration module is specifically configured to: downloading a source Code of the Visual Studio Code; and modifying front-end page display options of the source code to form the online integrated development environment.
In the implementation mode, the Visual Studio Code editor is improved and integrated into an online integrated development environment, and an online IDE (integrated development environment) which is an interactive modeling capability is provided, so that the problems of Code error real-time prompt and automatic completion during development are solved.
Optionally, the online integrated development environment integration module is specifically configured to: compiling and packaging the source code with the modified display options, and making the source code into a Docker mirror image; deploying the Docker mirror image, and configuring an access domain name and a link of the Docker mirror image; integrating the online integrated development environment into the interactive modeling environment at the front-end page based on the access domain name and the link.
In the implementation mode, the source code is compiled, packaged and then deployed and configured as a mirror image, so that each user can perform real-time prompt and automatic completion functions of code errors of an online integrated development environment through the front end during instance execution, and the modeling efficiency is improved.
An embodiment of the present application further provides an electronic device, where the electronic device includes a memory and a processor, where the memory stores program instructions, and the processor executes steps in any one of the above implementation manners when reading and executing the program instructions.
The embodiment of the present application further provides a readable storage medium, in which computer program instructions are stored, and the computer program instructions are read by a processor and executed to perform the steps in any of the above implementation manners.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.
Fig. 1 is a flowchart illustrating a computing framework integration step according to an embodiment of the present application.
Fig. 2 is a flowchart illustrating an integration step of an online integrated development environment according to an embodiment of the present application.
Fig. 3 is a schematic flowchart of an interactive modeling method based on Kubernetes according to an embodiment of the present application.
Fig. 4 is a schematic block diagram of an interactive modeling apparatus based on Kubernetes according to an embodiment of the present application.
Icon: 50-interactive modeling apparatus based on Kubernetes; 51-instance configuration module; 52-cluster building module; 53-a start module; 54-calling the module.
Detailed Description
The technical solution in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.
The research of the applicant finds that the existing interactive modeling mainly has the following problems: 1, a Jupyter Notebook is simply built based on a local computer or a server, a computing frame required by deep learning is developed, for example, Tensorflow, Mxnet, Pythroch, Keras and the like need to install a frame version corresponding to a CPU/GPU by self, various packages depended on by machine learning are developed, for example, numpy, dateFrame and the like need to be installed by self, the operating environment of Python also has inconsistency of various versions, various software packages depended by self need to be installed, the installation process is complex and tedious, the compatibility of software also has various problems, time and labor are wasted, errors are easy to occur, and particularly, for people who do not know a computer, the algorithm research or data analysis based on the Jupyter Notebook is particularly difficult; 2, interactive modeling only supports a simple Python environment and does not support visualization of the model and interpretability of the model, particularly, deep learning requires the model to be interpretable, some visualization means are needed to help the model analysis, the visualization means all need a user to use a command to install in Jupitter, the installation is complex, and many people cannot use the visualization means; 3, the management of the interactive modeling code needs to be uploaded to Git offline and is not integrated into a notewood (note wood in this embodiment is abbreviated as Jupyter note wood), and the difficulty and complexity of code management are increased by back-and-forth code uploading, updating and the like; 4, some users do not like Jupyter notewood modeling, because the users do not have the automatic code completion and code prompting capabilities, the syntax for developing the code needs to be written by themselves, whether a mistake exists or not can not be sensed in time, the mistake can be seen after the compiling is executed, and the development is difficult.
The Jupitter notewood is opened in a webpage form, codes and running codes can be directly written in a webpage, and running results of the codes can be directly displayed under a code block. If the explanation document needs to be written in the programming process, the explanation document can be directly written in the same page, so that the explanation and the explanation can be conveniently carried out in time.
Therefore, in order to solve the above problems, an embodiment of the present application provides an interactive modeling method based on kubernets, and first, an explanation is made of an integration step of jupiter notewood, an online integrated development environment, and the like in the interactive modeling method based on kubernets.
Among them, Kubernetes is abbreviated as K8s, and 8 characters "ubernete" are replaced with 8. The Kubernets aims to make the application of container deployment simple and efficient, and provides a mechanism for application deployment, planning, updating and maintenance.
Optionally, Git, which is an open-source distributed version control system, may be integrated in an environment where the interactive modeling method based on Kubernetes is applied in this embodiment, and may effectively and quickly process version management of projects from very small to very large. Codes developed by interactive modeling need to be submitted and stored in a code warehouse in time, otherwise the codes are easy to lose, and Git is a code management tool and can submit local codes to a remote code warehouse Gitlab and pull the codes from the remote code warehouse.
Specifically, the code submission steps during interactive modeling development in this embodiment may be as follows:
step S11: and (5) establishing a Gitlab at the back-end server, newly establishing a modeling project, and returning to project Git address connection.
Step S12: a Terminal is opened in a page provided in Jupiter, and a git clone command is used for downloading the original code of the project.
Step S13: for code developed based on original code, the developed code is converted into the ipynb format.
Step S14: and (4) pulling the code by using git pull, and remotely submitting the developed code by using git pull.
The following is the integration of Jupyter notewood, and the steps of the Jupyter notewood integration may be specifically as follows: integrating a computing frame into Jupyter notewood; and carrying out sharing setting on Jupyter notewood.
Referring to fig. 1 for an integration step of a computing framework, fig. 1 is a schematic flowchart of an integration step of a computing framework provided in an embodiment of the present application, and the integration step of the computing framework may specifically be as follows:
step S21: the computing framework is integrated into JupyterLab using ipykernel.
The JupyterHub is an integrated service system for providing Jupyter notewood for a plurality of users, user environments under the existing JupyterHub are isolated from each other and cannot be shared with each other, and each user needs to configure own Python/Conda environment and the like independently, so that data and resources can be isolated. When some model or algorithm files need to be shared to other people, the sharing becomes troublesome, the sharing needs to be transmitted by means of other file transmission tools, and at the same time, JupitterHub is an account system of the JupitterHub and is not communicated with other accounts, so that automatic login cannot be realized during login skip, a password needs to be input, the JupitterHub only integrates a bottom API (Application Programming Interface) of Python, and common data processing frames and algorithm frames need to be installed by themselves and are cumbersome to install.
Optionally, the computing framework in this embodiment may include a common big data processing framework, a machine learning framework, a deep learning framework, a model visualization framework, and the like, and specifically, the integrated framework mainly includes: pyspeak, Mxnet, Pytrch, Caffe, Tensorflow, Conda-Python, Xgboost, Sklearn, Tensorbboard, and the like. In the prior art, the default notewood only supports Python3, each algorithm frame needs to be installed by itself and cannot be opened and used directly, and a newly-built Python file needs to be installed repeatedly, which consumes time and labor. Therefore, in the embodiment, each computing framework is installed in advance, and is integrated into JupyterLab by using ipyKernel.
For example, such as integrating Mxnet, the commands are: the installation commands for the rest of the frameworks are similar.
Step S22: and making an image file based on the computing framework.
Optionally, the image file of the computing framework in this embodiment may be a Dockerfile image file, the Dockerfile is a text file containing all commands for creating an image, the image may be created according to the content of the Dockerfile through the Docker build command, and the Docker may automatically create the image through the content of the Dockerfile.
And putting all needed calculation frames into Dockerfile, and setting jupyter/base-notebook by the basic mirror image. Taking the mirror image production of Mxnet as an example, Mxnet is installed first, then Ipykernel is used to mount Mxnet into jupyterLab, and the core command is as follows:
from jupyter/base-notebook
pip install mxnet
python -m ipykernel install --user --name Mxnet--display-name "Python Mxnet"
the commands are used for all needed computing frames, and the commands are assembled together and edited into a Docker file.
Step S23: and uploading the image file to an image warehouse of JupyterLab to complete the integration of the computing framework in Jupyter Notebook.
Compiling and packaging the mirror image by using the Docker build, uploading the mirror image to a mirror image warehouse and naming the mirror image, namely xwfintech/notebook. So far, the integration of the computing frame in Jupiter Notebook can be completed, and some common data processing frames, algorithm frames and model visualization frames can be integrated into Jupiter Notebook, and can be directly and automatically installed and directly developed by clicking on a page.
Next, file sharing setting of Jupyter notewood is performed, and the specific steps may include:
step S24: and adopting the logic of the interactive system for acquiring the account password to replace the login code of JupyterLab.
Specifically, logging-in codes are logged out in jupyterhub/handles/login.
Step S25: and creating a shared file under the Jupiterbhub/handles directory, and performing shared setting on the account in the JupiterbLab based on the object storage.
In this embodiment, in order to implement file sharing between an accountA account and an accountB, a share file is created under a jupyterhub/handles directory, an upload interface and a download interface of the file are implemented in the py file by using object storage and code writing, the upload interface and the download interface are added to a tornado in jupyterhub/extension, and a menu of a right key is developed on a front-end page to implement sharing of a right key selection file. The detailed interaction process is as follows: the accountA shares a file to the accountB, the accountA uploads the file to an object for storage, the Python end records the user name and the file name of the accountA and the user name information shared to the account accountB to a database, the accountB obtains the information shared to the accountB in the database through a user-defined interface in a jupylerLab, and the file is downloaded to a folder of the accountB through a download interface, so that sharing is achieved.
Based on the Jupyter notewood modeling, if the Python programming syntax is not familiar, or the programming syntax of the mainstream machine learning or deep learning operation framework is not familiar (such as skleran, Tensorflow, Mxnet, Caffe and the like), the Jupyter notewood is inconvenient to use because the Jupyter notewood has no code error prompt and code automatic completion capability, the development efficiency is slow, the error can be seen in time only by compiling and executing at the same time, and the unfamiliar programming syntax can be found in time to know how to use. Therefore, an editor capable of performing automatic code completion and error prompt is needed, and after the integration of Jupyter notewood is completed, the online integrated development environment also needs to be integrated, please refer to fig. 2, where fig. 2 is a schematic flow diagram of an online integrated development environment integration step provided in the embodiment of the present application, and the online integrated development environment integration step may specifically be as follows:
step S31: and modifying the source code for the code editor.
Optionally, the Code editor in this embodiment may be a Visual Studio Code (VSCode/VSC for short), which is a free-sourced modern lightweight Code editor, supports characteristics of syntax highlighting, intelligent Code completion, custom hotkeys, bracket matching, Code fragments, Code comparison Diff, Git, and the like of almost all mainstream development languages, supports plug-in extension, and is optimized for web page development and cloud application development.
In this embodiment, the front-end page display options of the source code are modified, the home page display style is mainly modified, and redundant menus and buttons are shielded, so that a customized online integrated development environment is formed.
Among them, an Integrated Development Environment (IDE) is an application program for providing a program Development Environment, and generally includes tools such as a code editor, a compiler, a debugger, and a graphical user interface. The integrated development software service set integrates a code compiling function, an analyzing function, a compiling function, a debugging function and the like.
Step S32: and integrating the modified code editor into an interactive modeling environment as an online integrated development environment.
Specifically, step S32 may include the following sub-steps:
step S321: and compiling and packaging the source code with the modified display options, and manufacturing the source code into a Docker mirror image.
In the embodiment, the source code is recompiled, and is packaged by means of steps of https:// github.com/cdr/code-server/tree/v3.9.0/ci pages, and is manufactured into a Docker mirror image named xwfitech/ide.
Step S322: and deploying the Docker image, and configuring an access domain name and a link of the Docker image.
Optionally, a Docker run may be deployed for the Docker image, and subsequently configured.
Step S323: an online integrated development environment is integrated into the interactive modeling environment at the front-end page based on accessing the domain name and the link.
Specifically, the present embodiment integrates an online IDE into an interactive modeling system using iframe technology at the front-end page of the interactive modeling environment.
Step S33: in an interactive modeling environment, one instance is created for each user, and resources are not shared between each instance.
After completing the integration of the jupiter notewood and the interactive modeling environment of the online integrated development environment, interactive modeling may be performed through the interactive modeling environment during modeling, please refer to fig. 3, where fig. 3 is a schematic flow diagram of an interactive modeling method based on kubernets according to an embodiment of the present application, and the specific steps of the interactive modeling method based on kubernets may be as follows:
step S42: the example is configured at the front end page.
Specifically, the name of the instance is configured on the front-end page, the number of CPUs (Central Processing units) required for configuring the instance, the number of GPUs (Graphics Processing units), the memory used, and the number of cores used.
Step S44: and building a Kubernetes cluster, and connecting the Kubernetes cluster to a back-end server.
Alternatively, the backend server may connect to the kubernets cluster through a Java API.
Step S46: an interactive modeling environment based on Jupiter notewood or an online integrated development environment is started in a Kubernets cluster.
Specifically, using a kubecect run keras-app-image = xwfitech/notebook-port 5000, an interactive modeling environment based on jupitertnotewood may be automatically launched.
Using a kubecect run keras-app-image = xwfitech/ide-port 50001, an interactive modeling environment based on an online integrated development environment can be launched.
Step S48: and calling a calculation framework integrated in Jupyter Notebook to create a model based on an instance in an interactive modeling environment, or calling a real-time code completion and error prompt function creation model of an online integrated development environment based on an instance in an interactive modeling environment.
After the system is started, the system enters an interactive modeling environment, common data processing, an algorithm frame and a model visualization frame are integrated in Jupitter Notebook, the system can be directly used when being clicked, files in various coding formats can be built in an online integrated development environment, corresponding code development is carried out, and intelligent real-time code automatic completion and error prompt are accompanied.
In order to cooperate with the above-mentioned interactive modeling method based on kubernets provided in this embodiment, an interactive modeling apparatus 50 based on kubernets is further provided in this embodiment of the application, please refer to fig. 4, and fig. 4 is a schematic module diagram of the interactive modeling apparatus based on kubernets provided in this embodiment of the application.
The Kubernetes-based interactive modeling apparatus 50 includes:
an instance configuration module 51 for configuring instances on front-end pages;
the cluster building module 52 is used for building a Kubernetes cluster and connecting the Kubernetes cluster with a back-end server;
the starting module 53 is used for starting an interactive modeling environment based on Jupiter notewood or an online integrated development environment in a Kubernets cluster;
and the calling module 54 is used for calling a calculation framework creation model integrated in Jupitter Notebook in the interactive modeling environment based on the example, or calling a real-time code completion and error prompt function creation model of an online integrated development environment in the interactive modeling environment based on the example.
Optionally, the Kubernetes-based interactive modeling apparatus 50 further includes: the computing frame integration module is used for integrating the computing frame into Jupitter notewood; and carrying out sharing setting on Jupyter notewood.
Optionally, the computing framework integration module is specifically configured to: integrating the computing framework into JupyterLab by using ipyKernel; manufacturing a mirror image file based on a computing framework; and uploading the image file to an image warehouse of JupyterLab to complete the integration of the computing framework in Jupyter Notebook.
Optionally, the computing framework integration module is specifically configured to: adopting an interactive system to acquire a login code of a logic replacement JupyterLab of an account password; and creating a shared file under the Jupiterbhub/handles directory, and performing shared setting on the account in the JupiterbLab based on the object storage.
Optionally, the Kubernetes-based interactive modeling apparatus 50 further includes: the online integrated development environment integration module is used for modifying the source code of the code editor; integrating the modified code editor into an interactive modeling environment as an online integrated development environment; in an interactive modeling environment, one instance is created for each user, and resources are not shared between each instance.
Optionally, the online integrated development environment integration module is specifically configured to: downloading a source Code of the Visual Studio Code; and performing front-end page display option modification on the source code to form an online integrated development environment.
Optionally, the online integrated development environment integration module is specifically configured to: compiling and packaging the source code with the modified display options, and making the source code into a Docker mirror image; deploying a Docker mirror image, and configuring an access domain name and a link of the Docker mirror image; an online integrated development environment is integrated into the interactive modeling environment at the front-end page based on accessing the domain name and the link.
An embodiment of the present application further provides electronic equipment, where the electronic equipment includes a memory and a processor, where the memory stores program instructions, and when the processor reads and runs the program instructions, the processor executes steps in any one of the kubernets-based interactive modeling methods provided in this embodiment.
It should be understood that the electronic device may be a Personal Computer (PC), a tablet PC, a smart phone, a Personal Digital Assistant (PDA), or other electronic device having a logical computing function.
The embodiment of the application also provides a readable storage medium, wherein computer program instructions are stored in the readable storage medium, and the computer program instructions are read by a processor and executed to execute the steps in the Kubernetes-based interactive modeling method.
To sum up, the embodiment of the present application provides an interactive modeling method, an interactive modeling device, an electronic device, and a storage medium based on kubernets, where the method includes: configuring an instance at a front-end page; building a Kubernetes cluster, and connecting the Kubernetes cluster with a back-end server; starting an interactive modeling environment based on Jupiter notewood or an online integrated development environment in the Kubernets cluster; and in the interactive modeling environment, calling a computing framework integrated in the Jupiter Notebook, or creating a model based on the instance by using real-time code completion and error prompt of the online integrated development environment.
In the implementation mode, the problems of back-and-forth transmission and poor management of developed algorithm codes under a line can be solved by integrating Git, the problems of development packages and complex installation of the algorithm packages which are depended when the Jupyter Notebook is used for developing the algorithm are solved by a Docker technology, the problems of no code error prompt and automatic completion existing when the Jupyter Notebook is used are solved by an online IDE technology, and the problem of automatic start and stop of an interactive modeling system instance is solved by Kubernets. The method greatly improves the efficiency of algorithm development by using interactive modeling, reduces the use threshold, and solves the problem that the existing method for using Jupitter notewood to carry out interaction is modeling.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus may be implemented in other manners. The apparatus embodiments described above are merely illustrative, and for example, the block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of devices according to various embodiments of the present application. In this regard, each block in the block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams, and combinations of blocks in the block diagrams, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Therefore, the present embodiment further provides a readable storage medium, in which computer program instructions are stored, and when the computer program instructions are read and executed by a processor, the computer program instructions perform the steps of any of the block data storage methods. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a RanDom Access Memory (RAM), a magnetic disk, or an optical disk.
The above description is only an example of the present application and is not intended to limit the scope of the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Claims (10)
1. An interactive modeling method based on Kubernetes is characterized by comprising the following steps:
configuring an instance at a front-end page;
building a Kubernetes cluster, and connecting the Kubernetes cluster with a back-end server;
starting an interactive modeling environment based on Jupiter notewood or an online integrated development environment in the Kubernets cluster;
and calling a calculation framework creation model integrated in the Jupiter Notebook in the interactive modeling environment based on the example, or calling a real-time code completion and error prompt function creation model of the online integrated development environment in the interactive modeling environment based on the example.
2. The method of claim 1, wherein prior to initiating an interactive modeling environment based on Jupiter notewood or an online integrated development environment in the Kubernets cluster, the method further comprises:
integrating the computing framework into the Jupyter notewood;
and carrying out sharing setting on the Jupyter notewood.
3. The method of claim 2, wherein said integrating said computing framework into said Jupyter notewood comprises:
integrating the computing framework into JupyterLab using ipkerrel;
making a mirror image file based on the computing framework;
and uploading the image file to an image warehouse of the JupyterLab to complete the integration of the computing framework in the JupyterNotebook.
4. The method of claim 3, wherein the performing a shared setup on the Jupyter notewood comprises:
adopting logic of an interactive system to obtain an account password to replace the login code of the JupyterLab;
and creating a shared file under a Jupyterhub/handles directory, and performing shared setting on an account in the JupyterLab based on object storage.
5. The method of claim 1, wherein prior to initiating an interactive modeling environment based on Jupiter notewood or an online integrated development environment in the Kubernets cluster, the method further comprises:
modifying the source code of the code editor;
integrating the modified code editor as the online integrated development environment into the interactive modeling environment;
in the interactive modeling environment, one instance is created for each user and the resources between each instance are not shared.
6. The method of claim 5, wherein the Code editor is a Visual Studio Code, and wherein the source Code modification of the Code editor comprises:
downloading a source Code of the Visual Studio Code;
and modifying front-end page display options of the source code to form the online integrated development environment.
7. The method of claim 6, wherein said integrating the modified code editor into the interactive modeling environment as the online integrated development environment comprises:
compiling and packaging the source code with the modified display options, and making the source code into a Docker mirror image;
deploying the Docker mirror image, and configuring an access domain name and a link of the Docker mirror image;
integrating the online integrated development environment into the interactive modeling environment at the front-end page based on the access domain name and the link.
8. An interactive Kubernetes-based modeling apparatus, the apparatus comprising:
the instance configuration module is used for configuring instances on the front-end page;
the cluster building module is used for building a Kubernetes cluster and connecting the Kubernetes cluster with a back-end server;
the starting module is used for starting an interactive modeling environment based on Jupiter notewood or an online integrated development environment in the Kubernets cluster;
and the calling module is used for calling the computing framework integrated in the Jupiter Notebook in the interactive modeling environment based on the example, or calling the real-time code completion and error prompt function creation model of the online integrated development environment in the interactive modeling environment based on the example.
9. An electronic device comprising a memory having stored therein program instructions and a processor that, when executed, performs the steps of the method of any of claims 1-7.
10. A storage medium having stored thereon computer program instructions for executing the steps of the method according to any one of claims 1 to 7 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110407172.0A CN112988165A (en) | 2021-04-15 | 2021-04-15 | Kubernetes-based interactive modeling method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110407172.0A CN112988165A (en) | 2021-04-15 | 2021-04-15 | Kubernetes-based interactive modeling method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112988165A true CN112988165A (en) | 2021-06-18 |
Family
ID=76340699
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110407172.0A Pending CN112988165A (en) | 2021-04-15 | 2021-04-15 | Kubernetes-based interactive modeling method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112988165A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115048087A (en) * | 2022-08-15 | 2022-09-13 | 江苏博云科技股份有限公司 | Method, equipment and storage medium for realizing online IDE tool in Kubernetes environment |
| CN116627446A (en) * | 2023-07-25 | 2023-08-22 | 华控清交信息科技(北京)有限公司 | Method and device for deploying application on k8s container cluster and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109614202A (en) * | 2018-12-04 | 2019-04-12 | 北京京东尚科信息技术有限公司 | Backup, recovery and the mirror processing method and system of container environment |
| CN110493175A (en) * | 2019-07-01 | 2019-11-22 | 联想(北京)有限公司 | A kind of information processing method, electronic equipment and storage medium |
| US20200004891A1 (en) * | 2018-06-28 | 2020-01-02 | Business Objects Software Ltd. | Prediction integration for data management platforms |
| CN111400374A (en) * | 2020-03-18 | 2020-07-10 | 中国雄安集团数字城市科技有限公司 | Containerized data exploration isolation region oriented to data mining and using method thereof |
| CN112379869A (en) * | 2020-11-13 | 2021-02-19 | 广东电科院能源技术有限责任公司 | Standardized development training platform |
-
2021
- 2021-04-15 CN CN202110407172.0A patent/CN112988165A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200004891A1 (en) * | 2018-06-28 | 2020-01-02 | Business Objects Software Ltd. | Prediction integration for data management platforms |
| CN109614202A (en) * | 2018-12-04 | 2019-04-12 | 北京京东尚科信息技术有限公司 | Backup, recovery and the mirror processing method and system of container environment |
| CN110493175A (en) * | 2019-07-01 | 2019-11-22 | 联想(北京)有限公司 | A kind of information processing method, electronic equipment and storage medium |
| CN111400374A (en) * | 2020-03-18 | 2020-07-10 | 中国雄安集团数字城市科技有限公司 | Containerized data exploration isolation region oriented to data mining and using method thereof |
| CN112379869A (en) * | 2020-11-13 | 2021-02-19 | 广东电科院能源技术有限责任公司 | Standardized development training platform |
Non-Patent Citations (9)
| Title |
|---|
| DARKNESS303: "jupyter notebook 写代码自动补全", 《HTTPS://BLOG.CSDN.NET/WEIXIN_37595559/ARTICLE/DETAILS/81540482》 * |
| HE11OW0RLD: "vs IDE界面改造", 《HTTPS://WENKU.BAIDU.COM/VIEW/EE85C0E2A0116C175F0E48CB.HTML》 * |
| JACEK RAPIŃSKI等: "JupyTEP IDE as an Online Tool for Earth Observation Data Processing", 《REMOTE SENSING》 * |
| PARIWAT PRATHANRAT: "Performance Prediction of Jupyter Notebook in JupyterHub using Machine Learning", 《2018 INTERNATIONAL CONFERENCE ON INTELLIGENT INFORMATICS AND BIOMEDICAL SCIENCES (ICIIBMS)》 * |
| SALMAN IQBAL: "Scaling Jupyter notebooks with Kubernetes and Tensorflow", 《HTTPS://LEARNK8S.IO/BLOG/SCALING-MACHINE-LEARNING-WITH-KUBEFLOW-TENSORFLOW》 * |
| WEIXIN_39564807: "新建文件夹默认只读_Littlest JupyterHub| 05通过文件夹共享或收集文件", 《HTTPS://BLOG.CSDN.NET/WEIXIN_39564807/ARTICLE/DETAILS/111582299》 * |
| 康计良: "Python语言的可视化编程环境的设计与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 搜云库: "Kubeflow-K8S的机器学习工具包，太牛了！", 《HTTPS://TECH.SOUYUNKU.COM/?P=28579》 * |
| 王艺颖: "基于Kubernetes的Tensorflow分布式模型训练平台的设计与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115048087A (en) * | 2022-08-15 | 2022-09-13 | 江苏博云科技股份有限公司 | Method, equipment and storage medium for realizing online IDE tool in Kubernetes environment |
| CN116627446A (en) * | 2023-07-25 | 2023-08-22 | 华控清交信息科技(北京)有限公司 | Method and device for deploying application on k8s container cluster and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110825362B (en) | Low-code application software development system and method | |
| US10592319B2 (en) | API notebook tool | |
| US10162611B2 (en) | Method and apparatus for business rule extraction | |
| CN110928529B (en) | Method and system for assisting operator development | |
| US10831453B2 (en) | Connectors framework | |
| US20140173454A1 (en) | Method and system for designing, deploying and executing transactional multi-platform mobile applications | |
| US20150293764A1 (en) | Method and system to compose and execute business rules | |
| CN110941467A (en) | Data processing method, device and system | |
| US11797273B2 (en) | System and method for enhancing component based development models with auto-wiring | |
| CN101373431A (en) | Enhanced widget composition platform | |
| JP7280388B2 (en) | Apparatus and method, equipment and medium for implementing a customized artificial intelligence production line | |
| CN112988165A (en) | Kubernetes-based interactive modeling method and device, electronic equipment and storage medium | |
| Benouda et al. | Automatic code generation within MDA approach for cross-platform mobiles apps | |
| Snell et al. | Microsoft Visual Studio 2012 Unleashed: Micro Visua Studi 2012 Unl_p2 | |
| Uluca | Angular for Enterprise-Ready Web Applications: Build and deliver production-grade and cloud-scale evergreen web apps with Angular 9 and beyond | |
| Sasidharan | Full stack development with JHipster: build full stack applications and microservices with Spring Boot and modern JavaScript frameworks | |
| CN111221610B (en) | Page element acquisition method and device | |
| US11132374B2 (en) | Property painter | |
| Amanquah et al. | Rapid application development for mobile terminals | |
| Gammelgaard | Microservices in. NET | |
| EP4354281A1 (en) | Converting an api into a graph api | |
| Romão | Migration from Legacy to Reactive Applications in OutSystems | |
| Rashidi et al. | Software development and programming tools-a survey and recommendation for organization of small industries and industrial towns of Iran | |
| Patel | Spring 5.0 Projects: Build Seven Web Development Projects with Spring MVC, Angular 6, JHipster, WebFlux, and Spring Boot 2 | |
| Patel | Sitecore Cookbook for Developers |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210618 |