CN114995834A - Artificial intelligence application deployment environment construction method and device - Google Patents

Artificial intelligence application deployment environment construction method and device Download PDF

Info

Publication number
CN114995834A
CN114995834A CN202210476521.9A CN202210476521A CN114995834A CN 114995834 A CN114995834 A CN 114995834A CN 202210476521 A CN202210476521 A CN 202210476521A CN 114995834 A CN114995834 A CN 114995834A
Authority
CN
China
Prior art keywords
stage
configuration
deployment
script
packaging
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210476521.9A
Other languages
Chinese (zh)
Inventor
詹智财
王强
彭燕卿
王大闹
任秀明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
SuningCom Co ltd
Original Assignee
SuningCom Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by SuningCom Co ltd filed Critical SuningCom Co ltd
Priority to CN202210476521.9A priority Critical patent/CN114995834A/en
Publication of CN114995834A publication Critical patent/CN114995834A/en
Priority to CA3198338A priority patent/CA3198338A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/60—Software deployment
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/70—Software maintenance or management
    • G06F8/71—Version control; Configuration management

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Security & Cryptography (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Stored Programmes (AREA)

Abstract

The application relates to a method and a device for constructing an artificial intelligence application deployment environment, comprising the following steps: providing a user code warehouse required by an artificial intelligent application project in a development network segment, and acquiring project information created by a user and input environment dependence configuration information; generating an entry script for each stage, and acquiring a configuration script of each stage according to the environment-dependent configuration information; the packaging subsystem executes the configuration scripts of the corresponding packaging stage based on the entry scripts of the packaging stage, packages the configuration scripts of each stage corresponding to the project information into a compression packet and sends the compression packet to the deployment subsystem; the deployment subsystem receives the compression package and executes the configuration script of the deployment stage based on the entry script of the deployment stage; the deployment subsystem executes the configuration script of the initialization stage based on the entry script of the initialization stage and provides the application service corresponding to the project information. The method and the device can accelerate the deployment efficiency of the artificial intelligence application, and realize that the artificial intelligence application deployment environment is flexibly and conveniently constructed.

Description

Artificial intelligence application deployment environment construction method and device
Technical Field
The application relates to the technical field of artificial intelligence, in particular to a method and a device for constructing an artificial intelligence application deployment environment.
Background
With the development of artificial intelligence technology, more and more enterprises find that it is important to construct a stable and flexibly configured publishing system that does not require an algorithm engineer to pay much attention to the dependence of the underlying environment. Because the current algorithm engineers do not know the knowledge of the bottom layer of the computer and many algorithm engineers do not have the knowledge of the computer science, the deployment of artificial intelligence application becomes more difficult, and the excessive environmental dependence paying attention to the bottom layer causes the work of the algorithm engineers to become low in value.
The existing deployment system completes artificial intelligent application deployment and release through a kvm (Kernel-based Virtual Machine), or a docker (an open-source application container engine), and an enterprise isolates a network into a development network segment, an integration network segment, a pressure measurement network segment and a production network segment for the stability and the security of the whole application release. Meanwhile, the enterprise has the mixed use condition of kvm and docker. In order to ensure the performability of all applications, a container operation and maintenance team and a mirror image manufacturing team can continuously receive the requirements of upper-layer users, continuously complete the manufacturing of different kvm mirror images and docker mirror images, and realize the manufacturing and pushing of bottom-layer mirror images in different scenes. However, the above method has serious drawbacks: environmental consistency brought by network isolation becomes particularly difficult, and once a problem occurs, an algorithm engineer usually initiates the fault clearing and the fault clearing are performed by combining a container team, a mirror image team and an operation and maintenance team, so that the fault clearing efficiency is low; an algorithm engineer does not understand the knowledge of operation and maintenance of a bottom-layer computer, and the operation and maintenance of the bottom layer cannot follow the development of various frames and environment changes caused by the development of an algorithm, so that the operation and maintenance cost and the communication cost are too high and are not flexible enough, the development of business requirements cannot be followed, and different scenes can be flexibly coped with; cannot be simultaneously compatible with a kvm environment and a docker environment based on one set of dependencies.
Disclosure of Invention
Based on the above, the application provides a method and a device for constructing an artificial intelligence application deployment environment, so as to solve the problems in the prior art.
In a first aspect, a method for constructing an artificial intelligence application deployment environment is provided, and the method includes:
providing a user code warehouse required by an artificial intelligence application project in a development network segment, and acquiring project information created by a user based on the user code warehouse and input environment dependence configuration information;
generating an entry script for each stage according to the project information, and acquiring a configuration script of each stage according to the environment-dependent configuration information, wherein each stage comprises a packaging stage, a deployment stage and an initialization stage;
the packaging subsystem executes the configuration scripts of the corresponding packaging stages based on the entry scripts of the packaging stages corresponding to the project information, packages the configuration scripts of the stages corresponding to the project information into compressed packages and sends the compressed packages to the deployment subsystem;
the deployment subsystem receives the compressed package and executes a configuration script of a deployment stage based on an entry script of the deployment stage corresponding to the project information;
and the deployment subsystem executes the configuration script of the initialization stage based on the entry script of the initialization stage corresponding to the project information and provides the application service corresponding to the project information.
According to an implementable manner in an embodiment of the present application, the method further comprises: providing a multi-segment environment, the multi-segment environment comprising: developing a network segment, an integrated network segment, a pressure measurement network segment and a production network segment;
the packaging subsystem comprises the development network segment, and the deployment subsystem comprises the integration network segment, the pressure measurement network segment and the production network segment; the development network segment, the integration network segment, the pressure measurement network segment and the production network segment cannot be communicated with each other.
According to an implementation mode in the embodiment of the application, the configuration scripts of each stage are stored in a cloud configuration server; and the packaging subsystem acquires the configuration scripts of each stage from the cloud configuration server.
According to an implementation manner in the embodiment of the present application, the generating an entry script for each phase according to the project information includes: and creating codes required by the project information based on the user code warehouse, and synchronously creating entry scripts of each stage corresponding to the project information, wherein the entry scripts of each stage respectively read scripts under a unified directory.
According to an implementation manner in the embodiment of the present application, the performing, by the packaging subsystem, the configuration script of the corresponding packaging stage based on the entry script of the packaging stage corresponding to the item information, and packaging the configuration script of each stage corresponding to the item information into a compressed packet, where the compressed packet is sent to the deployment subsystem includes:
the packaging subsystem pulls a configuration script of a packaging stage corresponding to the project information from the cloud configuration server based on an entry script of the packaging stage corresponding to the project information and executes the configuration script, the configuration script of the packaging stage pulls a configuration script of a corresponding deployment stage and a configuration script of an initialization stage to be stored in a path corresponding to the project information, and an internal variable of the configuration script of the deployment stage is modified;
and packaging the configuration scripts of each stage corresponding to the project information into a compressed packet, and pushing the compressed packet to the integration network segment, the pressure measurement network segment and the production network segment through a preset audit path.
According to an implementation manner in the embodiment of the present application, the receiving, by the deployment subsystem, the compressed packet, and executing the configuration script of the deployment phase based on the entry script of the deployment phase corresponding to the project information includes:
and the deployment subsystem receives and decompresses the compressed packet, executes the configuration script of the deployment stage by reading the entry script of the deployment stage corresponding to the project information, and modifies the internal variable of the configuration script of the corresponding initialization stage.
According to an implementation manner in the embodiment of the present application, the executing, by the deployment subsystem, the configuration script of the initialization stage based on the entry script of the initialization stage corresponding to the project information, and providing the application service corresponding to the project information includes:
and the deployment subsystem executes the configuration script of the initialization stage by reading the entry script of the initialization stage corresponding to the project information, completes the initialization of the configuration, and pulls up the artificial intelligence application service corresponding to the project information.
In a second aspect, an artificial intelligence application deployment environment construction apparatus is provided, the apparatus comprising:
an acquisition unit: the system comprises a user code warehouse, a development network segment and a database, wherein the user code warehouse is used for providing a user code warehouse required by an artificial intelligent application project in the development network segment, and acquiring project information created by a user based on the user code warehouse and input environment dependence configuration information;
a configuration unit: generating an entry script for each stage according to the project information, and acquiring a configuration script of each stage according to the environment-dependent configuration information, wherein each stage comprises a packaging stage, a deployment stage and an initialization stage;
the packaging unit is arranged in the packaging subsystem and used for executing the configuration scripts of the corresponding packaging stages based on the entry scripts of the packaging stages corresponding to the project information, packaging the configuration scripts of the stages corresponding to the project information into compression packets and sending the compression packets to the deployment subsystem;
the deployment unit is arranged in the deployment subsystem and used for receiving the compressed package and executing the configuration script of the deployment stage based on the entry script of the deployment stage corresponding to the project information;
the initialization unit is arranged in the deployment subsystem and used for executing the configuration script of the initialization stage based on the entry script of the initialization stage corresponding to the project information and providing the application service corresponding to the project information.
In a third aspect, a computer device is provided, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores computer instructions executable by the at least one processor to enable the at least one processor to perform the method referred to in the first aspect above.
In a fourth aspect, a computer-readable storage medium is provided, on which computer instructions are stored, wherein the computer instructions are configured to cause a computer to perform the method of the first aspect.
According to the technical content provided by the embodiment of the application, the technical scheme provided by the application can have the following advantages: according to the method, all bottom layer dependencies are decomposed into a packaging stage, a deployment stage and an initialization stage, so that the problem that different basic mirror images need to be manufactured is avoided, and different scenes can be flexibly dealt with; meanwhile, the configuration scripts of all stages corresponding to the project are packaged into the compression packets through the packaging subsystem and are sent to the deployment subsystem, all changes are concentrated in the packaging stage, the condition that multi-department joint debugging is needed in the artificial intelligence application deployment process is improved, the debugging range is greatly reduced, the debugging efficiency is improved, the deployment efficiency of artificial intelligence application is accelerated, and the artificial intelligence application deployment environment is flexibly and conveniently constructed.
Drawings
FIG. 1 is a schematic flow chart illustrating a method for constructing an artificial intelligence application deployment environment in one embodiment;
FIG. 2 is a block diagram of an apparatus for constructing an artificial intelligence application deployment environment in one embodiment;
FIG. 3 is a schematic block diagram of a computer device in one embodiment.
Detailed Description
The present application will be described in further detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
Fig. 1 is a flowchart of a method for constructing an artificial intelligence application deployment environment according to an embodiment of the present disclosure, and as shown in fig. 1, the method may include the following steps:
step 101: and providing a user code warehouse required by the artificial intelligence application project in the development network segment, and acquiring project information created by the user based on the user code warehouse and input environment dependence configuration information.
Specifically, an artificial intelligence application is completely packaged, deployed and issued into a service, and a packaging stage and a deployment stage are required to be involved, wherein the deployment stage further comprises an initialization stage, the packaging stage is completed in a packaging subsystem, the deployment stage is completed in a deployment subsystem, the packaging subsystem is in a development network segment, and the deployment subsystem comprises an integration network segment, a pressure measurement network segment and a production network segment. Private source and user code warehouses gitlab such as yum, pip or conda required by the artificial intelligence application project are provided in the development network segment, and a web page is provided in the development network segment, so that a user can create project information based on the user code warehouses gitlab and select environment dependency configuration required by the current artificial intelligence application project. The environment dependent configuration refers to the low level dependent environment required by artificial intelligence application process items such as gcc, glibc, cuda, tensorflow, etc., such as the operating system, software dependence, etc. required by artificial intelligence application items.
Step 102: and generating an entry script aiming at each stage according to the project information, and acquiring the configuration script of each stage according to the environment-dependent configuration information, wherein each stage comprises a packaging stage, a deployment stage and an initialization stage.
Specifically, the user creates the artificial intelligence application project information based on a user code warehouse, wherein the user code warehouse generates a uniform entry script for each stage at each project initialization, and each stage comprises a packaging stage, a deployment stage and an initialization stage, so that the entry script of the packaging stage, the entry script of the deployment stage and the entry script of the initialization stage are generated.
Meanwhile, the development network segment web page of each application project receives environment-dependent configuration input by a user, and obtains configuration scripts of each stage, including a configuration script of a packaging stage, a configuration script of a deployment stage and a configuration script of an initialization stage. The configuration script of each phase refers to operations that need to be executed in the packaging phase, the deployment phase and the initialization phase, and can be provided in the form of a bash script, for example.
For simplicity of deployment and maintenance, different scripts are divided at different stages, a package script is used for code downloading of a package stage, environment-dependent package downloading, and downloading of a deployment script and an initialization script before final service pull-up, which are required to be executed in corresponding subsequent stages.
Step 103: the packaging subsystem executes the configuration scripts of the corresponding packaging stages based on the entry scripts of the packaging stages corresponding to the project information, packages the configuration scripts of the stages corresponding to the project information into a compressed package and sends the compressed package to the deployment subsystem.
Specifically, in the packing stage, the packing entry script in the current project in the packing subsystem pulls and executes the configuration scripts of all packing stages corresponding to the current project, packs the configuration scripts of all stages corresponding to the whole project into a compressed package, and pushes the compressed package into a basic kvm virtual machine or a docker container such as an integrated network segment, a pressure measurement network segment, a production network segment and the like through a specific audit path.
Step 104: and the deployment subsystem receives the compressed package and executes the configuration script of the deployment stage based on the entry script of the deployment stage corresponding to the project information.
Specifically, in the deployment phase, the deployment subsystem receives and decompresses the project compression package, and executes the deployment entry script in the current project, which executes the configuration deployment scripts stored before, to complete the deployment of the dependent package.
Step 105: the deployment subsystem executes the configuration script of the initialization stage based on the entry script of the initialization stage corresponding to the project information and provides the application service corresponding to the project information.
Specifically, in the initialization stage, i.e., before the application is pulled up, the deployment subsystem executes the environment initialization entry scripts in the current project, which execute the configuration initialization scripts stored before, and finally pulls up the artificial intelligence application process to expose the service port to the outside to provide the final service.
Therefore, the embodiment of the application solves the problem that different basic images need to be manufactured by decomposing all bottom layer dependencies into a packaging stage, a deployment stage and an initialization stage, and can flexibly cope with different scenes; meanwhile, the configuration scripts of all stages corresponding to the project are packaged into the compression packets through the packaging subsystem and are sent to the deployment subsystem, all changes are concentrated in the packaging stage, the condition that multi-department joint debugging is needed in the artificial intelligence application deployment process is improved, the debugging range is greatly reduced, the debugging efficiency is improved, the deployment efficiency of artificial intelligence application is accelerated, and the artificial intelligence application deployment environment is flexibly and conveniently constructed.
The steps in the above-described process flow are described in detail below with reference to examples.
In one embodiment of the present application, the method further comprises: providing a multi-network environment, wherein the multi-network environment comprises: developing a network segment, an integration network segment, a pressure measurement network segment and a production network segment; the packaging subsystem comprises a development network segment, and the deployment subsystem comprises an integration network segment, a pressure measurement network segment and a production network segment; the development network segment, the integration network segment, the pressure measurement network segment and the production network segment cannot be communicated with each other.
Specifically, the artificial intelligence application deployment environment construction system is divided into a packaging subsystem, a deployment subsystem, a corresponding packaging download script, a deployment installation script and an environment initialization script before application operation. The packaging subsystem is a plurality of systems for packaging in a development environment network, and the deployment subsystem is a plurality of systems for system integration, pressure testing and real online service in an integration network segment, a pressure testing network segment and a production network segment. Because network isolation tends to be strict, different roles borne by different network segments and the stability of the whole cluster system tend to be strict, the packaging subsystem is divided for packaging, the deployment subsystem is used for integrating pressure measurement and real online system management, and all network segments cannot be communicated with each other. The development network segment, the integration network segment, the pressure measurement network segment and the production network segment cannot be communicated with each other and can only transmit files through a specific channel, so that the stability and the safety of the whole artificial intelligence application release can be improved.
In an embodiment of the present application, the configuration scripts of each stage in step 102 are stored in a cloud configuration server; and the packaging subsystem acquires the configuration scripts of each stage from the cloud configuration server.
Specifically, the cloud configuration server may be divided into a global configuration folder and a configuration folder corresponding to each project, where the global configuration folder stores configuration scripts of all configured stages, including configuration scripts of a packaging stage, configuration scripts of a deployment stage, and configuration scripts of an initialization stage. When a user creates artificial intelligence application projects based on a gitlab user code warehouse, a web page of a development network segment of each application project receives environment dependence configuration of the user, and copies a corresponding configuration packaging script from a global configuration folder of a cloud configuration server into a configuration folder to which the current project belongs.
According to the embodiment of the application, all configuration scripts which depend on all stages are uniformly stored on the cloud configuration server, the subsequent newly added bottom layer dependence is facilitated, and the problem that an algorithm engineer needs to continuously install the configuration environment is avoided by receiving the environment dependence configuration of a user on the web page of the development network segment.
In an embodiment of the present application, the step 102 of generating an entry script for each stage according to the project information includes: and creating codes required by the project information based on the user code warehouse, synchronously creating entry scripts of all stages corresponding to the project information, and reading the scripts under the unified directory respectively by the entry scripts of all stages.
Specifically, the user creates artificial intelligence application project information and required codes based on the user code repository, and then creates corresponding entry scripts of each stage synchronously, that is, a package entry script, a deployment entry script, and an initialization entry script. By providing uniform and identical phase entry scripts, and each entry script follows the script under the read-out contract directory and is executed, the maintenance of the packaging subsystem and the deployment subsystem becomes very simple, and no specific environment preparation needs to be made for a specific project.
The embodiment of the application achieves the purpose that the bottom layer dependence can be dynamically added and deleted between different release versions of different projects or the same project by providing the unified same entry script.
In an embodiment of the present application, the step 103 of executing, by the packaging subsystem, the configuration script of the corresponding packaging stage based on the entry script of the packaging stage corresponding to the item information, and packaging the configuration script of each stage corresponding to the item information into a compressed package, and sending the compressed package to the deployment subsystem includes: the packaging subsystem pulls the configuration script of the packaging stage corresponding to the project information from the cloud configuration server based on the entry script of the packaging stage corresponding to the project information and executes the configuration script, the configuration script of the packaging stage pulls the configuration script of the corresponding deployment stage and the configuration script of the initialization stage to be stored in a path corresponding to the project information, and the internal variables of the configuration script of the deployment stage are modified; and packaging the configuration scripts of each stage corresponding to the project information into a compressed packet, and pushing the compressed packet to the integration network segment, the pressure measurement network segment and the production network segment through a preset audit path.
Specifically, in the packing stage, all configuration packing scripts in a current project attribution folder in the packing subsystem are pulled from the cloud configuration server and executed, the whole project is packed into a zip compressed package, and the zip compressed package is pushed into a basic kvm virtual machine or a docker container through a specific audit path, such as an integrated network segment, a pressure measurement network segment, a production network segment and the like. Each configuration packaging script pulls a respective configuration deployment script, the configuration initialization script is stored in a path corresponding to a project, variables in the corresponding configuration deployment script are dynamically modified, for example, the corresponding project name, the IP and the like are modified, and meanwhile, a required prepositive configuration dependency package is downloaded based on pip, conda and yum.
In an embodiment of the application, the receiving, by the deployment subsystem in step 104, the compressed packet, and executing, based on the entry script of the deployment phase corresponding to the project information, the configuration script of the deployment phase includes: and the deployment subsystem receives and decompresses the compressed packet, executes the configuration script of the deployment stage by reading the entry script of the deployment stage corresponding to the project information, and modifies the internal variable of the configuration script of the corresponding initialization stage.
Specifically, in the deployment phase, the deployment subsystem receives and decompresses the project compression package, and executes the deployment entry script in the current project, which executes the configuration deployment scripts stored before, to complete the deployment of the dependent package. Each configuration deployment script will complete the decompression of the respective configuration dependency package and dynamically modify the dependent environment variables in the corresponding configuration initialization script, such as the gcc rpm package is decompressed through rpm2 cpio.
In an embodiment of the application, the executing, by the deployment subsystem in step 105, the configuration script in the initialization stage based on the entry script in the initialization stage corresponding to the project information, and the providing the application service corresponding to the project information includes: and the deployment subsystem executes the configuration script of the initialization stage by reading the entry script of the initialization stage corresponding to the project information, completes the initialization of the configuration, and pulls up the artificial intelligence application service corresponding to the project information.
Specifically, in the initialization phase, that is, before the application is pulled up, the deployment subsystem executes the environment initialization entry script in the current project, sequentially executes the configuration initialization scripts stored before, and finally pulls up the artificial intelligence application process to expose the service port to the outside to provide the final service. Each configuration initialization script is initialization work which needs to be completed before the final service is pulled up, such as modification of environment variables like LD _ LIBRARY _ PATH, and the like.
It should be understood that, although the steps in the flowchart of fig. 1 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in a strict order unless explicitly stated in the application, and may be performed in other orders. Moreover, at least a portion of the steps in fig. 1 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
With reference to the implementation manner in the foregoing embodiment, a preferred method flow provided by the embodiment of the present application is described below by way of example, and the method may include the following steps:
step 1: a multi-segment environment is provided. An artificial intelligence application is completely packaged, deployed and issued into a service, and a packaging stage and a deployment stage are required, wherein the packaging stage is completed in a packaging subsystem, the deployment stage is completed in a deployment subsystem, the packaging subsystem is in a development network segment, and the deployment subsystem comprises an integration network segment, a pressure measurement network segment and a production network segment.
Step 2: the user selects the dependent configuration required for the current project based on the web. The method specifically comprises the following steps: after a project is newly built, a user jumps to a dependent configuration selection web page, all current selectable configurations are presented on the dependent configuration selection web page, the user only needs to perform check, and the check includes configurations such as cuda, gcc, glibc, zbar, opencv, cafe and the like, and requirement is provided for the user to write a python dependent package.
And step 3: the git repository for the project is initialized and the user uploads the application code for the current project. The method specifically comprises the following steps: and if a user creates a git warehouse, the corresponding packing entry scripts can be synchronously created, the entry scripts are deployed, and the entry scripts are initialized. The Git warehouse is initialized to generate a directory structure as shown in table 1, and the three-phase entry script content is shown in table 2. By providing uniform, phase-specific entry scripts, and following the reading of scripts from a defined directory and executing them, maintenance of the packaging subsystem and deployment subsystem becomes very simple, without the need for specific environmental preparation for a particular project.
Directory Function(s)
bin Built-in three scripts are divided into package.sh, setup.sh and start.sh
bin/packages Packaged download script for storing all configurations
bin/setups Deployment installation script for storing all configurations
bin/starts Environment initialization script for storing all configurations
conf For storing required configuration files
lib For storing environment dependent packages
src The method is used for uploading application codes, wherein basic codes of flash are built in the application codes, and a user needs to finish code integration of the flash based on template codes
TABLE 1
Figure BDA0003625795110000101
TABLE 2
And 4, step 4: and the packaging subsystem calls a packaging entry script in the git warehouse and packages the whole project into a zip compressed package after the execution is finished. The method specifically comprises the following steps: each configuration packaging script will pull its own configuration deployment script, the configuration initialization script will be stored in the corresponding path of the project, and dynamically modify the variables in the corresponding configuration deployment script, and download the required configuration dependency package based on pip, conda, yum, if zbar, miniconda dependency is selected by the current user, and if zbar is downloaded in yum form, the packaging script content is as shown in table 3.
Figure BDA0003625795110000102
TABLE 3
And 5: and pushing the zip package to each network segment in the deployment subsystem, decompressing and executing the deployment entry script. And instantiating the kvm mirror image or the docker mirror image, decompressing the zip packet and executing the deployment entry script, wherein each configuration deployment script completes decompression of the respective configuration dependent packet and dynamically modifies the dependent environment variables in the corresponding configuration initialization script, and the deployment script content is shown in table 4 if the zbar dependent packet is downloaded as described above.
Deploying installation scripts Content schematic
bin/setups/zbar_setup.sh cdzbar&&foriin`ls`；dorpm2cpio$i|cpio-idv；done
Bin/setups/miniconda_setup.sh shconda_local.sh-b-p/opt/files/miniconda3
TABLE 4
Step 6: and the deployment subsystem executes the initialized entry script, and pulls up the built-in artificial intelligence application process to provide a corresponding service interface after the execution is finished.
Each configuration initialization script is the initialization work that needs to be completed before the final service is pulled up, such as the modification of environment variables like LD _ LIBRARY _ PATH, etc., the deployment subsystem completes the initialization of all configurations on which its project depends by reading the initialization entry script built in the project, the content of the initialization script is shown in table 5, and pulls up the corresponding artificial intelligence service processes, such as zbar and miniconda, mentioned above.
Figure BDA0003625795110000111
TABLE 5
According to the embodiment of the application, all bottom layer dependencies are decomposed into the packaging downloading stage, the deploying and installing stage and the initializing stage, the problem that different basic images need to be manufactured is avoided, and the purpose that different projects or the same project can be dynamically added and deleted among different release versions is achieved through unifying the same entry script. And all changes are concentrated in the packing stage, so that the cost of multi-department joint debugging is greatly improved, the debugging range is greatly reduced, the participation of operation and maintenance departments is reduced, and the deployment efficiency of artificial intelligent application is accelerated. Meanwhile, all the dependencies are uniformly stored on the cloud configuration server in a configuration mode, subsequent newly-added bottom layer dependencies are facilitated, and the problem that an algorithm engineer needs to continuously install a configuration environment is solved through selection of a web user.
The method embodiment can be applied to various application scenarios, for example, the method embodiment can be applied to a linux operating system or other operating systems.
Fig. 2 is a schematic structural diagram of an artificial intelligence application deployment environment construction apparatus according to an embodiment of the present application, configured to execute the method flow shown in fig. 1. As shown in fig. 2, the apparatus may include: the obtaining unit 201, the configuring unit 202, the packaging unit 203, the deploying unit 204, and the initializing unit 205 may further include: a segmentation unit 206. The main functions of each component module are as follows:
the acquisition unit 201: the system comprises a user code warehouse, a development network segment and a database, wherein the user code warehouse is used for providing a user code warehouse required by an artificial intelligent application project in the development network segment, and acquiring project information created by a user based on the user code warehouse and input environment-dependent configuration information;
the configuration unit 202: generating an entry script aiming at each stage according to the project information, and acquiring a configuration script of each stage according to the environment-dependent configuration information, wherein each stage comprises a packaging stage, a deployment stage and an initialization stage;
the packing unit 203 is arranged in the packing subsystem, and is configured to execute the configuration scripts of the corresponding packing stage based on the entry scripts of the packing stage corresponding to the item information, and pack the configuration scripts of each stage corresponding to the item information into a compressed packet, which is sent to the deployment subsystem;
the deployment unit 204 is disposed in the deployment subsystem, and is configured to receive the compressed packet, and execute a configuration script of the deployment phase based on an entry script of the deployment phase corresponding to the project information;
the initialization unit 205 is disposed in the deployment subsystem, and is configured to execute the configuration script in the initialization stage based on the entry script in the initialization stage corresponding to the project information, and provide the application service corresponding to the project information.
In one embodiment of the present application, the apparatus further comprises a segmentation unit 206: the method is used for providing a multi-network segment environment, wherein the multi-network segment environment comprises the following steps: developing a network segment, an integration network segment, a pressure measurement network segment and a production network segment; the packaging subsystem comprises a development network segment, and the deployment subsystem comprises an integration network segment, a pressure measurement network segment and a production network segment; the development network segment, the integration network segment, the pressure measurement network segment and the production network segment cannot be communicated with each other.
In an embodiment of the present application, the configuration scripts of each stage in the configuration unit 202 are stored on the cloud configuration server.
In an embodiment of the present application, the configuration unit 202 generates an entry script for each phase according to the item information, including: and creating codes required by the project information based on the user code warehouse, synchronously creating entry scripts of all stages corresponding to the project information, and reading the scripts under the unified directory by the entry scripts of all stages respectively.
In an embodiment of the present application, the step of executing, by the packaging subsystem in the packaging unit 203, the configuration script of the corresponding packaging stage based on the entry script of the packaging stage corresponding to the project information, packaging the configuration script of each stage corresponding to the project information into a compressed packet, and sending the compressed packet to the deployment subsystem includes: the packaging subsystem pulls the configuration script of the packaging stage corresponding to the project information from the cloud configuration server based on the entry script of the packaging stage corresponding to the project information and executes the configuration script, the configuration script of the packaging stage pulls the configuration script of the corresponding deployment stage and the configuration script of the initialization stage to be stored in a path corresponding to the project information, and the variable in the configuration script of the deployment stage is modified; and packaging the configuration scripts of each stage corresponding to the project information into a compressed packet, and pushing the compressed packet to the integration network segment, the pressure measurement network segment and the production network segment through a preset audit path.
In an embodiment of the present application, the receiving, by the deployment subsystem in the deployment unit 204, the compressed package, and executing the configuration script of the deployment phase based on the entry script of the deployment phase corresponding to the project information includes: and the deployment subsystem receives and decompresses the compressed packet, executes the configuration script of the deployment stage by reading the entry script of the deployment stage corresponding to the project information, and modifies the internal variable of the configuration script of the corresponding initialization stage.
In an embodiment of the application, the executing, by the deployment subsystem in the initialization unit 205, the configuration script of the initialization stage based on the entry script of the initialization stage corresponding to the project information, and providing the application service corresponding to the project information includes: and the deployment subsystem executes the configuration script of the initialization stage by reading the entry script of the initialization stage corresponding to the project information, completes the initialization of the configuration and pulls up the artificial intelligence application service corresponding to the project information.
According to the specific embodiment provided by the application, the technical scheme provided by the application can have the following advantages:
1) by decomposing all the bottom layer dependencies into a packaging downloading phase, a deployment installation phase and an initialization phase, the problem that different base images need to be manufactured is avoided.
2) By unifying the same entry script, the purpose that different projects or the same project can dynamically add and delete bottom layer dependence between different release versions is realized.
3) By concentrating all changes in the packing stage, the cost of multi-department joint debugging is greatly improved, the debugging range is greatly reduced, the participation of operation and maintenance departments is reduced, and the deployment efficiency of artificial intelligent application is accelerated.
4) All the dependencies are uniformly stored on the cloud configuration server in a configuration mode, subsequent newly-added bottom layer dependencies are facilitated, and the problem that an algorithm engineer needs to continuously install a configuration environment is solved through selection of a web user.
It is to be understood that not necessarily all such advantages may be achieved in the practice of any one method or product of the present application.
The same and similar parts among the various embodiments described above can be referred to each other, and each embodiment is described with emphasis on differences from other embodiments. In particular, as for the apparatus embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
It should be noted that, in the embodiments of the present application, the use of user data may be involved, and in practical applications, the user-specific personal data may be used in the scheme described herein within the scope permitted by applicable laws and regulations, under the condition of meeting the requirements of applicable laws and regulations in the country (for example, the user explicitly agrees, the user is informed, the user explicitly authorizes, etc.).
According to an embodiment of the present application, a computer device and a computer-readable storage medium are also provided.
As shown in fig. 3, a block diagram of a computer device according to an embodiment of the present application is shown. Computer apparatus is intended to represent various forms of digital computers or mobile devices. Which may include desktop computers, laptop computers, workstations, personal digital assistants, servers, mainframe computers, and other suitable computers. The mobile device may include a tablet, smartphone, wearable device, and the like.
As shown in fig. 3, the apparatus 300 includes a computing unit 301, a ROM 302, a RAM 303, a bus 304, and an input/output (I/O) interface 305, and the computing unit 301, the ROM 302, and the RAM 303 are connected to each other via the bus 304. An input/output (I/O) interface 305 is also connected to bus 304.
The calculation unit 301 may perform various processes in the method embodiments of the present application according to computer instructions stored in a Read Only Memory (ROM)302 or computer instructions loaded from a storage unit 308 into a Random Access Memory (RAM) 303. Computing unit 301 may be a variety of general and/or special purpose processing components with processing and computing capabilities. The computing unit 301 may include, but is not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. In some embodiments, the methods provided by the embodiments of the present application may be implemented as a computer software program tangibly embodied in a computer-readable storage medium, such as storage unit 308.
The RAM 303 may also store various programs and data required for the operation of the device 300. Part or all of the computer program may be loaded and/or installed onto the device 300 via the ROM 802 and/or the communication unit 309.
The input unit 303, the output unit 307, the storage unit 308, and the communication unit 309 in the device 300 may be connected to the I/O interface 305. Among them, the input unit 303 may be, for example, a keyboard, a mouse, a touch screen, a microphone, or the like; the output unit 307 may be, for example, a display, a speaker, an indicator lamp, or the like. The device 300 is capable of exchanging information, data, etc. with other devices through the communication unit 309.
It should be noted that the device may also include other components necessary to achieve proper operation. It may also contain only the components necessary to implement the solution of the present application and not necessarily all of the components shown in the figures.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof.
Computer instructions for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer instructions may be provided to the computing unit 301 such that the computer instructions, when executed by the computing unit 301, such as a processor, cause the steps involved in embodiments of the method of the present application to be performed.
The computer-readable storage medium provided herein may be a tangible medium that may contain, or store, computer instructions for performing the steps involved in the method embodiments of the present application. The computer readable storage medium may include, but is not limited to, storage media in the form of electronic, magnetic, optical, electromagnetic, and the like.
The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims (10)

1. A construction method for an artificial intelligence application deployment environment is characterized by comprising the following steps:
providing a user code warehouse required by an artificial intelligence application project in a development network segment, and acquiring project information created by a user based on the user code warehouse and input environment dependence configuration information;
generating an entry script for each stage according to the project information, and acquiring a configuration script of each stage according to the environment-dependent configuration information, wherein each stage comprises a packaging stage, a deployment stage and an initialization stage;
the packaging subsystem executes the configuration scripts of the corresponding packaging stages based on the entry scripts of the packaging stages corresponding to the project information, packages the configuration scripts of the stages corresponding to the project information into compressed packages and sends the compressed packages to the deployment subsystem;
the deployment subsystem receives the compressed package and executes a configuration script of a deployment stage based on an entry script of the deployment stage corresponding to the project information;
and the deployment subsystem executes the configuration script of the initialization stage based on the entry script of the initialization stage corresponding to the project information and provides the application service corresponding to the project information.
2. The artificial intelligence application deployment environment construction method of claim 1, further comprising:
providing a multi-segment environment, the multi-segment environment comprising: developing a network segment, an integrated network segment, a pressure measurement network segment and a production network segment;
the packaging subsystem comprises the development network segment, and the deployment subsystem comprises the integration network segment, the pressure measurement network segment and the production network segment; the development network segment, the integration network segment, the pressure measurement network segment and the production network segment can not be communicated with each other.
3. The method for constructing an artificial intelligence application deployment environment according to claim 1, wherein the configuration scripts of each stage are stored on a cloud configuration server;
and the packaging subsystem acquires the configuration scripts of each stage from the cloud configuration server.
4. The method according to claim 1, wherein the generating an entry script for each stage according to the project information comprises:
and creating codes required by the project information based on the user code warehouse, and synchronously creating entry scripts of each stage corresponding to the project information, wherein the entry scripts of each stage respectively read scripts under a unified directory.
5. The method according to claim 3, wherein the step of executing the configuration scripts of the corresponding packaging stages by the packaging subsystem based on the entry scripts of the packaging stages corresponding to the project information, and packaging the configuration scripts of the stages corresponding to the project information into a compressed package to be sent to the deployment subsystem comprises:
the packaging subsystem pulls a configuration script of a packaging stage corresponding to the project information from the cloud configuration server based on an entry script of the packaging stage corresponding to the project information and executes the configuration script, the configuration script of the packaging stage pulls a configuration script of a corresponding deployment stage and a configuration script of an initialization stage to be stored in a path corresponding to the project information, and an internal variable of the configuration script of the deployment stage is modified;
and packaging the configuration scripts of each stage corresponding to the project information into a compressed packet, and pushing the compressed packet to the integration network segment, the pressure measurement network segment and the production network segment through a preset audit path.
6. The method according to claim 5, wherein the receiving the compressed package and executing the configuration script of the deployment phase based on the entry script of the deployment phase corresponding to the project information by the deployment subsystem comprises:
and the deployment subsystem receives and decompresses the compressed packet, executes the configuration script of the deployment stage by reading the entry script of the deployment stage corresponding to the project information, and modifies the internal variable of the configuration script of the corresponding initialization stage.
7. The method according to claim 6, wherein the executing, by the deployment subsystem, the configuration script of the initialization phase based on the entry script of the initialization phase corresponding to the project information, and the providing the application service corresponding to the project information comprises:
and the deployment subsystem executes the configuration script of the initialization stage by reading the entry script of the initialization stage corresponding to the project information, completes the initialization of the configuration, and pulls up the artificial intelligence application service corresponding to the project information.
8. An artificial intelligence application deployment environment construction apparatus, characterized in that the apparatus comprises:
an acquisition unit: the system comprises a user code warehouse, a development network segment and a database, wherein the user code warehouse is used for providing a user code warehouse required by an artificial intelligent application project in the development network segment, and acquiring project information created by a user based on the user code warehouse and input environment dependence configuration information;
a configuration unit: generating an entry script for each stage according to the project information, and acquiring a configuration script of each stage according to the environment-dependent configuration information, wherein each stage comprises a packaging stage, a deployment stage and an initialization stage;
the packaging unit is arranged in the packaging subsystem and used for executing the configuration scripts of the corresponding packaging stages based on the entry scripts of the packaging stages corresponding to the project information, packaging the configuration scripts of the stages corresponding to the project information into compression packets and sending the compression packets to the deployment subsystem;
the deployment unit is arranged in the deployment subsystem and used for receiving the compressed package and executing the configuration script of the deployment stage based on the entry script of the deployment stage corresponding to the project information;
the initialization unit is arranged in the deployment subsystem and used for executing the configuration script of the initialization stage based on the entry script of the initialization stage corresponding to the project information and providing the application service corresponding to the project information.
9. A computer device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores computer instructions executable by the at least one processor to cause the at least one processor to perform the method of any one of claims 1-7.
10. A computer-readable storage medium having computer instructions stored thereon for causing a computer to perform the method of any one of claims 1 to 7.
CN202210476521.9A 2022-04-29 2022-04-29 Artificial intelligence application deployment environment construction method and device Pending CN114995834A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202210476521.9A CN114995834A (en) 2022-04-29 2022-04-29 Artificial intelligence application deployment environment construction method and device
CA3198338A CA3198338A1 (en) 2022-04-29 2023-05-01 Artificial intelligence application deployment environment construction method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210476521.9A CN114995834A (en) 2022-04-29 2022-04-29 Artificial intelligence application deployment environment construction method and device

Publications (1)

Publication Number Publication Date
CN114995834A true CN114995834A (en) 2022-09-02

Family

ID=83024879

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210476521.9A Pending CN114995834A (en) 2022-04-29 2022-04-29 Artificial intelligence application deployment environment construction method and device

Country Status (2)

Country Link
CN (1) CN114995834A (en)
CA (1) CA3198338A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116737174A (en) * 2023-08-15 2023-09-12 中国兵器装备集团兵器装备研究所 Automatic subsystem generating tool and method based on open source hong Meng system

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116737174A (en) * 2023-08-15 2023-09-12 中国兵器装备集团兵器装备研究所 Automatic subsystem generating tool and method based on open source hong Meng system
CN116737174B (en) * 2023-08-15 2023-11-14 中国兵器装备集团兵器装备研究所 Automatic subsystem generating tool and method based on open source hong Meng system

Also Published As

Publication number Publication date
CA3198338A1 (en) 2023-10-29

Similar Documents

Publication Publication Date Title
US10353913B2 (en) Automating extract, transform, and load job testing
US8302093B2 (en) Automated deployment of defined topology in distributed computing environment
US20100162230A1 (en) Distributed computing system for large-scale data handling
US10162735B2 (en) Distributed system test automation framework
US9959336B2 (en) Compiling extract, transform, and load job test data cases
US9418241B2 (en) Unified platform for big data processing
US10585785B2 (en) Preservation of modifications after overlay removal from a container
CN106681718A (en) Toolkit-based desktop side plugin frame and implementation method thereof
CN111309734B (en) Method and system for automatically generating table data
US20210224084A1 (en) Logging of scripts executed in an information technology workflow orchestration system
CN111651219A (en) Method and equipment for managing multi-module project configuration file
CN114995834A (en) Artificial intelligence application deployment environment construction method and device
CN113741954A (en) System software generation method and device, electronic equipment and storage medium
CN111367512B (en) Method and device for creating Android library module dependency relationship in application development
CN113010180A (en) Firmware generation method, system, device and medium
US20140344802A1 (en) Shared application binary storage
CN110955415A (en) Method for adapting projection multi-platform service
CN113138768B (en) Application package generation method and device, electronic equipment and readable storage medium
US20180341475A1 (en) Just In Time Deployment with Package Managers
CN116431137A (en) Cross-development-framework component conversion method, processing system and component conversion device
US20230259347A1 (en) Container based generation of inputs for generic functions
US11150992B2 (en) File entity relationship management and source product backup
CN116126334A (en) File generation method, device, electronic equipment and computer readable storage medium
CN117311785A (en) Code packaging method, code packaging device, computer equipment and storage medium
CN114594959A (en) Method, apparatus, device, medium and product for generating application installation package

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination