CN116820663A - Mirror image construction method, device, storage medium and apparatus - Google Patents
Mirror image construction method, device, storage medium and apparatus Download PDFInfo
- Publication number
- CN116820663A CN116820663A CN202210314891.2A CN202210314891A CN116820663A CN 116820663 A CN116820663 A CN 116820663A CN 202210314891 A CN202210314891 A CN 202210314891A CN 116820663 A CN116820663 A CN 116820663A
- Authority
- CN
- China
- Prior art keywords
- image
- construction
- mirror image
- source code
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000010276 construction Methods 0.000 title claims abstract description 351
- 230000001419 dependent effect Effects 0.000 claims description 50
- 238000000034 method Methods 0.000 abstract description 43
- 238000012545 processing Methods 0.000 abstract description 5
- 238000010801 machine learning Methods 0.000 description 14
- 238000012549 training Methods 0.000 description 9
- 238000012795 verification Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 4
- 238000004806 packaging method and process Methods 0.000 description 4
- 239000010979 ruby Substances 0.000 description 4
- 229910001750 ruby Inorganic materials 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000008520 organization Effects 0.000 description 2
- 238000013468 resource allocation Methods 0.000 description 2
- HPTJABJPZMULFH-UHFFFAOYSA-N 12-[(Cyclohexylcarbamoyl)amino]dodecanoic acid Chemical compound OC(=O)CCCCCCCCCCCNC(=O)NC1CCCCC1 HPTJABJPZMULFH-UHFFFAOYSA-N 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/40—Transformation of program code
- G06F8/41—Compilation
- G06F8/43—Checking; Contextual analysis
- G06F8/433—Dependency analysis; Data or control flow analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
- G06F8/61—Installation
- G06F8/63—Image based installation; Cloning; Build to order
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45562—Creating, deleting, cloning virtual machine instances
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Facsimiles In General (AREA)
Abstract
The invention relates to the technical field of mirror image processing, and discloses a mirror image construction method, equipment, a storage medium and a device, wherein the method comprises the following steps: determining a source code construction package according to the source code type of a source code file, generating a construction mirror image based on the source code construction package, acquiring an operation mirror image, and constructing a target mirror image according to the construction mirror image and the operation mirror image; the invention can generate the built mirror image and build the target mirror image based on the built mirror image and the running mirror image, thereby realizing the automatic construction of the mirror image and further improving the efficiency of the mirror image construction.
Description
Technical Field
The present invention relates to the field of mirror image processing technologies, and in particular, to a method, an apparatus, a storage medium, and a device for constructing a mirror image.
Background
The container technology is a novel virtualization technology, resources of a single operating system are divided into isolated groups, and the resources are distributed and managed in a balanced and coordinated mode among the isolated groups, so that the purpose of efficiently improving the utilization efficiency of the resources is achieved. The development of container technology plays an important role in the field of machine learning. The machine learning training process requires relying on a large amount of computing resources such as CPU and GPU for computational power support. According to the training process and the model size, the calculation force requirement and the calculation force occupation time are different. The container technology has good flexible resource allocation characteristic, and can flexibly solve the resource allocation problem in the machine learning training process, so that the container technology has wide application in the field of machine learning training.
The mirror image is the basis of container technology, and the container is created based on the mirror image, and the running mirror image becomes a container. The image typically contains the complete operating system, the user's running program, and the environment on which the program depends. In the current machine learning field, there are a plurality of frames of machine learning, such as TensorFlow, pytorch, caffe and the like. Different machine learning frameworks are dependent on the operating environment, and users need to build an operating image based on the framework according to the characteristics of the framework before performing machine learning training. And adding a software environment related to a drive file, a CUDA version and the like related to cluster hardware in the running image. At present, a user manually builds a machine learning platform mirror image, so that the defects of long building time and low building efficiency exist.
The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present invention and is not intended to represent an admission that the foregoing is prior art.
Disclosure of Invention
The invention mainly aims to provide a mirror image construction method, equipment, a storage medium and a device, and aims to solve the technical problems of long construction time and low construction efficiency in the prior art that a user manually constructs a machine learning platform mirror image.
In order to achieve the above object, the present invention provides a mirror image construction method, including the steps of:
determining a source code construction package according to the source code type of the source code file;
generating a construction mirror image based on the source code construction package;
and acquiring an operation mirror image, and constructing a target mirror image according to the construction mirror image and the operation mirror image.
Optionally, the step of generating the build image based on the source code build packet includes:
determining an environment dependent file according to the source code construction package;
downloading the environment dependent file;
and generating a built image based on the environment dependent file.
Optionally, the step of downloading the environment-dependent file includes:
acquiring a downloaded file, and determining a file to be downloaded according to the environment-dependent file and the downloaded file;
and downloading the file to be downloaded.
Optionally, after the step of obtaining the running mirror image and constructing the target mirror image according to the constructed mirror image and the running mirror image, the method further includes:
when the target image is successfully constructed, acquiring an image name of the target image;
and uploading the target image and the image name to a preset cloud.
Optionally, after the step of obtaining the running mirror image and constructing the target mirror image according to the constructed mirror image and the running mirror image, the method further includes:
When the target mirror image is successfully constructed, acquiring a mirror image state and a mirror image name of the target mirror image;
and sending the mirror image state and the mirror image name to the client terminal.
Optionally, after the step of obtaining the running mirror image and constructing the target mirror image according to the constructed mirror image and the running mirror image, the method further includes:
when the construction of the target image fails, acquiring construction step information and input and output information of the target image;
and generating an error log according to the construction step information and the input/output information.
Optionally, before the step of determining the source code construction package according to the source code type of the source code file, the method further includes:
when receiving a mirror image construction request, determining a mirror image construction mode according to the mirror image construction request;
correspondingly, the determining the source code construction package according to the source code type of the source code file comprises the following steps:
when the image construction mode is that the text file without the image is constructed, a source code construction package is determined according to the source code type of the source code file.
Optionally, after the step of determining the image construction mode according to the image construction request when the image construction request is received, the method further includes:
when the image construction mode is based on image text file construction, decompressing the image text file and the dependent file to obtain a source code construction environment;
And under the source code construction environment, analyzing the mirror image text file to obtain a target mirror image.
Optionally, when the image construction mode is based on image text file construction, decompressing the image text file and the dependent file to obtain a source code construction environment, including:
when the mirror image construction mode is based on the construction of the mirror image text file, configuring the mirror image text file and the dependent file;
decompressing the configured mirror image text file and the dependency file to obtain a source code construction environment.
Optionally, after the step of analyzing the image text file in the source code construction environment to obtain the target image, the method further includes:
when the target image is successfully constructed, acquiring an image name of the target image;
and uploading the target image and the image name to a preset cloud.
Optionally, after the step of uploading the target image and the image name to a preset cloud, the method further includes:
acquiring an uploaded image, and verifying the uploaded image;
and after passing the verification, the mirror name is sent to the client terminal.
Optionally, the step of obtaining the uploaded image and verifying the uploaded image includes:
Acquiring an uploaded image, and matching the uploaded image with the target image;
and verifying the uploaded mirror image according to the matching result.
In addition, in order to achieve the above object, the present invention also proposes a mirror image construction apparatus including a memory, a processor, and a mirror image construction program stored on the memory and executable on the processor, the mirror image construction program being configured to implement the mirror image construction method as described above.
In addition, in order to achieve the above object, the present invention also proposes a storage medium having stored thereon a mirror image construction program which, when executed by a processor, implements the mirror image construction method as described above.
In addition, in order to achieve the above object, the present invention also proposes a mirror image construction apparatus including: the system comprises a construction package determining module, a mirror image generating module and a mirror image construction module;
the construction package determining module is used for determining a source code construction package according to the source code type of the source code file;
the image generation module is used for generating a built image based on the source code built packet;
the image construction module is used for acquiring the running image and constructing the target image according to the constructed image and the running image.
Optionally, the image generating module is further configured to determine an environment dependent file according to the source code building packet;
the mirror image generation module is also used for downloading the environment dependent file;
the image generation module is further used for generating a built image based on the environment dependent file.
Optionally, the image generating module is further configured to obtain a downloaded file, and determine a file to be downloaded according to the environment-dependent file and the downloaded file;
the image generation module is also used for downloading the file to be downloaded.
Optionally, the image construction device further includes: an information uploading module;
the information uploading module is used for acquiring the image name of the target image when the target image is successfully constructed;
the information uploading module is further configured to upload the target image and the image name to a preset cloud.
Optionally, the image construction device further includes: an information transmitting module;
the information sending module is used for acquiring the image state and the image name of the target image when the target image is successfully constructed;
the information sending module is further configured to send the image state and the image name to a client terminal.
Optionally, the image construction device further includes: a log generation module;
the log generation module is used for acquiring construction step information and input and output information of the target image when the construction of the target image fails;
the log generation module is further configured to generate an error log according to the construction step information and the input/output information.
The invention discloses a method for constructing a target mirror image by determining a source code construction package according to a source code type of a source code file, generating a construction mirror image based on the source code construction package, acquiring an operation mirror image, and constructing the target mirror image according to the construction mirror image and the operation mirror image; the invention can generate the built mirror image and build the target mirror image based on the built mirror image and the running mirror image, thereby realizing the automatic construction of the mirror image and further improving the efficiency of the mirror image construction.
Drawings
FIG. 1 is a schematic diagram of a hardware running environment mirror image construction device according to an embodiment of the present invention;
FIG. 2 is a flow chart of a first embodiment of the mirror image construction method of the present invention;
FIG. 3 is a diagram of a mirror image construction page according to an embodiment of the present invention;
FIG. 4 is a flow chart of a second embodiment of the mirror image construction method of the present invention;
FIG. 5 is a flow chart of a third embodiment of the mirror image construction method of the present invention;
fig. 6 is a block diagram showing the construction of a first embodiment of the mirror image construction apparatus of the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a mirror image construction device of a hardware running environment according to an embodiment of the present invention.
As shown in fig. 1, the mirror image construction apparatus may include: a processor 1001, such as a central processing unit (Central Processing Unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, a memory 1005. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display (Display), and the optional user interface 1003 may also include a standard wired interface, a wireless interface, and the wired interface for the user interface 1003 may be a USB interface in the present invention. The network interface 1004 may optionally include a standard wired interface, a Wireless interface (e.g., a Wireless-Fidelity (Wi-Fi) interface). The Memory 1005 may be a high-speed random access Memory (Random Access Memory, RAM) or a stable Memory (NVM), such as a disk Memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above.
Those skilled in the art will appreciate that the structure shown in FIG. 1 is not limiting of the mirror image build apparatus and may include more or fewer components than shown, or may combine certain components, or may be a different arrangement of components.
As shown in fig. 1, an operating system, a network communication module, a user interface module, and a mirror image construction program may be included in a memory 1005, which is considered to be one type of computer storage medium.
In the mirror image construction apparatus shown in fig. 1, the network interface 1004 is mainly used for connecting to a background server, and performing data communication with the background server; the user interface 1003 is mainly used for connecting user equipment; the image construction apparatus calls the image construction program stored in the memory 1005 through the processor 1001 and executes the image construction method provided by the embodiment of the present invention.
Based on the above hardware structure, an embodiment of the mirror image construction method of the present invention is presented.
Referring to fig. 2, fig. 2 is a schematic flow chart of a first embodiment of the mirror image construction method according to the present invention.
In a first embodiment, the mirror image construction method includes the steps of:
step S10: and determining a source code construction package according to the source code type of the source code file.
It should be understood that the execution body of the method of this embodiment may be a mirror image construction device with functions of data processing, network communication and program running, for example, a computer, a server, or other electronic devices capable of implementing the same or similar functions, which is not limited in this embodiment.
It can be appreciated that in the existing machine learning training process, a user needs to manually construct a specific running mirror image according to a selected machine learning frame, and the designed model can be run in the computing cluster after the mirror image construction is completed. Typically, the process of manual image construction involves several steps, first requiring the local installation of the Docker environment and then writing the corresponding Dockerfile file. Dockerfile mainly defines information such as depending on environment package base images, program code paths, container entries and the like, and needs to follow certain editing rules. After the writing of the Dockerfile file is completed, a 'dock build' command is executed locally to execute the packaging of the mirror image. After the mirror image packaging is completed, a user needs to input account and password information of the remote mirror image warehouse, and manually pushes the local mirror image to the remote mirror image warehouse. After the mirror image is pushed successfully, the mirror image can be successfully pulled in the computing cluster, and a corresponding training process is executed.
The manual mirror image construction process has a plurality of problems, firstly, a user needs to install a Docker environment locally in advance, meanwhile, the user needs to be familiar with writing rules of Docker files, and certain learning cost exists. The image construction process also depends on the network connection condition, and has the problems of longer image construction time, failure in downloading depending on the software and the like caused by the network problem. Under a plurality of mirror image construction scenes, the manual construction process cannot effectively organize and manage the mirror image construction history, and the problems of missing or repeated construction, disordered mirror image version numbers and the like exist, so that the manual mirror image construction efficiency of a user is low.
Therefore, in order to overcome the above-mentioned drawbacks, in this embodiment, a source code construction package is determined according to a source code type of a source code file, then a construction image is generated based on the source code construction package, and then an operation image is acquired, and a target image is constructed according to the construction image and the operation image.
It should be noted that, the source code file may be uploaded in advance by the user. For example, the user may upload the source code file in advance in the form of a Git repository address, an S3 bucket address, a tar file package, etc., which is not limited in this embodiment.
It should be appreciated that the mirror image construction method corresponding to different source code types is different. Therefore, in order to ensure the reliability of image construction, in this embodiment, the source code construction package is determined according to the source code type of the source code file. For example, the Python source code type corresponds to the Python build package, the Java source code type corresponds to the Java build package, the Node source code type corresponds to the Node build package, the Ruby source code type corresponds to the Ruby build package, and the like, which is not limited in this embodiment.
Step S20: and generating a construction mirror image based on the source code construction package.
It should be noted that, after the construction of the environment layer image is completed, the image may be constructed by combining the running image to generate the target image.
It should be appreciated that generating the build image based on the source build package may be executing the source build package to generate the build image.
Step S30: and acquiring an operation mirror image, and constructing a target mirror image according to the construction mirror image and the operation mirror image.
It should be noted that the running image may be an image currently in a running state.
It is to be understood that the construction of the target image according to the build image and the running image may be performed to build the environment layer image by the build image, and after the environment layer image is built, the build image is combined with the running image to generate the target image, thereby completing the construction of the image.
Further, after the image construction is completed, in order to facilitate the user to find the constructed image, the step S30 further includes:
when the target image is successfully constructed, acquiring an image name of the target image;
and uploading the target image and the image name to a preset cloud.
It should be noted that, the image name of the target image may be input by the user at the same time when uploading the source code file, or may be generated by the image construction device, which is not limited in this embodiment.
It will be appreciated that, in order to facilitate the user to find the built image, in this embodiment, the target image and the image name may be uploaded to the image repository. The mirror image warehouse can be a preset storage area in a preset cloud end and is used for storing mirror images.
Further, in order to alert the user, after the step S30, the method further includes:
when the target mirror image is successfully constructed, acquiring a mirror image state and a mirror image name of the target mirror image;
and sending the mirror image state and the mirror image name to the client terminal.
It should be noted that the mirror state may be a state of whether or not the construction is successful.
It should be understood that, in order to remind the user of the image construction state, in this embodiment, when the target image is successfully constructed, the image state and the image name of the target image may also be obtained, and the image state and the image name may be sent to the client terminal.
Further, in order to facilitate the user to find the failure reason after the mirror image construction fails, the step S30 further includes:
when the construction of the target image fails, acquiring construction step information and input and output information of the target image;
and generating an error log according to the construction step information and the input/output information.
It should be noted that the construction step information may be different construction flow history step information. The input/output information may be input/output information at the time of mirror image construction.
It can be understood that, in order to facilitate the user to find the failure reason after the image construction fails, the embodiment can obtain the construction step information and the input/output information of the target image when the construction of the target image fails, and generate an error log according to the construction step information and the input/output information.
For ease of understanding, the description is given with reference to fig. 3, but the present solution is not limited thereto. Fig. 3 is a schematic diagram of a mirror image construction page, in fig. 3, the page may be divided into three key areas, and the area (1) mainly provides a task list query function, and may construct a history record according to task names and mirror image name queries. The "create new image build" may initiate a new image build process. The area (2) builds a history list menu for building a successful mirror list and a mirror. The mirror image construction history list displays all successfully constructed mirror image addresses, and a user can conveniently inquire the mirror images successfully constructed by the history for organization and management of an available mirror image warehouse. The image construction history list mainly records the history of the image construction task as shown in area (3). The mirror image construction history list contains all construction history records, and each construction record contains a construction sequence number, a construction task state, a construction task name, a mirror image name, a construction log, start and stop and the like. The construction log can inquire historical step information and input and output information of different construction flows, when a construction task fails, a user can inquire specific failure reasons according to the construction log, modify a Dockerfile file or a source code according to the information, ensure that the construction requirements of images can be met depending on software environments or related parameters, and complete the image construction flow.
In a first embodiment, a source code construction package is determined according to a source code type of a source code file, a construction mirror image is generated based on the source code construction package, an operation mirror image is obtained, and a target mirror image is constructed according to the construction mirror image and the operation mirror image; because the image construction can be generated and the target image is constructed based on the image construction and the running image in the embodiment, the automatic construction of the image is realized, and the image construction efficiency is improved.
Referring to fig. 4, fig. 4 is a schematic flow chart of a second embodiment of the mirror image construction method according to the present invention, and the second embodiment of the mirror image construction method according to the present invention is proposed based on the first embodiment shown in fig. 2.
In a second embodiment, the step S20 includes:
step S201: and determining an environment dependent file according to the source code construction package.
It should be appreciated that different source build packages correspond to different build flows, e.g., where the source build package is a Python build package, the two flows are checked and built separately. Therefore, in order to ensure the reliability of image construction, in this embodiment, the environment-dependent file is first determined according to the source code construction package.
Step S202: and downloading the environment dependent file.
It is understood that the download environment dependent file may be a software package corresponding to the download environment dependent file.
Further, in order to reduce the number of file downloads, the step S202 includes:
acquiring a downloaded file, and determining a file to be downloaded according to the environment-dependent file and the downloaded file;
and downloading the file to be downloaded.
It should be appreciated that the number of file downloads is excessive in view of the complete download environment dependent files. Therefore, in order to reduce the number of file downloads, in this embodiment, the downloaded files are acquired first, then the files to be downloaded are determined according to the environment-dependent files and the downloaded files, and finally the files to be downloaded are downloaded.
Step S203: and generating a built image based on the environment dependent file.
In a specific implementation, for example, within a Python build package, the two flows of checking and building are performed separately. The detection flow can explore environment dependency files in the code, such as a requirement. Txt file, and read a dependent software package in the file; the build flow may gradually download all software packages and then perform a hierarchical mirror build.
In a second embodiment, determining an environment-dependent file according to a source code build package, downloading the environment-dependent file, and generating a build image based on the environment-dependent file; because the embodiment determines the environment dependent file first and then generates the constructed image based on the environment dependent file, the reliability of constructing the image is ensured.
Referring to fig. 5, fig. 5 is a schematic flow chart of a third embodiment of the mirror image construction method according to the present invention, and based on the first embodiment shown in fig. 2, the third embodiment of the mirror image construction method according to the present invention is proposed.
In a third embodiment, before the step S10, the method further includes:
step S01: and when receiving the image construction request, determining an image construction mode according to the image construction request.
It should be understood that in practical applications, in addition to the mirroring manner described above, mirroring may also be automatically constructed based on the Dockerfile file. Therefore, when receiving the image construction request, the embodiment determines the image construction mode according to the image construction request.
Accordingly, the step S10 includes:
step S10': when the image construction mode is that the text file without the image is constructed, a source code construction package is determined according to the source code type of the source code file.
It can be appreciated that when the image construction mode is no image text file construction, the source code construction package can be determined according to the source code type of the source code file.
It should be noted that the mirrored text file may be a Dockerfile file.
In a third embodiment, it is disclosed that when a mirror image construction request is received, a mirror image construction mode is determined according to the mirror image construction request, and when the mirror image construction mode is a construction without a mirror image text file, a source code construction package is determined according to a source code type of a source code file; when receiving the image construction request, the embodiment determines the image construction mode according to the image construction request, so that the image construction in multiple modes can be realized, and the convenience of image construction is improved.
In a third embodiment, after the step S01, the method further includes:
step S10': when the image construction mode is based on image text file construction, decompressing the image text file and the dependent file to obtain a source code construction environment.
It should be noted that, the mirrored text file may be uploaded in advance by the user. For example, the user may upload the mirrored text file in advance by way of local path upload, tar file upload, standard input, S3 bucket, and Git repository upload, to which this embodiment is not limited.
It should be understood that, in order to prepare the image construction for the related file, in this embodiment, the image text file and the dependent file may be decompressed first to construct a source code construction environment.
Further, in order to ensure that the mirrored text file and the dependent file are executed correctly, the step S10″ includes:
when the mirror image construction mode is based on the construction of the mirror image text file, configuring the mirror image text file and the dependent file;
decompressing the configured mirror image text file and the dependency file to obtain a source code construction environment.
It should be understood that, in order to ensure that the image text file and the dependency file execute correctly, the image text file and the dependency file may be configured first in this embodiment, for example, the image text file and the dependency file may be configured to ensure that the Dockerfile file and the dependency file are in the same hierarchy.
Step S20': and under the source code construction environment, analyzing the mirror image text file to obtain a target mirror image.
It can be understood that in the source code construction environment, the image text file is parsed, and the target image can be obtained by parsing the instruction of the Dockerfile file one by one in the source code construction environment, downloading the corresponding software and environment, and completing the image construction after the image layered construction work is executed.
Further, after the image construction is completed, in order to facilitate the user to find the constructed image, the step S20″ further includes:
when the target image is successfully constructed, acquiring an image name of the target image;
and uploading the target image and the image name to a preset cloud.
It should be noted that, the image name of the target image may be input by the user at the same time when uploading the source code file, or may be generated by the image construction device, which is not limited in this embodiment.
It will be appreciated that, in order to facilitate the user to find the built image, in this embodiment, the target image and the image name may be uploaded to the image repository. The mirror image warehouse can be a preset storage area in a preset cloud end and is used for storing mirror images.
Further, in order to return the image name to the user after the target image passes the verification, after uploading the target image and the image name to the preset cloud, the method further includes:
acquiring an uploaded image, and verifying the uploaded image;
and after passing the verification, the mirror name is sent to the client terminal.
It should be noted that the uploaded image may be an image uploaded to a preset cloud.
It should be appreciated that the verification of the uploaded image may be by a preset algorithm. Wherein, the preset algorithm can be preset.
Further, to ensure that the constructed target image is completely consistent with the cloud image, the obtaining the uploaded image and verifying the uploaded image includes:
acquiring an uploaded image, and matching the uploaded image with the target image;
and verifying the uploaded mirror image according to the matching result.
It is to be appreciated that matching the uploaded image to the target image may be based on the SHA256 algorithm.
It should be understood that the verification of the uploaded image according to the matching result may be that when the matching result is successful, it is determined that the verification of the uploaded image is passed; and when the matching result is that the matching fails, judging that the uploaded image fails to pass the verification.
In a third embodiment, when the mirror image construction mode is based on mirror image text file construction, decompressing the mirror image text file and the dependent file to obtain a source code construction environment, and analyzing the mirror image text file under the source code construction environment to obtain a target mirror image; the embodiment can also construct the target mirror image based on the mirror image text file, so that convenience in constructing the mirror image is improved.
In addition, the embodiment of the invention also provides a storage medium, wherein the storage medium stores a mirror image construction program, and the mirror image construction program realizes the mirror image construction method when being executed by a processor.
In addition, referring to fig. 6, an embodiment of the present invention further proposes a mirror image construction apparatus, including: a build package determination module 10, a mirror image generation module 20, and a mirror image build module 30;
the building packet determining module 10 is configured to determine a source code building packet according to a source code type of a source code file.
It can be appreciated that in the existing machine learning training process, a user needs to manually construct a specific running mirror image according to a selected machine learning frame, and the designed model can be run in the computing cluster after the mirror image construction is completed. Typically, the process of manual image construction involves several steps, first requiring the local installation of the Docker environment and then writing the corresponding Dockerfile file. Dockerfile mainly defines information such as depending on environment package base images, program code paths, container entries and the like, and needs to follow certain editing rules. After the writing of the Dockerfile file is completed, a 'dock build' command is executed locally to execute the packaging of the mirror image. After the mirror image packaging is completed, a user needs to input account and password information of the remote mirror image warehouse, and manually pushes the local mirror image to the remote mirror image warehouse. After the mirror image is pushed successfully, the mirror image can be successfully pulled in the computing cluster, and a corresponding training process is executed.
The manual mirror image construction process has a plurality of problems, firstly, a user needs to install a Docker environment locally in advance, meanwhile, the user needs to be familiar with writing rules of Docker files, and certain learning cost exists. The image construction process also depends on the network connection condition, and has the problems of longer image construction time, failure in downloading depending on the software and the like caused by the network problem. Under a plurality of mirror image construction scenes, the manual construction process cannot effectively organize and manage the mirror image construction history, and the problems of missing or repeated construction, disordered mirror image version numbers and the like exist, so that the manual mirror image construction efficiency of a user is low.
Therefore, in order to overcome the above-mentioned drawbacks, in this embodiment, a source code construction package is determined according to a source code type of a source code file, then a construction image is generated based on the source code construction package, and then an operation image is acquired, and a target image is constructed according to the construction image and the operation image.
It should be noted that, the source code file may be uploaded in advance by the user. For example, the user may upload the source code file in advance in the form of a Git repository address, an S3 bucket address, a tar file package, etc., which is not limited in this embodiment.
It should be appreciated that the mirror image construction method corresponding to different source code types is different. Therefore, in order to ensure the reliability of image construction, in this embodiment, the source code construction package is determined according to the source code type of the source code file. For example, the Python source code type corresponds to the Python build package, the Java source code type corresponds to the Java build package, the Node source code type corresponds to the Node build package, the Ruby source code type corresponds to the Ruby build package, and the like, which is not limited in this embodiment.
The image generating module 20 is configured to generate a build image based on the source code build packet.
It should be noted that, after the construction of the environment layer image is completed, the image may be constructed by combining the running image to generate the target image.
It should be appreciated that generating the build image based on the source build package may be executing the source build package to generate the build image.
The image construction module 30 is configured to obtain a running image, and construct a target image according to the constructed image and the running image.
It should be noted that the running image may be an image currently in a running state.
It is to be understood that the construction of the target image according to the build image and the running image may be performed to build the environment layer image by the build image, and after the environment layer image is built, the build image is combined with the running image to generate the target image, thereby completing the construction of the image.
Further, after the mirror image construction is completed, in order to facilitate the user to find the constructed mirror image, the mirror image construction apparatus further includes: an information uploading module;
the information uploading module is used for acquiring the image name of the target image when the target image is successfully constructed;
the information uploading module is further configured to upload the target image and the image name to a preset cloud.
It should be noted that, the image name of the target image may be input by the user at the same time when uploading the source code file, or may be generated by the image construction device, which is not limited in this embodiment.
It will be appreciated that, in order to facilitate the user to find the built image, in this embodiment, the target image and the image name may be uploaded to the image repository. The mirror image warehouse can be a preset storage area in a preset cloud end and is used for storing mirror images.
Further, in order to alert the user, the image construction apparatus further includes: an information transmitting module;
the information sending module is used for acquiring the image state and the image name of the target image when the target image is successfully constructed;
the information sending module is further configured to send the image state and the image name to a client terminal.
It should be noted that the mirror state may be a state of whether or not the construction is successful.
It should be understood that, in order to remind the user of the image construction state, in this embodiment, when the target image is successfully constructed, the image state and the image name of the target image may also be obtained, and the image state and the image name may be sent to the client terminal.
Further, in order to facilitate the user to find the failure reason after the image construction fails, the image construction device further includes: a log generation module;
the log generation module is used for acquiring construction step information and input and output information of the target image when the construction of the target image fails;
the log generation module is further configured to generate an error log according to the construction step information and the input/output information.
It should be noted that the construction step information may be different construction flow history step information. The input/output information may be input/output information at the time of mirror image construction.
It can be understood that, in order to facilitate the user to find the failure reason after the image construction fails, the embodiment can obtain the construction step information and the input/output information of the target image when the construction of the target image fails, and generate an error log according to the construction step information and the input/output information.
For ease of understanding, the description is given with reference to fig. 3, but the present solution is not limited thereto. Fig. 3 is a schematic diagram of a mirror image construction page, in fig. 3, the page may be divided into three key areas, and the area (1) mainly provides a task list query function, and may construct a history record according to task names and mirror image name queries. The "create new image build" may initiate a new image build process. The area (2) builds a history list menu for building a successful mirror list and a mirror. The mirror image construction history list displays all successfully constructed mirror image addresses, and a user can conveniently inquire the mirror images successfully constructed by the history for organization and management of an available mirror image warehouse. The image construction history list mainly records the history of the image construction task as shown in area (3). The mirror image construction history list contains all construction history records, and each construction record contains a construction sequence number, a construction task state, a construction task name, a mirror image name, a construction log, start and stop and the like. The construction log can inquire historical step information and input and output information of different construction flows, when a construction task fails, a user can inquire specific failure reasons according to the construction log, modify a Dockerfile file or a source code according to the information, ensure that the construction requirements of images can be met depending on software environments or related parameters, and complete the image construction flow.
In the embodiment, a source code construction package is determined according to a source code type of a source code file, a construction mirror image is generated based on the source code construction package, an operation mirror image is obtained, and a target mirror image is constructed according to the construction mirror image and the operation mirror image; because the image construction can be generated and the target image is constructed based on the image construction and the running image in the embodiment, the automatic construction of the image is realized, and the image construction efficiency is improved.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. read only memory mirror (Read Only Memory image, ROM)/random access memory (Random Access Memory, RAM), magnetic disk, optical disk), comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
The invention discloses A1, a mirror image construction method, which comprises the following steps:
determining a source code construction package according to the source code type of the source code file;
generating a construction mirror image based on the source code construction package;
and acquiring an operation mirror image, and constructing a target mirror image according to the construction mirror image and the operation mirror image.
A2, the image construction method according to A1, wherein the step of generating the constructed image based on the source code construction package comprises the following steps:
determining an environment dependent file according to the source code construction package;
downloading the environment dependent file;
and generating a built image based on the environment dependent file.
A3, the image construction method according to A2, wherein the step of downloading the environment-dependent file includes:
acquiring a downloaded file, and determining a file to be downloaded according to the environment-dependent file and the downloaded file;
and downloading the file to be downloaded.
A4, after the step of obtaining the running mirror image and constructing the target mirror image according to the constructed mirror image and the running mirror image, the mirror image construction method according to A1 further includes:
when the target image is successfully constructed, acquiring an image name of the target image;
And uploading the target image and the image name to a preset cloud.
A5, after the step of obtaining the running mirror image and constructing the target mirror image according to the constructed mirror image and the running mirror image, the mirror image construction method according to A1 further includes:
when the target mirror image is successfully constructed, acquiring a mirror image state and a mirror image name of the target mirror image;
and sending the mirror image state and the mirror image name to the client terminal.
A6, after the step of obtaining the running mirror image and constructing the target mirror image according to the constructed mirror image and the running mirror image, the mirror image construction method according to A1 further includes:
when the construction of the target image fails, acquiring construction step information and input and output information of the target image;
and generating an error log according to the construction step information and the input/output information.
A7, the mirror image construction method according to any one of A1 to A6, before the step of determining the source code construction package according to the source code type of the source code file, further comprises:
when receiving a mirror image construction request, determining a mirror image construction mode according to the mirror image construction request;
correspondingly, the determining the source code construction package according to the source code type of the source code file comprises the following steps:
When the image construction mode is that the text file without the image is constructed, a source code construction package is determined according to the source code type of the source code file.
A8, the image construction method according to A7, after the step of determining the image construction mode according to the image construction request when receiving the image construction request, further includes:
when the image construction mode is based on image text file construction, decompressing the image text file and the dependent file to obtain a source code construction environment;
and under the source code construction environment, analyzing the mirror image text file to obtain a target mirror image.
A9, when the image construction mode is based on image text file construction, decompressing the image text file and the dependent file to obtain a source code construction environment, wherein the method comprises the following steps:
when the mirror image construction mode is based on the construction of the mirror image text file, configuring the mirror image text file and the dependent file;
decompressing the configured mirror image text file and the dependency file to obtain a source code construction environment.
A10, the mirror image construction method as described in A9, wherein after the step of analyzing the mirror image text file in the source code construction environment to obtain the target mirror image, the method further comprises:
When the target image is successfully constructed, acquiring an image name of the target image;
and uploading the target image and the image name to a preset cloud.
A11, after the step of uploading the target image and the image name to a preset cloud, the image construction method as described in A10 further includes:
acquiring an uploaded image, and verifying the uploaded image;
and after passing the verification, the mirror name is sent to the client terminal.
A12, the image construction method as described in A11, wherein the steps of obtaining the uploaded image and verifying the uploaded image include:
acquiring an uploaded image, and matching the uploaded image with the target image;
and verifying the uploaded mirror image according to the matching result.
The invention also discloses B13, a mirror image construction device, the mirror image construction device comprises: the system comprises a memory, a processor and a mirror image construction program stored on the memory and capable of running on the processor, wherein the mirror image construction program realizes the mirror image construction method when being executed by the processor.
The invention also discloses C14, a storage medium, the storage medium stores a mirror image construction program, and the mirror image construction program realizes the mirror image construction method when being executed by a processor.
The invention also discloses a D15 mirror image construction device, which comprises: the system comprises a construction package determining module, a mirror image generating module and a mirror image construction module;
the construction package determining module is used for determining a source code construction package according to the source code type of the source code file;
the image generation module is used for generating a built image based on the source code built packet;
the image construction module is used for acquiring the running image and constructing the target image according to the constructed image and the running image.
The image construction device as described in D15, wherein the image generation module is further configured to determine an environment dependent file according to the source code construction package;
the mirror image generation module is also used for downloading the environment dependent file;
the image generation module is further used for generating a built image based on the environment dependent file.
The image construction device as described in D17, wherein the image generation module is further configured to obtain a downloaded file, and determine a file to be downloaded according to the environment-dependent file and the downloaded file;
the image generation module is also used for downloading the file to be downloaded.
D18, the mirror image construction apparatus according to D15, further comprising: an information uploading module;
The information uploading module is used for acquiring the image name of the target image when the target image is successfully constructed;
the information uploading module is further configured to upload the target image and the image name to a preset cloud.
D19, the mirror image construction apparatus according to D15, further comprising: an information transmitting module;
the information sending module is used for acquiring the image state and the image name of the target image when the target image is successfully constructed;
the information sending module is further configured to send the image state and the image name to a client terminal.
D20, the image construction apparatus according to any one of D15 to D19, further comprising: a log generation module;
the log generation module is used for acquiring construction step information and input and output information of the target image when the construction of the target image fails;
the log generation module is further configured to generate an error log according to the construction step information and the input/output information.
Claims (10)
1. The mirror image construction method is characterized by comprising the following steps of:
determining a source code construction package according to the source code type of the source code file;
Generating a construction mirror image based on the source code construction package;
and acquiring an operation mirror image, and constructing a target mirror image according to the construction mirror image and the operation mirror image.
2. The image construction method according to claim 1, wherein the step of generating the constructed image based on the source code construction package includes:
determining an environment dependent file according to the source code construction package;
downloading the environment dependent file;
and generating a built image based on the environment dependent file.
3. The image construction method according to claim 2, wherein the step of downloading the environment-dependent file comprises:
acquiring a downloaded file, and determining a file to be downloaded according to the environment-dependent file and the downloaded file;
and downloading the file to be downloaded.
4. The image construction method according to claim 1, wherein after the step of acquiring the running image and constructing the target image from the constructed image and the running image, further comprising:
when the target image is successfully constructed, acquiring an image name of the target image;
and uploading the target image and the image name to a preset cloud.
5. The image construction method according to claim 1, wherein after the step of acquiring the running image and constructing the target image from the constructed image and the running image, further comprising:
When the target mirror image is successfully constructed, acquiring a mirror image state and a mirror image name of the target mirror image;
and sending the mirror image state and the mirror image name to the client terminal.
6. The image construction method according to claim 1, wherein after the step of acquiring the running image and constructing the target image from the constructed image and the running image, further comprising:
when the construction of the target image fails, acquiring construction step information and input and output information of the target image;
and generating an error log according to the construction step information and the input/output information.
7. The image construction method according to any one of claims 1 to 6, wherein before the step of determining the source code construction package according to the source code type of the source code file, further comprising:
when receiving a mirror image construction request, determining a mirror image construction mode according to the mirror image construction request;
correspondingly, the determining the source code construction package according to the source code type of the source code file comprises the following steps:
when the image construction mode is that the text file without the image is constructed, a source code construction package is determined according to the source code type of the source code file.
8. A mirror image construction apparatus, characterized in that the mirror image construction apparatus comprises: memory, a processor and a mirror construction program stored on the memory and executable on the processor, which when executed by the processor implements the mirror construction method according to any one of claims 1 to 7.
9. A storage medium having stored thereon a mirror image construction program which, when executed by a processor, implements the mirror image construction method according to any one of claims 1 to 7.
10. A mirror image construction apparatus, characterized in that the mirror image construction apparatus comprises: the system comprises a construction package determining module, a mirror image generating module and a mirror image construction module;
the construction package determining module is used for determining a source code construction package according to the source code type of the source code file;
the image generation module is used for generating a built image based on the source code built packet;
the image construction module is used for acquiring the running image and constructing the target image according to the constructed image and the running image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210314891.2A CN116820663A (en) | 2022-03-22 | 2022-03-22 | Mirror image construction method, device, storage medium and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210314891.2A CN116820663A (en) | 2022-03-22 | 2022-03-22 | Mirror image construction method, device, storage medium and apparatus |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116820663A true CN116820663A (en) | 2023-09-29 |
Family
ID=88127981
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210314891.2A Pending CN116820663A (en) | 2022-03-22 | 2022-03-22 | Mirror image construction method, device, storage medium and apparatus |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116820663A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20240036990A1 (en) * | 2021-06-15 | 2024-02-01 | Inspur Suzhou Intelligent Technology Co., Ltd. | Inference service management method, apparatus and system for inference platform, and medium |
-
2022
- 2022-03-22 CN CN202210314891.2A patent/CN116820663A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20240036990A1 (en) * | 2021-06-15 | 2024-02-01 | Inspur Suzhou Intelligent Technology Co., Ltd. | Inference service management method, apparatus and system for inference platform, and medium |
| US11994958B2 (en) * | 2021-06-15 | 2024-05-28 | Inspur Suzhou Intelligent Technology Co., Ltd. | Inference service management method, apparatus and system for inference platform, and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10637796B2 (en) | Linking instances within a cloud computing environment | |
| CN107766126B (en) | Container mirror image construction method, system and device and storage medium | |
| CN108037961B (en) | Application program configuration method, device, server and storage medium | |
| US10970194B2 (en) | Systems and methods for initializing test environments | |
| US9912759B2 (en) | Dynamically generating solution stacks | |
| CN110825479A (en) | Page processing method and device, terminal equipment, server and storage medium | |
| CN112769706B (en) | Componentized routing method and system | |
| CN111694620A (en) | Interaction method, device and equipment of third-party service and computer storage medium | |
| US12014170B2 (en) | Mini program batch processing method and apparatus, electronic device, and readable storage medium | |
| CN116820663A (en) | Mirror image construction method, device, storage medium and apparatus | |
| US11429596B2 (en) | Update for configuration file | |
| US20220012065A1 (en) | Registry image management | |
| CN112491940B (en) | Request forwarding method and device of proxy server, storage medium and electronic equipment | |
| CN112235132A (en) | Method, device, medium and server for dynamically configuring service | |
| WO2023084345A1 (en) | Automated deployment of enterprise archive with dependency on application server via script | |
| CN109725944A (en) | Remote procedure calling (PRC) method and device, electronic equipment, storage medium | |
| CN114489674A (en) | Data verification method and device of dynamic data model | |
| CN114237736A (en) | Page resource loading method, system, device, computer equipment and storage medium | |
| CN112083945A (en) | NPM installation package update prompting method and device, electronic equipment and storage medium | |
| CN113505036B (en) | Application monitoring method, client and server | |
| WO2023193682A1 (en) | Local arrangement of remote deployment | |
| CN117369946A (en) | Container deployment method and device based on DPU, electronic equipment and medium | |
| CN116680213A (en) | Data processing method, computer device, and readable storage medium | |
| CN114095495A (en) | Terminal control method, terminal control equipment, storage medium and device | |
| CN117251250A (en) | Container management method based on cloud native platform and related equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |