CN114996134A - Containerized deployment method, electronic equipment and storage medium - Google Patents
Containerized deployment method, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114996134A CN114996134A CN202210614674.5A CN202210614674A CN114996134A CN 114996134 A CN114996134 A CN 114996134A CN 202210614674 A CN202210614674 A CN 202210614674A CN 114996134 A CN114996134 A CN 114996134A
- Authority
- CN
- China
- Prior art keywords
- target
- mpi
- container
- production environment
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000003860 storage Methods 0.000 title claims description 16
- 238000004519 manufacturing process Methods 0.000 claims abstract description 77
- 238000005457 optimization Methods 0.000 claims abstract description 51
- 238000011161 development Methods 0.000 claims abstract description 45
- 238000004891 communication Methods 0.000 claims description 63
- 238000004590 computer program Methods 0.000 claims description 20
- 238000004422 calculation algorithm Methods 0.000 claims description 18
- 244000035744 Hura crepitans Species 0.000 claims description 11
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000004458 analytical method Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 abstract description 12
- 230000006978 adaptation Effects 0.000 abstract description 11
- 238000005516 engineering process Methods 0.000 description 16
- 230000004044 response Effects 0.000 description 15
- 238000010586 diagram Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 11
- 238000012545 processing Methods 0.000 description 6
- 230000005236 sound signal Effects 0.000 description 4
- 238000012546 transfer Methods 0.000 description 4
- 206010016754 Flashback Diseases 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000004806 packaging method and process Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000005315 distribution function Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/362—Software debugging
- G06F11/3644—Software debugging by instrumenting at runtime
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45587—Isolation or security of virtual machine instances
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45591—Monitoring or debugging support
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Stored Programmes (AREA)
Abstract
In the embodiment of the application, the MPI application program is subjected to containerized deployment, and in the containerized deployment process, optimized target operation parameters of a development environment are reserved in a mirror image file, so that the target operation parameters of the development environment can be automatically reproduced in a production environment, the reproducibility and the portability of the target operation parameters in the development environment are enhanced, the operation environment adaptation work is greatly simplified, and the time cost and the energy of performance optimization personnel for the operation environment adaptation are reduced; furthermore, the operation performance of the MPI application program in the production environment is better ensured through the containerization of the MPI application program.
Description
Technical Field
The present application relates to the field of high-performance computing, and in particular, to a containerization deployment method, an electronic device, and a storage medium.
Background
In an HPC (High Performance Computing) scenario, MPI (Message Passing Interface) is widely used as an underlying communication component to develop MPI applications. In practical application, after an MPI application developed in a development environment is released to a production environment, due to a difference between the development environment and the production environment, the MPI application needs to perform running environment adaptation and perform performance optimization according to the production environment, which often requires performance optimization personnel to consume a large amount of time, cost and energy to complete an optimization task. However, in addition to being inefficient, such manual optimization approaches may also be difficult to ensure the operational performance of MPI applications in a production environment.
Disclosure of Invention
Aspects of the present disclosure provide a containerization deployment method, an electronic device, and a storage medium, which are used to ensure the operational performance of an MPI application in a production environment.
The embodiment of the application provides a containerization deployment method, which comprises the following steps: aiming at a development environment stage, optimizing the operation parameters of the development environment when debugging the executable file of the message passing interface MPI application program to obtain target operation parameters; creating an image file of the MPI application program at least according to the executable file and the target operation parameters; and aiming at the production environment stage, creating a container of the MPI application program in the production environment according to the image file, and configuring the production environment to have target operation parameters.
An embodiment of the present application further provides an electronic device, including: a memory and a processor; a memory for storing a computer program; a processor is coupled to the memory for executing the computer program for performing the steps in the containerized deployment method.
Embodiments of the present application further provide a computer storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the steps in the containerization deployment method.
In the embodiment of the application, the MPI application program is subjected to containerized deployment, and in the containerized deployment process, the optimized target operation parameters of the development environment are reserved in the image file, so that the production environment can automatically reproduce the target operation parameters of the development environment, the reproducibility and portability of the target operation parameters in the development environment are enhanced, the operation environment adaptation work is greatly simplified, and the time cost and the energy of performance optimization personnel for adapting the operation environment are reduced; furthermore, the operation performance of the MPI application program in the production environment is better ensured through the containerization of the MPI application program.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
fig. 1 is an application scenario diagram provided in an embodiment of the present application;
fig. 2 is a flowchart of a containerization deployment method according to an embodiment of the present disclosure;
fig. 3 is a flowchart of another containerization deployment method provided in an embodiment of the present application;
fig. 4 is a schematic structural diagram of a containerization deployment device according to an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
First, terms related to embodiments of the present application will be described:
MPI: the interface is an interface for writing parallel computing application programs, is used for communication among processes, and is often used in a cluster environment of a non-shared memory.
RDMA (Remote Direct Memory Access): the method is a technology for accessing data in the internal memory by bypassing the kernel of the operating system of the host, can improve the network throughput of the system and reduce the network communication delay of the system, and is widely applied to large-scale parallel computing scenes.
intel oneAPI toolset: the system is a tool set which is used for constructing, analyzing and optimizing cross-platform application on the basis of a CPU and an XPU, wherein an HPC suite comprises an MPI communication framework. The MPI components are closed source, but can be used for free; the MPI component is internally provided with a debugging component, and can also completely support the RDMA network protocol of each current cloud computing manufacturer.
Singularity container technology: the great difference between the technology and the Docker container technology is that the Singulary container used in the production environment is a read-only SIF format file, the packaging characteristic and the portability can be provided, and the performance of an application program can be lossless.
And (3) developing environment: is the operating environment of the software for the code developer.
The production environment is as follows: the release environment is an operating environment of software directly oriented to users, and the environment formally provides external services and is an environment used on a real subscriber line.
In practical application, after an MPI application developed in a development environment is released to a production environment, due to a difference between the development environment and the production environment, the MPI application needs to perform running environment adaptation and perform performance optimization according to the production environment, which often requires performance optimization personnel to consume a large amount of time, cost and energy to complete an optimization task. However, in addition to being inefficient, such manual optimization approaches may also be difficult to ensure the operational performance of MPI applications in a production environment. In view of the foregoing technical problems, an embodiment of the present application provides a containerized deployment method, an electronic device, and a storage medium, in which a MPI application is subjected to containerized deployment, and in the containerized deployment process, optimized target operation parameters of a development environment are retained in a mirror image file, so that a production environment can automatically reproduce the target operation parameters of the development environment, reproducibility and portability of the target operation parameters in the development environment are enhanced, operation environment adaptation work is greatly simplified, and time cost and effort of performance optimization personnel for operation environment adaptation are reduced; furthermore, the operation performance of the MPI application program in the production environment is better ensured through the containerization of the MPI application program.
The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.
Fig. 1 is an application scenario diagram provided in an embodiment of the present application. In this application scenario, two Singularity containers are deployed on different compute nodes. When the MPI application program is communicated across nodes, the whole communication flow is as follows: first, the Singularity container 1 on the computing node 1 is started, then the MPI application 1 is started in the Singularity container 1, then the MPI application 1 starts the Singularity container 2 on the computing node 2 over the RDMA network, the Singularity container 2 starts the MPI application 2, and the MPI application 1 and the MPI application 2 communicate with each other.
In this embodiment, the MPI application program is subjected to containerization deployment, so that the operation performance of the MPI application program in a production environment can be ensured, and particularly, when the MPI application program is subjected to cross-node communication, the communication success rate and the communication quality of the cross-node communication can be effectively ensured.
Notably, the computing node may be a terminal device or a server providing computing services. The terminal device may be hardware or software. When the terminal device is hardware, the terminal device is, for example, a mobile phone, a tablet computer, a desktop computer, a wearable smart device, a smart home device, or the like. When the terminal device is software, it may be installed in the above listed hardware devices, in this case, the terminal device is, for example, a plurality of software modules or a single software module, and the embodiments of the present application are not limited. The server may be hardware or software. When the server is hardware, the server is a single server or a distributed server cluster composed of a plurality of servers. When the server is software, the server may be a plurality of software modules or a single software module, and the embodiments of the present application are not limited.
It should be understood that the embodiments of the present application do not limit the number of compute nodes, Singularity containers, MPI applications, and the like included in fig. 1.
Fig. 2 is a flowchart of a containerization deployment method according to an embodiment of the present disclosure. The method may be performed by a containerization deployment apparatus, which may be implemented in software and/or hardware, and may generally be integrated into an electronic device. Referring to fig. 2, the method includes:
201. aiming at the development environment stage, when the executable file of the message transfer interface MPI application program is debugged, the operation parameters of the development environment are optimized to obtain the target operation parameters.
202. And creating an image file of the MPI application program according to at least the executable file and the target operation parameters.
203. And aiming at the production environment stage, creating a container of the MPI application program in the production environment according to the image file, and configuring the production environment to have target operation parameters.
In this embodiment, in order to implement containerized deployment of an MPI application program, in a development environment, a source code written for the MPI application program is first acquired, where the source code can implement a service function of the MPI application program; and compiling the source code to obtain an executable file of the MPI application program. In order to enhance the performance of the MPI application program, the target machine code compiled by the source code in the executable file is subjected to program debugging, and in the program debugging process, the operating parameters of the development environment are optimized to obtain the target operating parameters. The operating parameters of the development environment include, but are not limited to, operating parameters of a software environment and operating parameters of a hardware environment. The operating parameters of the software environment include, but are not limited to, various databases, message middleware, communication parameters of MPI, types and versions of operating systems or drivers; the operating parameters of the hardware environment include, but are not limited to, device parameters of an input/output I/O device, device parameters of a CPU (Central Processing Unit), device parameters of a GPU (graphics Processing Unit), device parameters of a disk or a memory, and the like.
In the HPC scenario, higher requirements are placed on network communication latency and throughput bandwidth. Therefore, as a further alternative, in order to enhance the communication quality between MPI applications, at least a target communication algorithm needs to be included in the target operation parameters during program debugging. Therefore, when the operating parameters of the development environment are optimized to obtain the target operating parameters, the target communication algorithm may be selected from a plurality of communication algorithms included in the MPI library according to at least one of a load, a calculation task, or a communication mode of the MPI application.
In practical applications, the tuning tool in the MPI component provided by the intel oneAPI may be used to perform automatic tuning, so as to select a target communication algorithm from a plurality of communication algorithms included in the MPI library through the tuning tool according to at least one of the load, the computing task, or the communication mode of the MPI application. Wherein, different MPI communication functions in the MPI library are realized by a plurality of communication algorithms. For example, the MPI library provides a plurality of communication functions such as a message transfer function int MPI _ Send (…), a message transfer function int MPI _ Recv (…), a message distribution function int MPI _ Scatter (…), and a message collection function int MPI _ Gather (…), each communication function is implemented by using one or more communication algorithms, and a communication algorithm that makes the application performance better is selected from the algorithm implementations.
The load of the MPI application may refer to an access traffic of the MPI application, and the access traffic may be estimated according to a user group size or a user demand.
The calculation task refers to a task that the MPI application undertakes, for example, a face recognition task, a voice recognition task, or a gesture recognition task.
The communication modes include, but are not limited to: a point-to-point communication mode, a multicast communication mode, and a multicast communication mode.
After target operating parameters of the MPI application in the development environment are obtained, the MPI application can be deployed in a container manner. Containerized deployments may employ, for example and without limitation: docker container technology or Singularity container technology. If the Docker container technology is adopted, a Docker container corresponding to the MPI application program is obtained through containerization deployment; if the Singularity container technology is adopted, the Singularity container corresponding to the MPI application program is obtained through containerization deployment.
When the MPI application program is subjected to containerized deployment, an image file of the MPI application program needs to be created. In order to enable the target operation parameters in the development environment to be reproduced in the production environment, reduce the time cost and the energy of performance optimization personnel, and ensure the operation performance of the MPI application program in the production environment, when the image file of the MPI application program is created, the image file of the MPI application program is created at least according to the executable file and the target operation parameters. Furthermore, in order to better ensure the operation performance of the MPI application program, in addition to packaging the executable file and the target operation parameter into the image file, the dependency library of the executable file, the MPI library and its version information, and the driver of the communication network may be integrally packaged into the image file. Preferably, in order to guarantee the communication quality between MPI applications, the communication network is an RDMA network.
In practical applications, the image file can be created in the following ways:
mode 1: generating a definition file for guiding to create an image file at least according to the executable file and a dependency library thereof, the MPI library and version information thereof, a driving program of a communication network and a target operation parameter; an image file is created from the definition file.
Taking the creation of the Singularity container as an example, firstly, a definition file in the def format is made, and then an image file in the image SIF format is created based on the definition file in the def format. For more description on creating an image file in the image SIF format using a definition file in the def format, refer to the related art.
Mode 2: creating an image file of the MPI application based on at least the executable file and the target operating parameters comprises: generating a file in a Sandbox Sandbox format at least according to the executable file and a dependency library thereof, the MPI library and version information thereof, a driving program of a communication network and a target operation parameter; and carrying out format conversion on the file in the sandbox format to obtain a mirror image file in a mirror image SIF format.
Taking creating a singleness container as an example, a file in a Sandbox format is generated first, and then format conversion is performed on the file in the Sandbox format, so that a mirror image file in a mirror image SIF format can be obtained. For more description of the conversion of a file in the Sandbox format into an image file in the image SIF format, refer to the related art.
After the MPI application program is subjected to containerization deployment and enters a stage of a production environment, an image file obtained in a development environment is issued to the production environment, a container of the MPI application program is created in the production environment by using the image file, and the production environment is configured according to target operation parameters in the image file, so that the production environment has the target operation parameters. The production environment can reproduce the development environment, the running performance of the MPI application program in the production environment is basically consistent with the running performance of the MPI application program in the development environment, and the workload of performance optimization personnel is greatly reduced.
According to the technical scheme provided by the embodiment of the application, containerized deployment is performed on the MPI application program, and in the containerized deployment process, the optimized target operation parameters of the development environment are reserved in the image file, so that the production environment can automatically reproduce the target operation parameters of the development environment, the reproducibility and the transportability of the target operation parameters in the development environment are enhanced, the operation environment adaptation work is greatly simplified, and the time cost and the energy of performance optimization personnel for operation environment adaptation are reduced; furthermore, the operation performance of the MPI application program in the production environment is better ensured through the containerization of the MPI application program.
Fig. 3 is a flowchart of another containerization deployment method according to an embodiment of the present disclosure. The method may be performed by a containerization deployment apparatus, which may be implemented in software and/or hardware, and may generally be integrated into an electronic device. Referring to fig. 3, the method includes:
301. aiming at the development environment stage, when the executable file of the message transfer interface MPI application program is debugged, the operation parameters of the development environment are optimized to obtain the target operation parameters.
302. And creating an image file of the MPI application program according to at least the executable file and the target operation parameters.
303. And aiming at the production environment stage, creating a container of the MPI application program in the production environment according to the image file, and configuring the production environment to have target operation parameters.
304. And periodically acquiring online operation data of the container, and evaluating the operation stability of the container in each period according to the online operation data in each period.
305. And if the operation stability of the continuous multiple periods does not meet the preset stability condition, continuously optimizing the target operation parameters of the production environment until the operation stability of the subsequent continuous multiple periods meets the preset stability condition.
For implementation manners of steps 301, 302, and 303, reference may be made to implementation manners of steps 201, 202, and 203 in the foregoing embodiments, which are not described herein again.
In practical application, after the container corresponding to the MPI application program is deployed to the production environment in a containerization deployment manner, although the production environment reproduces the target operation parameters of the development environment, the probability of bug (fault) occurring during online operation of the container corresponding to the MPI application program is greatly reduced. However, the development environment is different from the real production environment, and an exception may occur during the online operation of the container corresponding to the MPI application, where the exception includes, but is not limited to: crash (flash back), no response, slow response, etc. Therefore, in order to better ensure the running performance of the MPI application program in the production environment, after the container corresponding to the MPI application program is deployed in the production environment, online running data of the container is periodically acquired, and the running stability of the container in each period is evaluated according to the online running data in the period. And if the operation stability of the continuous multiple periods does not meet the preset stability condition, continuously optimizing the target operation parameters of the production environment until the operation stability of the subsequent continuous multiple periods meets the preset stability condition.
The number of consecutive periods is not limited, and for example, the number of consecutive periods is 5 consecutive periods, or 10 consecutive periods.
The operation stability can be reflected by various considerations such as the number of crashes (flash backs), the number of non-responses, and the response time.
The preset stability condition is flexibly set according to various factors such as the number of times of collapse (flash back), the number of times of no response, the response time and the like. For example, the number of times of collapse is less than the corresponding specified number of times, the number of times of no response is less than the corresponding specified number of times, or the response duration is less than the corresponding duration, it is determined that the preset stability condition is satisfied. Otherwise, the number of times of collapse is more than the corresponding specified number of times, the number of times of no response is more than the corresponding specified number of times or the response time length is more than the corresponding time length, and the condition that the preset stability condition is not met is confirmed.
In this embodiment, the types of faults occurring in a plurality of consecutive cycles may be the same or different, and the different types of faults may have different effects on the operation of the container. Further optionally, in order to improve operability of optimization work and timely and effectively ensure operation performance of the MPI application program in the production environment, before target operation parameters of the production environment are optimized, fault analysis may be performed based on online operation data of each period in a plurality of continuous periods to locate fault types of the container in the period; determining an optimization target according to the fault type of the container in each period of a plurality of continuous periods; and continuing to optimize the target operation parameters of the production environment according to the optimization target.
The fault category includes, but is not limited to: collapse, no response, and slow response. In practical application, whether the period has a fault or not and the fault type when the fault occurs are recorded in the online operation data of each period, and for the condition, when the fault is analyzed, whether the period has the fault or not and the fault type can be known by reading the online operation data. In practical application, online operation data can be mined by a big data mining technology so as to know whether a fault occurs in the period and the fault type when the fault occurs. The present embodiment does not limit the manner of failure analysis.
The embodiment of the application does not limit the determination mode of the optimization target. In an alternative implementation, the occurrence frequency of each fault type is determined according to the fault type of each period in a plurality of continuous periods of the container, and the fault type with the largest occurrence frequency is taken as an optimization target.
In another alternative implementation, the respective safety risks of the fault types of the container in each of the consecutive multiple cycles are determined, and the fault type with the highest safety risk is taken as the optimization target. For example, a security risk of a crash is greater than a security risk of no response, and a security risk of no response is greater than a security risk of slow response,
further optionally, after the duration of continuing to optimize the target operation parameter of the production environment reaches the preset duration, if the operation stability of the continuous multiple periods does not meet the preset stability condition, prompt information is output, and the prompt information is used for prompting performance optimizers to participate in the optimization task.
The preset duration is flexibly set according to the application scene.
In this embodiment, the running parameters of the production environment can not be optimized automatically for a long time, so that the MPI application program can still run stably, and at this time, a manual intervention mode can be introduced to prompt performance optimization personnel to participate in the optimization task, thereby ensuring the safety redundancy of the optimization task.
According to the technical scheme provided by the embodiment of the application, the MPI application program is subjected to containerized deployment, and in the containerized deployment process, the optimized target operation parameters of the development environment are reserved in the image file, so that the production environment can automatically reproduce the target operation parameters of the development environment, the reproducibility and the transportability of the target operation parameters in the development environment are enhanced, the operation environment adaptation work is greatly simplified, and the time cost and the energy of performance optimization personnel for the operation environment adaptation are reduced; furthermore, the operation performance of the MPI application program in the production environment is better ensured through the containerization of the MPI application program. Furthermore, after the MPI application program is deployed to the production environment in a container mode, the target operation parameters of the production environment are continuously optimized according to the online operation data of the container in each period, and the probability of bug (fault) occurring in online operation of the container corresponding to the MPI application program is greatly reduced.
Fig. 4 is a schematic structural diagram of a containerization deployment device according to an embodiment of the present application. Referring to fig. 4, the apparatus may include:
the debugging module 41 is configured to, in a development environment stage, optimize an operation parameter of the development environment when performing program debugging on an executable file of an MPI application program of a message passing interface, to obtain a target operation parameter;
a creating module 42, configured to create an image file of the MPI application according to at least the executable file and the target operation parameter;
and the deployment module 43 is configured to create a container of the MPI application in the production environment according to the image file for the production environment stage, and configure the production environment to have the target operating parameter.
Further optionally, when the creating module 42 creates the image file of the MPI application according to at least the executable file and the target operating parameter, it is specifically configured to: generating a definition file for guiding to create an image file at least according to the executable file and a dependency library thereof, the MPI library and version information thereof, a driving program of a communication network and a target operation parameter; an image file is created from the definition file.
Further optionally, when the creating module 42 creates the image file of the MPI application according to at least the executable file and the target operating parameter, it is specifically configured to: generating a file in a Sandbox Sandbox format at least according to the executable file and a dependent library thereof, the MPI library and version information thereof, a driving program of a communication network and a target operation parameter; and carrying out format conversion on the file in the sandbox format to obtain a mirror image file in a mirror image SIF format.
Further optionally, the target operation parameter includes a target communication algorithm, and accordingly, the debugging module 41 optimizes the operation parameter of the development environment, and when the target operation parameter is obtained, the method is specifically configured to: the target communication algorithm is selected from a plurality of communication algorithms included in the MPI library according to at least one of a load of the MPI application, a computing task, or a communication mode.
Further optionally, the apparatus further comprises: the acquisition module is used for periodically acquiring online operation data of the container and evaluating the operation stability of the container in each period according to the online operation data in the period; and the optimization module is used for continuously optimizing the target operation parameters of the production environment if the operation stability of the continuous multiple periods does not meet the preset stability condition until the operation stability of the subsequent continuous multiple periods meets the preset stability condition.
Further optionally, before the optimizing module continues to optimize the target operating parameter of the production environment, the optimizing module is further configured to: performing fault analysis based on the online operation data of each period in a plurality of continuous periods to locate the fault category of the container in the period; determining an optimization target according to the fault type of the container in each period of a plurality of continuous periods; and continuing to optimize the target operation parameters of the production environment according to the optimization target.
Further optionally, when the optimization module determines the optimization target according to the fault type of the container in each of the multiple continuous cycles, the optimization module is specifically configured to: determining the occurrence frequency of each fault type according to the fault type of each period of the container in a plurality of continuous periods, and taking the fault type with the maximum occurrence frequency as an optimization target; or determining the safety risk of each fault type of the container in each continuous multiple period, and taking the fault type with the maximum safety risk as an optimization target.
Further optionally, the optimization module is further configured to: and after the time length for continuously optimizing the target operation parameters of the production environment reaches the preset time length and the operation stability of the continuous multiple periods does not meet the preset stability condition, outputting prompt information, wherein the prompt information is used for prompting performance optimization personnel to participate in the optimization task.
Further optionally, the container is a Singularity container.
The specific manner in which each module of the containerization deployment apparatus shown in fig. 4 performs operations may be referred to in the description of the method embodiment, and is not described herein again.
It should be noted that the execution subjects of the steps of the methods provided in the above embodiments may be the same device, or different devices may be used as the execution subjects of the methods. For example, the execution subjects of step 201 to step 203 may be device a; for another example, the execution subject of steps 201 and 202 may be device a, and the execution subject of step 203 may be device B; and so on.
In addition, in some of the flows described in the above embodiments and the drawings, a plurality of operations are included in a specific order, but it should be clearly understood that the operations may be executed out of the order presented herein or in parallel, and the sequence numbers of the operations, such as 201, 202, etc., are merely used for distinguishing different operations, and the sequence numbers do not represent any execution order per se. Additionally, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first", "second", etc. in this document are used for distinguishing different messages, devices, modules, etc., and do not represent a sequential order, nor limit the types of "first" and "second" to be different.
Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 5, the electronic apparatus includes: a memory 51 and a processor 52;
the memory 51 is used to store computer programs and may be configured to store other various data to support operations on the computing platform. Examples of such data include instructions for any application or method operating on the computing platform, contact data, phonebook data, messages, pictures, videos, and so forth.
The memory 51 may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.
A processor 52 coupled to the memory 51 for executing the computer program in the memory 51 for: aiming at a development environment stage, optimizing the operation parameters of the development environment when debugging the executable file of the message passing interface MPI application program to obtain target operation parameters; creating an image file of the MPI application program at least according to the executable file and the target operation parameters; and aiming at the production environment stage, creating a container of the MPI application program in the production environment according to the image file, and configuring the production environment to have target operation parameters.
Further optionally, when the processor 52 creates the image file of the MPI application according to at least the executable file and the target operation parameter, it is specifically configured to: generating a definition file for guiding to create an image file at least according to the executable file and a dependency library thereof, the MPI library and version information thereof, a driving program of a communication network and a target operation parameter; an image file is created from the definition file.
Further optionally, when the processor 52 creates the image file of the MPI application according to at least the executable file and the target operation parameter, it is specifically configured to: generating a file in a Sandbox Sandbox format at least according to the executable file and a dependent library thereof, the MPI library and version information thereof, a driving program of a communication network and a target operation parameter; and carrying out format conversion on the file in the sandbox format to obtain a mirror image file in a mirror image SIF format.
Further optionally, the target operation parameter includes a target communication algorithm, and accordingly, the processor 52 optimizes the operation parameter of the development environment, and when the target operation parameter is obtained, the method is specifically configured to: the target communication algorithm is selected from a plurality of communication algorithms included in the MPI library according to at least one of a load of the MPI application, a computing task, or a communication mode.
Further optionally, the processor 52 is further configured to periodically obtain online operation data of the container, and evaluate the operation stability of the container in each period according to the online operation data in the period; and the optimization module is used for continuously optimizing the target operation parameters of the production environment if the operation stability of the continuous multiple periods does not meet the preset stability condition until the operation stability of the subsequent continuous multiple periods meets the preset stability condition.
Further optionally, before the processor 52 continues to optimize the target operating parameter of the production environment, it is further configured to: performing fault analysis based on the online operation data of each period in a plurality of continuous periods to locate the fault category of the container in the period; determining an optimization target according to the fault type of the container in each period of a plurality of continuous periods; and continuing to optimize the target operation parameters of the production environment according to the optimization target.
Further optionally, when determining the optimization objective according to the fault category of the container in each of the multiple consecutive cycles, the processor 52 is specifically configured to: determining the occurrence frequency of each fault type according to the fault type of each period of the container in a plurality of continuous periods, and taking the fault type with the maximum occurrence frequency as an optimization target; or determining the safety risk of each fault type of the container in each continuous multiple period, and taking the fault type with the maximum safety risk as an optimization target.
Further optionally, the processor 52 is further configured to: and after the time length for continuously optimizing the target operation parameters of the production environment reaches the preset time length and the operation stability of the continuous multiple periods does not meet the preset stability condition, outputting prompt information, wherein the prompt information is used for prompting performance optimization personnel to participate in the optimization task.
Further optionally, the container is a Singularity container.
Further, as shown in fig. 5, the electronic device further includes: communication components 53, display 54, power components 55, audio components 56, and the like. Only some of the components are schematically shown in fig. 5, and it is not meant that the electronic device comprises only the components shown in fig. 5. In addition, the components within the dashed line frame in fig. 5 are optional components, not necessary components, and may be determined according to the product form of the electronic device. The electronic device of this embodiment may be implemented as a terminal device such as a desktop computer, a notebook computer, a smart phone, or an IOT device, or may be a server device such as a conventional server, a cloud server, or a server array. If the electronic device of this embodiment is implemented as a terminal device such as a desktop computer, a notebook computer, a smart phone, etc., the electronic device may include components within a dashed line frame in fig. 5; if the electronic device of this embodiment is implemented as a server device such as a conventional server, a cloud server, or a server array, the components in the dashed box in fig. 5 may not be included.
For details of the implementation process of each action performed by the processor, reference may be made to the foregoing method embodiment or the related description in the device embodiment, and details are not described herein again.
Accordingly, the present application further provides a computer-readable storage medium storing a computer program, where the computer program is capable of implementing the steps that can be executed by the electronic device in the foregoing method embodiments when executed.
Accordingly, embodiments of the present application also provide a computer program product, which includes a computer program/instruction, when the computer program/instruction is executed by a processor, the processor is enabled to implement the steps that can be executed by an electronic device in the foregoing method embodiments.
The communication component is configured to facilitate wired or wireless communication between the device in which the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as a WiFi, a 2G, 3G, 4G/LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
The display includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.
The power supply assembly provides power for various components of the device in which the power supply assembly is located. The power components may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device in which the power component is located.
The audio component may be configured to output and/or input an audio signal. For example, the audio component includes a Microphone (MIC) configured to receive an external audio signal when the device in which the audio component is located is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal may further be stored in a memory or transmitted via a communication component. In some embodiments, the audio assembly further comprises a speaker for outputting audio signals.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both permanent and non-permanent, removable and non-removable media, may implement the information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.
The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (11)
1. A containerized deployment method, comprising:
aiming at a development environment stage, optimizing operation parameters of the development environment when an executable file of an MPI application program is debugged to obtain target operation parameters;
creating an image file of the MPI application program according to the executable file and the target operation parameters;
and aiming at the stage of the production environment, creating a container of the MPI application program in the production environment according to the image file, and configuring the production environment to have the target operation parameters.
2. The method of claim 1, wherein creating an image file of the MPI application from at least the executable file and the target operating parameters comprises:
generating a definition file for guiding the creation of the image file at least according to the executable file and a dependency library thereof, an MPI library and version information thereof, a driving program of a communication network and the target operation parameters;
and creating the mirror image file according to the definition file.
3. The method of claim 1, wherein creating an image file of the MPI application from at least the executable file and the target operating parameters comprises:
generating a file in a Sandbox Sandbox format at least according to the executable file and the dependency library thereof, the MPI library and version information thereof, a driving program of a communication network and the target operation parameters;
and carrying out format conversion on the file in the sandbox format to obtain the mirror image file in the mirror image SIF format.
4. The method of claim 1, wherein the target operating parameters comprise a target communication algorithm, and wherein optimizing the operating parameters of the development environment to obtain target operating parameters comprises:
selecting the target communication algorithm from a plurality of communication algorithms included in the MPI library according to at least one of a load, a computing task, or a communication mode of the MPI application.
5. The method of claim 1, further comprising, after configuring the production environment with the target operating parameters:
periodically acquiring online operation data of the container, and evaluating the operation stability of the container in each period according to the online operation data in the period;
and if the operation stability of the continuous multiple periods does not meet the preset stability condition, continuously optimizing the target operation parameters of the production environment until the operation stability of the subsequent continuous multiple periods meets the preset stability condition.
6. The method of claim 5, further comprising, prior to continuing to optimize the target operating parameter of the production environment:
performing fault analysis based on-line operation data of each period in a plurality of continuous periods to locate the fault type of the container in the period;
determining an optimization target according to the fault type of the container in each period of a plurality of continuous periods;
and continuously optimizing the target operation parameters of the production environment according to the optimization target.
7. The method of claim 6, wherein determining an optimization objective based on the type of fault of the vessel in each of the plurality of successive cycles comprises:
determining the occurrence frequency of each fault type according to the fault type of the container in each period of a plurality of continuous periods, and taking the fault type with the maximum occurrence frequency as an optimization target; or,
and determining the safety risk of each fault type of the container in each period of a plurality of continuous periods, and taking the fault type with the maximum safety risk as an optimization target.
8. The method of claim 5, further comprising:
and after the time length for continuously optimizing the target operation parameters of the production environment reaches the preset time length, if the operation stability of the target operation parameters for a plurality of continuous periods does not meet the preset stability condition, outputting prompt information, wherein the prompt information is used for prompting performance optimization personnel to participate in the optimization task.
9. The method according to any one of claims 1 to 8, wherein the container is a Singularity container.
10. An electronic device, comprising: a memory and a processor; the memory for storing a computer program; the processor is coupled to the memory for executing the computer program for performing the steps in the method of any of claims 1-9.
11. A computer storage medium having a computer program stored thereon, which, when executed by a processor, causes the processor to carry out the steps of the method of any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210614674.5A CN114996134A (en) | 2022-05-30 | 2022-05-30 | Containerized deployment method, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210614674.5A CN114996134A (en) | 2022-05-30 | 2022-05-30 | Containerized deployment method, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114996134A true CN114996134A (en) | 2022-09-02 |
Family
ID=83030494
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210614674.5A Pending CN114996134A (en) | 2022-05-30 | 2022-05-30 | Containerized deployment method, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114996134A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116107714A (en) * | 2023-02-02 | 2023-05-12 | 北京天云融创软件技术有限公司 | Method and device for running MPI task |
| CN117972670A (en) * | 2024-03-28 | 2024-05-03 | 北京大学 | Cloud container mirror image building method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109683909A (en) * | 2018-11-30 | 2019-04-26 | 昆明理工大学 | A kind of MPI parallel environment automatic deployment method based on Docker |
| US20190349305A1 (en) * | 2018-05-11 | 2019-11-14 | Huazhong University Of Science And Technology | Container communication method and system for parallel applications |
| CN111221541A (en) * | 2019-12-26 | 2020-06-02 | 曙光信息产业(北京)有限公司 | Cluster parallel program deployment method and device |
| CN111831396A (en) * | 2020-07-10 | 2020-10-27 | 融慧金科金融服务外包（北京）有限公司 | Docker software and hardware integration-based delivery method and device |
-
2022
- 2022-05-30 CN CN202210614674.5A patent/CN114996134A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190349305A1 (en) * | 2018-05-11 | 2019-11-14 | Huazhong University Of Science And Technology | Container communication method and system for parallel applications |
| CN109683909A (en) * | 2018-11-30 | 2019-04-26 | 昆明理工大学 | A kind of MPI parallel environment automatic deployment method based on Docker |
| CN111221541A (en) * | 2019-12-26 | 2020-06-02 | 曙光信息产业(北京)有限公司 | Cluster parallel program deployment method and device |
| CN111831396A (en) * | 2020-07-10 | 2020-10-27 | 融慧金科金融服务外包（北京）有限公司 | Docker software and hardware integration-based delivery method and device |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116107714A (en) * | 2023-02-02 | 2023-05-12 | 北京天云融创软件技术有限公司 | Method and device for running MPI task |
| CN117972670A (en) * | 2024-03-28 | 2024-05-03 | 北京大学 | Cloud container mirror image building method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106980625B (en) | Data synchronization method, device and system | |
| CN110413346B (en) | Parameter updating method and device | |
| US9083608B2 (en) | Automatically selecting appropriate platform to run application in cloud computing environment | |
| US11379943B2 (en) | Optimizing compilation of shaders | |
| CN114996134A (en) | Containerized deployment method, electronic equipment and storage medium | |
| CN107645483B (en) | Risk identification method, risk identification device, cloud risk identification device and system | |
| CN111831396A (en) | Docker software and hardware integration-based delivery method and device | |
| CN111611145B (en) | Breakdown information collection method and device, storage medium and electronic equipment | |
| CN112395098B (en) | Application program interface calling method and device, storage medium and electronic equipment | |
| CN104113576A (en) | Method and device for updating client | |
| CN111435943B (en) | Data processing method, device, system and storage medium | |
| CN109522198A (en) | Processing method, device, electronic equipment and the readable storage medium storing program for executing of application program | |
| CN105630683A (en) | Cloud testing architecture | |
| CN114385164A (en) | Page generation and rendering method and device, electronic equipment and storage medium | |
| CN112346965A (en) | Test case distribution method, device and storage medium | |
| CN117724852B (en) | Cloud computer computing resource allocation method and device | |
| CN109558143A (en) | The method and device of application deployment in a kind of cluster | |
| CN114840214A (en) | Program compiling and starting method, device and storage medium | |
| CN113448690A (en) | Monitoring method and device | |
| CN112346761B (en) | Front-end resource online method, device, system and storage medium | |
| CN112114804A (en) | Application program generation method, device and system | |
| WO2023179183A1 (en) | Subprogram synchronization processing | |
| CN113138717B (en) | Node deployment method, device and storage medium | |
| WO2020135129A1 (en) | Method and device for loading plug-in of application, and terminal | |
| CN115580667A (en) | Data transmission method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |