CN113434194A - Continuous integration and delivery system, method, electronic device and storage medium - Google Patents

Continuous integration and delivery system, method, electronic device and storage medium Download PDF

Info

Publication number
CN113434194A
CN113434194A CN202110529679.3A CN202110529679A CN113434194A CN 113434194 A CN113434194 A CN 113434194A CN 202110529679 A CN202110529679 A CN 202110529679A CN 113434194 A CN113434194 A CN 113434194A
Authority
CN
China
Prior art keywords
project
submodule
git
sub
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110529679.3A
Other languages
Chinese (zh)
Inventor
任红强
唐阳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan Kuangshi Jinzhi Technology Co ltd
Beijing Megvii Technology Co Ltd
Original Assignee
Wuhan Kuangshi Jinzhi Technology Co ltd
Beijing Megvii Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan Kuangshi Jinzhi Technology Co ltd, Beijing Megvii Technology Co Ltd filed Critical Wuhan Kuangshi Jinzhi Technology Co ltd
Priority to CN202110529679.3A priority Critical patent/CN113434194A/en
Publication of CN113434194A publication Critical patent/CN113434194A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/70—Software maintenance or management
    • G06F8/73—Program documentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/40—Transformation of program code
    • G06F8/41—Compilation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/70—Software maintenance or management
    • G06F8/75—Structural analysis for program understanding
    • G06F8/751—Code clone detection

Abstract

The invention discloses a continuous integration and delivery system, a method, an electronic device and a storage medium, comprising: the system comprises a management background service module and an actuator service module, wherein the management background service module comprises an engineering management and engineering release submodule, and the actuator service module comprises a code cloning, compiling, mirror image constructing and aggregation packaging submodule; the project management submodule receives project management operation input by a user; the code cloning submodule clones the code of the git project when the project management operation is used for newly adding or editing the git project; compiling the code by the compiling submodule to obtain a compiled file; the mirror image construction sub-module carries out mirror image construction on the compiled file to obtain a mirror image file; the project release submodule receives release operation input by a user; and the aggregation and packaging submodule responds to the issuing operation, aggregates and packages the image files corresponding to all git projects of the target project into a tar package, generates a delivery compression package based on the tar package and sends the delivery compression package to a remote platform for other users to download.

Description

Continuous integration and delivery system, method, electronic device and storage medium
Technical Field
The invention relates to the technical field of software development, in particular to a continuous integration and delivery system, a continuous integration and delivery method, electronic equipment and a storage medium.
Background
With the development of micro service architecture and container virtualization technology, more and more companies are beginning to use continuous integration and development systems to solve quality problems caused by frequent software release. Currently, existing continuous integration and delivery systems include: a CI (Continuous Integration)/CD (Continuous payment) system based on gitlab, an open-source jenkins platform providing a friendly operation interface, a sustainable Integration and Delivery platform expanding based on jenkins kernel, a sustainable Integration and Delivery platform customized by the enterprise, and the like.
However, when the existing sustainable integration and delivery systems are applied to special scenarios, for example, scenarios such as compiling, mirror image building, offline hybrid deployment tar package distribution, and the like, based on multiple git projects, management of multiple git projects of different language types are low in development efficiency. Taking a CI/CD system based on gitlab as an example, the CI system integrated on gitlab needs each access party to write customized git-ci.yml when accessing a git project, and when implementing the construction of complex sub-modules, complex scripts need to be written in yml, resulting in lower development efficiency.
Disclosure of Invention
Embodiments of the present invention provide a system and a method for sustainable integration and delivery, an electronic device, and a storage medium, so as to solve a technical problem in the prior art that a sustainable integration and delivery system has a low development efficiency.
According to a first aspect of the present invention, there is disclosed a continuous integration and delivery system, the system comprising: the system comprises a management background service module and an actuator service module, wherein the management background service module comprises: the engineering management submodule and the engineering release submodule, wherein the executor service module comprises: the code cloning submodule, the compiling submodule, the mirror image constructing submodule and the aggregation packing submodule;
the project management submodule is used for receiving project management operation input by a user, wherein the project management operation is used for adding, deleting, editing or searching git projects of a target project, and the target project corresponds to a plurality of git projects;
the code cloning submodule is used for cloning the codes of the git project from a code hosting platform under the condition that the project management operation is used for newly adding or editing the git project and providing the codes of the git project to the compiling submodule;
the compiling submodule is used for compiling the code of the git project to obtain a compiled file and providing the compiled file to the mirror image construction submodule;
the image construction submodule is used for carrying out image construction on the compiled file to obtain an image file and providing the image file to the aggregation and packaging submodule;
the project publishing submodule is used for receiving a publishing operation input by a user, wherein the publishing operation is used for indicating the aggregation and packaging submodule to generate a delivery compression package corresponding to the target project based on the image file and sending the delivery compression package to a remote platform for other users to download;
and the aggregation and packaging submodule is used for responding to the issuing operation, aggregating and packaging the image files corresponding to all git projects of the target project into a tar package, generating the delivery compression package based on the tar package, and sending the delivery compression package to the remote platform.
Optionally, as an embodiment, the system operates in a docker environment of a host;
the image construction sub-module is specifically configured to acquire a dockerfile corresponding to the git project from a preset database, and construct an image file corresponding to the compiled file based on the dockerfile.
Optionally, as an embodiment, the git project includes a plurality of project sub-modules, each project sub-module corresponds to a dockerfile, each dockerfile records a name value of a label attribute, the label attribute is used to control a generated mirror name, and the name values of the label attributes recorded in different dockerfiles are different;
the compiling submodule is specifically used for compiling the codes of each project submodule to obtain a compiling file corresponding to each project submodule and providing the compiling file to the mirror image constructing submodule;
the mirror image construction sub-module is specifically configured to recursively find out compiled files corresponding to each project sub-module, transfer the compiled files to a root directory, acquire dockerfile files corresponding to each project sub-module from a preset database, and perform mirror image construction on the compiled files corresponding to each project sub-module based on the dockerfile files corresponding to each project sub-module to obtain mirror image files corresponding to each project sub-module;
and the name of the mirror image file corresponding to each project sub-module is the name value of the label attribute in the dockerfile corresponding to the project sub-module.
Optionally, as an embodiment, a plurality of instances are deployed in at least one sub-module of the executor service module, and each instance includes a plurality of threads.
Optionally, as an embodiment, the engineering management submodule is further configured to:
after the project management operation is received, the git project configuration information input by a user is received, and the git project configuration information is sent to a preset database to be stored;
wherein the git project configuration information comprises: the name of the git project, the source code address of the git project, the compiling language to which the git project belongs, the branch of the git project and the access path of the dockerfile file.
Optionally, as an embodiment, the management background service module further includes: constructing an information display submodule; wherein the content of the first and second substances,
and the construction information display submodule is used for displaying the construction record and the construction state information of the git project in the target project.
According to a second aspect of the present invention, there is disclosed a continuous integration and delivery method based on the continuous integration and delivery system of the first aspect, the method comprising:
the project management sub-module receives project management operation input by a user, wherein the project management operation is used for adding, deleting, editing or searching git projects of a target project, and the target project corresponds to a plurality of git projects;
under the condition that the project management operation is used for newly adding or editing the git project, the code cloning submodule clones the code of the git project from the code hosting platform and provides the code to the compiling submodule;
compiling the code of the git project by the compiling submodule to obtain a compiled file, and providing the compiled file to the mirror image constructing submodule;
the mirror image construction submodule carries out mirror image construction on the compiled file to obtain a mirror image file, and the mirror image file is provided for the aggregation packaging submodule;
the project release submodule receives a release operation input by a user, wherein the release operation is used for indicating the aggregation and packaging submodule to generate a delivery compression package corresponding to the target project based on the image file and sending the delivery compression package to a remote platform for other users to download;
and the aggregation and packaging submodule responds to the issuing operation, aggregates and packages the image files corresponding to all git projects of the target project into a tar package, generates the delivery compression package based on the tar package, and sends the delivery compression package to the remote platform.
Optionally, as an embodiment, the system operates in a docker environment of a host;
and the mirror image construction sub-module acquires a dockerfile corresponding to the git project from a preset database, and constructs a mirror image file corresponding to the compiled file based on the dockerfile.
Optionally, as an embodiment, the git project includes a plurality of project sub-modules, each project sub-module corresponds to a dockerfile, each dockerfile records a name value of a label attribute, the label attribute is used to control a generated mirror name, and the name values of the label attributes recorded in different dockerfiles are different;
the compiling submodule compiles the codes of each project submodule to obtain a compiling file corresponding to each project submodule and provides the compiling file to the mirror image construction submodule;
the image construction sub-module recursively searches compiled files corresponding to the project sub-modules, transfers the compiled files to a root directory, acquires dockerfile files corresponding to the project sub-modules from a preset database, and performs image construction on the compiled files corresponding to the project sub-modules based on the dockerfile files corresponding to the project sub-modules to obtain image files corresponding to the project sub-modules;
and the name of the mirror image file corresponding to each project sub-module is the name value of the label attribute in the dockerfile corresponding to the project sub-module.
Optionally, as an embodiment, a plurality of instances are deployed in at least one sub-module of the executor service module, and each instance includes a plurality of threads.
Optionally, as an embodiment, after receiving the project management operation, the project management sub-module receives git project configuration information input by a user, and sends the git project configuration information to a preset database for storage;
wherein the git project configuration information comprises: the name of the git project, the source code address of the git project, the compiling language to which the git project belongs, the branch of the git project and the access path of the dockerfile file.
Optionally, as an embodiment, the management background service module further includes: constructing an information display submodule; the method further comprises the following steps:
and the construction information display submodule displays the construction record and the construction state information of the git project in the target project.
According to a third aspect of the present invention, there is disclosed an electronic device comprising: a memory, a processor and a program stored on the memory and executable on the processor, which program, when executed by the processor, performs the steps of the continuous integration and delivery method as in the first aspect.
According to a fourth aspect of the present invention, a computer readable storage medium is disclosed, having a program stored thereon, which when executed by the processor, performs the steps of the method of continuous integration and delivery as in the first aspect.
In the embodiment of the invention, a management background service module in the continuous integration and delivery system can provide services of increasing, deleting, updating and viewing git projects used by a user side, an executor service module can provide services of code cloning, compiling, mirror image construction and mixed tar package delivery, and joint management of a plurality of git projects of a project can be realized through the mutual cooperation of the management background service module and the executor service module. Compared with the prior art, the embodiment of the invention can fully abstract the service scene, rely on the core logic of code integration, construction and delivery to the system according to the principle of simplicity and high efficiency, does not need to compile any complex script file, and has simple operation and higher development efficiency. In addition, the method can also support the construction of multiple different programming language git projects and meet the requirements of developers with different programming language types.
Drawings
FIG. 1 is a block diagram of a continuous integration and delivery system of one embodiment of the present invention;
FIG. 2 is an exemplary diagram of a user-oriented front-end operational interface of the continuous integration and delivery system of one embodiment of the present invention;
FIG. 3 is a block diagram of a continuous integration and delivery system of another embodiment of the present invention;
FIG. 4 is an exemplary diagram of a mount-type implementation of docker in docker of one embodiment of the invention;
FIG. 5 is an exemplary diagram of a multi-project sub-module image build flow, according to one embodiment of the invention;
FIG. 6 is an exemplary diagram of a persistent integration and delivery scenario of one embodiment of the present invention;
FIG. 7 is a flow chart of a method of continuous integration and delivery of an embodiment of the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.
At present, the existing sustainable integration and delivery systems include a gitlab-based CI/CD system, an open-source jenkins platform providing a friendly operation interface, various sustainable integration delivery platforms expanding based on jenkins kernel, a enterprise customized sustainable integration delivery platform, and the like, and when the sustainable integration delivery systems listed above are applied to special scenes, such as scenes based on multi-git project management, compiling and mirror image construction of git projects of various different language types, and offline hybrid deployment tar package release, the development efficiency is low, and actual development requirements cannot be met practically.
To solve the foregoing technical problems, embodiments of the present invention provide a continuous integration and delivery system, a method, an electronic device, and a storage medium.
For ease of understanding, some concepts involved in embodiments of the present invention will be described first below.
Compilation refers to the process of processing source code into machine code that a machine can recognize as running.
The mirror image construction refers to a process of processing a source program package and a set operation environment into a mirror image, wherein the mirror image is a lightweight executable independent software package and is used for packaging a software operation environment and software developed based on the operation environment, and the mirror image comprises all contents of certain software, including: code, runtime, libraries, environment variables, and configuration files.
The release of the offline hybrid deployment tar package refers to a process of compressing and sending the service image package and related service files which are independent of the environment to a designated platform.
docker is an open source application container engine that allows developers to package their applications and dependencies into a portable image and then distribute it to any popular Linux or Windows machine, as well as to implement virtualization. The containers are fully sandboxed without any interface between each other.
dockerfile is a text file used to construct an image, and the text content contains a piece of instructions and descriptions required to construct the image.
A continuous integration and delivery system provided by an embodiment of the present invention is described next.
It should be noted that the continuous integration and delivery system provided by the embodiment of the present invention is suitable for an electronic device, and in practical applications, the electronic device may be a server, or may also be other types of devices, which is not limited in the embodiment of the present invention.
FIG. 1 is a block diagram of a continuous integration and delivery system according to an embodiment of the present invention, and as shown in FIG. 1, the continuous integration and delivery system 100 may include: the management background service module 110 and the executor service module 120, where the management background service module 110 may include: the project management submodule 111, the project release submodule 112, and the executor service module 120 may include: a code cloning sub-module 121, a compiling sub-module 122, a mirror image constructing sub-module 123 and an aggregation packing sub-module 124;
the project management submodule 111 is configured to receive a project management operation input by a user, where the project management operation is used to add, delete, edit, or search a git project of a target project, and the target project corresponds to multiple git projects;
the code cloning sub-module 121 is used for cloning the codes of the git project from the code hosting platform and providing the codes to the compiling sub-module 122 under the condition that the project management operation is used for newly adding or editing the git project;
the compiling submodule 122 is configured to compile a code of the git project to obtain a compiled file, and provide the compiled file to the mirror image constructing submodule 123;
the mirror image construction sub-module 123 is configured to perform mirror image construction on the compiled file to obtain a mirror image file, and provide the mirror image file to the aggregation packaging sub-module 124;
the project publishing sub-module 112 is configured to receive a publishing operation input by a user, where the publishing operation is used to instruct the aggregation and packaging sub-module 124 to generate a delivery compression package corresponding to a target project based on the image file, and send the delivery compression package to a remote platform for downloading by other users;
and the aggregation and packaging submodule 124 is configured to aggregate and package the image files corresponding to all git projects of the target project into a tar package in response to the publishing operation, generate a delivery compression package based on the tar package, and send the delivery compression package to the remote platform.
In the embodiment of the invention, the management background service module is a platform used for the user side, and the user finishes various operations and final integral delivery of the project through the platform.
In the embodiment of the invention, the executor service module mainly works to execute corresponding operations of code cloning, compiling, mirror image construction, mirror image release and the like after receiving the pipeline task message from the management background service module.
In the embodiment of the invention, the project management submodule is mainly responsible for joint management of git projects, and comprises the following steps: and adding, deleting, editing and searching corresponding git projects, wherein the added and edited git projects are mainly embodied as basic metadata information (including git project names, git project branches, and compiling languages such as java, go, nodes and the like) of the configured git projects, and the subsequent actuator service module can execute code cloning, compiling, mirror image construction, mirror image release and other operations according to the metadata information.
In the embodiment of the present invention, the project management sub-module may provide an operation interface for the user to use, and the user inputs the configuration information of the project related git project on the operation interface, specifically,
the engineering management submodule is also used for: after receiving the project management operation, receiving git project configuration information input by a user, and sending the git project configuration information to a preset database for storage;
wherein, the git project configuration information comprises: the name of the git project, the source code address of the git project, the compiling language to which the git project belongs, the branch of the git project and the access path of the dockerfile file.
In the embodiment of the invention, after a user operates a new project and a new git project on the front-end interface, the configuration information is stored in the relational database, and when the user inquires the project management list, the data is acquired from the database and is converted and displayed to the front-end interface.
In an example, as shown in fig. 2, fig. 2 shows an example diagram of a front-end operation interface used for a user side, a target project corresponds to two git projects, which are an a project and a B project, for example, a newly added B project, a user may input a project name in a column of "git project name", a source address of a project in a column of "git project source address", a programming language to which the project belongs in a column of "compilation language", and a branch to which the project belongs in a column of "git project branch", and if the system is implemented based on a docker service, an access path of a dockerfile file corresponding to the project may be input in a column of "dockerfile".
In the embodiment of the invention, the engineering release submodule is used for releasing list display, release configuration and release detail preview, and the release is to aggregate and print the manufactured mirror images (pushed to a mirror image warehouse) of the actuator service module into a tar packet, print a delivery compression packet by adding a mixture of a deployment file, an explanation document and the like, and send the delivery compression packet to a specified remote platform, so that a field project supports downloading the delivery compression packet from the remote platform, and the corresponding deployment maintenance work is completed.
In this embodiment of the present invention, in order to facilitate developers to know the construction situation of a project in time, as shown in fig. 3, the management background service module 110 may further include: and the construction information display submodule 113 is used for displaying the construction record and the construction state information of the git project in the target project, and the construction information display submodule 113 is used for displaying the construction record and the construction state information of the git project in the target project. The step of displaying the construction records refers to that after a user submits codes of corresponding branches of each git item, an executor service module is triggered to execute one construction operation on the corresponding branches of the git item, all the construction records form a construction list to be displayed, and the construction state displaying refers to displaying the construction state of the git item, such as success or failure, and displaying logs constructed by the item in real time.
In order to solve the performance problem possibly faced by the platform, improve the capability of the whole system for accessing the git project, and perform horizontal expansion according to the performance requirement, in the embodiment of the present invention, a distributed deployment scheme may be adopted, multiple instances are deployed in the executor service module in a distributed manner, specifically, multiple instances may be deployed in at least one sub-module of the executor service module, and each instance includes multiple threads.
In this case, the executor service module may distribute the task to multiple instances when receiving the task message from the management background service module, and each instance may enable the multithreading pipeline to perform operations such as code detection, code compilation, mirror construction, mirror publishing, and the like.
In order to facilitate management of application services, typically service deployment, maintenance, capacity expansion, and the like, in an embodiment provided by the present invention, the persistent integration and delivery system in the embodiment of the present invention may operate in a docker environment of a host; specifically, the service may be Docker-based, that is, the service is made into a mirror image, and then operations such as starting, deploying, deleting, and expanding the service are performed by means of a Docker service instruction or a related Docker service management platform (e.g., Docker Swam).
Correspondingly, the mirror image construction sub-module is specifically configured to obtain a dockerfile corresponding to the git project from the preset database, and construct a mirror image file corresponding to the compiled file based on the dockerfile.
In the embodiment of the invention, considering that the situation of the continuous integration and delivery system is special, the system itself is firstly used as a docker container service to operate, meanwhile, a docker service instruction is needed to operate related operations for constructing the mirror image in the docker container service (for example, a project of a specified language needs to use a docker run instruction to operate a specified compiling mirror service to compile codes, use a docker build instruction to pack the mirror image, use a docker push instruction to push the mirror image to a mirror image warehouse, and the like), which means that the docker service needs to operate in the docker service.
In order to implement the operation of the docker service in the docker service, firstly, a docker file for making an image needs to be written, then, the docker service needs to be deployed on the host (i.e., a docker program is installed), then, a base dependent service needs to be deployed on the host, and finally, the/var/run/docker.sock and/usr/bin/docker file in the host are mounted to the persistent integration and delivery system, as shown in fig. 4, the docker service of the host can be shared after mounting, and a corresponding docker instruction is executed.
Deploying a docker service: before deploying the docker service on the host, it is required to first ensure whether the current environment has the docker service or not and whether a base image in the docker file exists or not. If there is no docker service, there is an error in executing the docker instruction. If the local mirror store is not present, it will be pulled from the central store. And meanwhile, before construction, a service code needs to be compiled to obtain xxxx.
Deploying the base dependent service: no matter the management background service module or the executor service module depends on mysql service, hdfs service and ftp service, so related services need to be started first, and for the mysql service, corresponding table data needs to be initialized after the mysql service is started.
File mounting: socket and socket files are mounted in a starting command, related runtime environment variables are set, a mounting use-v parameter and an environment variable use-e are set.
In one example, the execution instruction for mounting is docker run-it-v/var/run/docker.socket:/var/run/docker.socket-v/usr/bin/docker:/usr/bin/docker;
the execution instructions for setting the environment variables are: -e MYSQL _ ADDRESS ═ xxx-ex1 ═ x2 amphetary: 5000/xxxx: 1.0.
Therefore, in the embodiment of the invention, the continuous integration and delivery system realizes a simple cloud deployment scheme (docker in docker) for operating the container in the container, and one arrangement can easily and quickly start the whole service by relying on the docker service.
At present, in the daily development process, especially for a large Java project, a git project usually includes a plurality of project sub-modules, each project sub-module can generate an executable jar package after being compiled, and the project sub-modules also need to be deployed into services, so that each project sub-module itself needs to be mirrored separately. Meanwhile, in addition to the business submodules related to the codes, the large git project sometimes needs to mirror files such as documents and initialization scripts, which means that one git project often needs to mirror a plurality of images, and the respective image names need to be meaningful and cannot be repeated.
For the situation that one git project comprises a plurality of project sub-modules, in one embodiment provided by the invention, each project sub-module can respectively correspond to one dockerfile, the name value of a label attribute is recorded in each dockerfile, the label attribute is used for controlling a generated mirror name, and the name values of the label attributes recorded in different dockerfiles are different; accordingly, the method can be used for solving the problems that,
the compiling submodule is specifically used for compiling the codes of each project submodule to obtain a compiling file corresponding to each project submodule and providing the compiling file to the mirror image construction submodule;
the mirror image construction sub-module is specifically used for recursively finding out the compiled files corresponding to the project sub-modules, transferring the compiled files to a root directory, acquiring dockerfile files corresponding to the project sub-modules from a preset database, and performing mirror image construction on the compiled files corresponding to the project sub-modules based on the dockerfile files corresponding to the project sub-modules to obtain mirror image files corresponding to the project sub-modules; and the name of the mirror image file corresponding to each project sub-module is the name value of the label attribute in the dockerfile corresponding to the project sub-module.
In the embodiment of the invention, the root directory is a docker root directory.
In an example, taking an item written by a git item as a java language as an example, the git item includes three item sub-modules, which are a sub-item module 1, a sub-item module 2, and a sub-item module 3, respectively, a dockerfile corresponding to the sub-item module 1 is written, an LABEL attribute "LABEL name ═ xxx-module-1" is added to the dockerfile, and the content of the dockerfile is exemplified as follows:
FROM alpine:2.1.0
ENV TZ Asia/Shanghai
LABEL name＝xxx-module-1
WORKDIR$PRO_PATH
ADD xxx-module-1.jar$PRO_PATH
similarly, writing a dockerfile corresponding to the sub-item module 2, and adding an LABEL attribute "LABEL name-xxx-module-2" to the dockerfile; writing a dockerfile corresponding to the sub-item module 3, and adding an LABEL attribute "LABEL name-xxx-module-3" to the dockerfile.
And then adding a dockerfile list needing mirroring on an engineering management submodule of the management background server. The executor service module pulls the code of the git project, executes compilation to obtain jar packets corresponding to each project sub-module, recursively finds out jar packets generated by each project sub-module as shown in fig. 5, and transfers the jar packets to the root directory of the docker.
The executor service module reads a Dockerfile list configured in the engineering management submodule from the mysql, analyzes each Dockerfile, obtains the name value of the label attribute, executes a docker build-t amqregistry instruction of 5000/xxx-module-1:1.0 (the obtained name value) -fxxx/Dockerfile-1 (the used current Dockerfile path) under the root directory, and each Dockerfile can add a jar package which is required by self definition under the root directory, so that the mirror image construction of the complex multi-submodule project is completed.
For ease of understanding, the delivery process of the engineering delivery package will be described with reference to the scenario diagram shown in fig. 6. Taking an engineering containing 3 git projects as an example, for convenience of description, the 3 git projects are abbreviated as A, B, C, and the core objective is to obtain a complete installation and deployment file of the engineering, namely a delivery compression package of the engineering, wherein the installation and deployment file contains a tar package which is obtained by compiling and mirroring the 3 git projects and jointly compressed together, and a related deployment manuscript.
The user inputs the configuration information of the git item A, B, C through operation on the management background service module of the continuous integration and delivery system, and the executor server module pulls the codes of the items from the external code hosting platform according to the configuration information of the items, and executes operations such as compiling, mirror image construction and mirror image publishing.
A user triggers an executor server module to compile and print 3 git projects on a management background service module, then jointly compresses the tar packets together after the images are printed, and then mixes the tar packets with related deployment documents, description documents and the like to print delivery compression packets, and the delivery compression packets are pushed to an external file storage platform, such as hdfs or ftp.
As can be seen from the above embodiments, in this embodiment, the management background service module in the persistent integration and delivery system can provide services for adding, deleting, updating, and viewing git items used by the user side, the executor service module can provide services for code cloning, compiling, mirror image construction, and mixed tar package delivery, and through the mutual cooperation of the management background service module and the executor service module, joint management of multiple git items of a project can be achieved. Compared with the prior art, the embodiment of the invention can fully abstract the service scene, rely on the core logic of code integration, construction and delivery to the system according to the principle of simplicity and high efficiency, does not need to compile any complex script file, and has simple operation and higher development efficiency. In addition, the method can also support the construction of multiple different programming language git projects and meet the requirements of developers with different programming language types.
Fig. 7 is a flowchart of a continuous integration and delivery method according to an embodiment of the present invention, which is based on the continuous integration and delivery system described above, and the continuous integration and delivery system includes: manage backstage service module and executor service module, wherein, manage backstage service module and include: engineering management submodule and engineering release submodule, executor service module includes: the code cloning submodule, the compiling submodule, the mirror image constructing submodule and the aggregation packing submodule; as shown in fig. 7, the method may include the steps of: step 701, step 702, step 703, step 704, step 705 and step 706, wherein,
in step 701, the project management sub-module receives a project management operation input by a user, where the project management operation is used to add, delete, edit, or search a git project of a target project, and the target project corresponds to multiple git projects.
In step 702, the code cloning sub-module clones the code for the git item from the code hosting platform and provides it to the compiling sub-module, in case the item management operation is used to add or edit the git item.
In step 703, the compiling submodule compiles the code of the git project to obtain a compiled file, and provides the compiled file to the mirror image constructing submodule.
In step 704, the image construction sub-module performs image construction on the compiled file to obtain an image file, and provides the image file to the aggregate packaging sub-module.
In step 705, the project publishing sub-module receives a publishing operation input by a user, where the publishing operation is used to instruct the aggregation and packaging sub-module to generate a compressed delivery package corresponding to the target project based on the image file, and send the compressed delivery package to the remote platform for downloading by other users.
In step 706, the aggregation and packaging sub-module aggregates and packages the image files corresponding to all git projects of the target project into a tar package in response to the publishing operation, generates a delivery compression package based on the tar package, and sends the delivery compression package to the remote platform.
As can be seen from the above embodiments, in this embodiment, the management background service module in the persistent integration and delivery system can provide services for adding, deleting, updating, and viewing git items used by the user side, the executor service module can provide services for code cloning, compiling, mirror image construction, and mixed tar package delivery, and through the mutual cooperation of the management background service module and the executor service module, joint management of multiple git items of a project can be achieved. Compared with the prior art, the embodiment of the invention can fully abstract the service scene, rely on the core logic of code integration, construction and delivery to the system according to the principle of simplicity and high efficiency, does not need to compile any complex script file, and has simple operation and higher development efficiency. In addition, the method can also support the construction of multiple different programming language git projects and meet the requirements of developers with different programming language types.
Optionally, as an embodiment, the system operates in a docker environment of a host;
and the mirror image construction sub-module acquires a dockerfile corresponding to the git project from a preset database, and constructs a mirror image file corresponding to the compiled file based on the dockerfile.
Optionally, as an embodiment, the git project includes a plurality of project sub-modules, each project sub-module corresponds to a dockerfile, each dockerfile records a name value of a label attribute, the label attribute is used to control a generated mirror name, and the name values of the label attributes recorded in different dockerfiles are different;
the compiling submodule compiles the codes of each project submodule to obtain a compiling file corresponding to each project submodule and provides the compiling file to the mirror image construction submodule;
the image construction sub-module recursively searches compiled files corresponding to the project sub-modules, transfers the compiled files to a root directory, acquires dockerfile files corresponding to the project sub-modules from a preset database, and performs image construction on the compiled files corresponding to the project sub-modules based on the dockerfile files corresponding to the project sub-modules to obtain image files corresponding to the project sub-modules;
and the name of the mirror image file corresponding to each project sub-module is the name value of the label attribute in the dockerfile corresponding to the project sub-module.
Optionally, as an embodiment, a plurality of instances are deployed in at least one sub-module of the executor service module, and each instance includes a plurality of threads.
Optionally, as an embodiment, after receiving the project management operation, the project management sub-module receives git project configuration information input by a user, and sends the git project configuration information to a preset database for storage;
wherein the git project configuration information comprises: the name of the git project, the source code address of the git project, the compiling language to which the git project belongs, the branch of the git project and the access path of the dockerfile file.
Optionally, as an embodiment, the management background service module further includes: constructing an information display submodule; the method further comprises the following steps:
and the construction information display submodule displays the construction record and the construction state information of the git project in the target project.
As for the method embodiment, since it is basically similar to the system embodiment, the description is simple, and the relevant points can be referred to the partial description of the system embodiment.
According to an embodiment of the present invention, there is also provided an electronic apparatus including: a memory, a processor and a program stored on the memory and executable on the processor, the program when executed by the processor implementing the steps in the method of continuous integration and delivery as described in any of the embodiments above.
According to an embodiment of the present invention, there is also provided a computer-readable storage medium having a program stored thereon, the program, when executed by a processor, implementing the steps in the persistent integration and delivery method according to any one of the above embodiments.
The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.
The above continuous integration and delivery system, method, electronic device and storage medium provided by the present invention are introduced in detail, and a specific example is applied in the present document to explain the principle and the implementation of the present invention, and the description of the above embodiment is only used to help understanding the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims (14)

1. A continuous integration and delivery system, the system comprising: the system comprises a management background service module and an actuator service module, wherein the management background service module comprises: the engineering management submodule and the engineering release submodule, wherein the executor service module comprises: the code cloning submodule, the compiling submodule, the mirror image constructing submodule and the aggregation packing submodule;
the project management submodule is used for receiving project management operation input by a user, wherein the project management operation is used for adding, deleting, editing or searching git projects of a target project, and the target project corresponds to a plurality of git projects;
the code cloning submodule is used for cloning the codes of the git project from a code hosting platform under the condition that the project management operation is used for newly adding or editing the git project and providing the codes of the git project to the compiling submodule;
the compiling submodule is used for compiling the code of the git project to obtain a compiled file and providing the compiled file to the mirror image construction submodule;
the image construction submodule is used for carrying out image construction on the compiled file to obtain an image file and providing the image file to the aggregation and packaging submodule;
the project publishing submodule is used for receiving a publishing operation input by a user, wherein the publishing operation is used for indicating the aggregation and packaging submodule to generate a delivery compression package corresponding to the target project based on the image file and sending the delivery compression package to a remote platform for other users to download;
and the aggregation and packaging submodule is used for responding to the issuing operation, aggregating and packaging the image files corresponding to all git projects of the target project into a tar package, generating the delivery compression package based on the tar package, and sending the delivery compression package to the remote platform.
2. The system of claim 1, wherein the system operates in a docker environment of a host machine;
the image construction sub-module is specifically configured to acquire a dockerfile corresponding to the git project from a preset database, and construct an image file corresponding to the compiled file based on the dockerfile.
3. The system of claim 2, wherein the git project comprises a plurality of project sub-modules, each project sub-module corresponds to a dockerfile, each dockerfile has a name value of a label attribute recorded therein, the label attribute is used for controlling the generated mirror name, and the name values of the label attributes recorded in different dockerfiles are different;
the compiling submodule is specifically used for compiling the codes of each project submodule to obtain a compiling file corresponding to each project submodule and providing the compiling file to the mirror image constructing submodule;
the mirror image construction sub-module is specifically configured to recursively find out compiled files corresponding to each project sub-module, transfer the compiled files to a root directory, acquire dockerfile files corresponding to each project sub-module from a preset database, and perform mirror image construction on the compiled files corresponding to each project sub-module based on the dockerfile files corresponding to each project sub-module to obtain mirror image files corresponding to each project sub-module;
and the name of the mirror image file corresponding to each project sub-module is the name value of the label attribute in the dockerfile corresponding to the project sub-module.
4. The system of claim 1, wherein a plurality of instances are deployed in at least one submodule of the executor service module, each instance comprising a plurality of threads.
5. The system of any one of claims 2 to 4, wherein the engineering management submodule is further configured to:
after the project management operation is received, the git project configuration information input by a user is received, and the git project configuration information is sent to a preset database to be stored;
wherein the git project configuration information comprises: the name of the git project, the source code address of the git project, the compiling language to which the git project belongs, the branch of the git project and the access path of the dockerfile file.
6. The system of any of claims 1 to 5, wherein the management background service module further comprises: constructing an information display submodule; wherein the content of the first and second substances,
and the construction information display submodule is used for displaying the construction record and the construction state information of the git project in the target project.
7. A continuous integration and delivery method based on the continuous integration and delivery system of claim 1, the method comprising:
the project management sub-module receives project management operation input by a user, wherein the project management operation is used for adding, deleting, editing or searching git projects of a target project, and the target project corresponds to a plurality of git projects;
under the condition that the project management operation is used for newly adding or editing the git project, the code cloning submodule clones the code of the git project from the code hosting platform and provides the code to the compiling submodule;
compiling the code of the git project by the compiling submodule to obtain a compiled file, and providing the compiled file to the mirror image constructing submodule;
the mirror image construction submodule carries out mirror image construction on the compiled file to obtain a mirror image file, and the mirror image file is provided for the aggregation packaging submodule;
the project release submodule receives a release operation input by a user, wherein the release operation is used for indicating the aggregation and packaging submodule to generate a delivery compression package corresponding to the target project based on the image file and sending the delivery compression package to a remote platform for other users to download;
and the aggregation and packaging submodule responds to the issuing operation, aggregates and packages the image files corresponding to all git projects of the target project into a tar package, generates the delivery compression package based on the tar package, and sends the delivery compression package to the remote platform.
8. The method of claim 7, wherein the system operates in a docker environment of a host machine;
and the mirror image construction sub-module acquires a dockerfile corresponding to the git project from a preset database, and constructs a mirror image file corresponding to the compiled file based on the dockerfile.
9. The method of claim 8, wherein the git project comprises a plurality of project sub-modules, each project sub-module corresponds to a dockerfile, each dockerfile has a name value of a label attribute recorded therein, the label attribute is used for controlling the generated mirror name, and the name values of the label attributes recorded in different dockerfiles are different;
the compiling submodule compiles the codes of each project submodule to obtain a compiling file corresponding to each project submodule and provides the compiling file to the mirror image construction submodule;
the image construction sub-module recursively searches compiled files corresponding to the project sub-modules, transfers the compiled files to a root directory, acquires dockerfile files corresponding to the project sub-modules from a preset database, and performs image construction on the compiled files corresponding to the project sub-modules based on the dockerfile files corresponding to the project sub-modules to obtain image files corresponding to the project sub-modules;
and the name of the mirror image file corresponding to each project sub-module is the name value of the label attribute in the dockerfile corresponding to the project sub-module.
10. The method of claim 7, wherein a plurality of instances are deployed in at least one submodule of the executor service module, each instance comprising a plurality of threads.
11. The method according to any one of claims 8 to 10, wherein the project management submodule receives git project configuration information input by a user after receiving the project management operation, and sends the git project configuration information to a preset database for storage;
wherein the git project configuration information comprises: the name of the git project, the source code address of the git project, the compiling language to which the git project belongs, the branch of the git project and the access path of the dockerfile file.
12. The method of any of claims 7 to 11, wherein the managing the background services module further comprises: constructing an information display submodule; the method further comprises the following steps:
and the construction information display submodule displays the construction record and the construction state information of the git project in the target project.
13. An electronic device, comprising: memory, processor and program stored on the memory and executable on the processor, which when executed by the processor implements the steps of the method of continuous integration and delivery according to any one of claims 7 to 12.
14. A computer readable storage medium having stored thereon a program which, when executed by a processor, carries out the steps of the method of continuous integration and delivery according to any one of claims 7 to 12.
CN202110529679.3A 2021-05-14 2021-05-14 Continuous integration and delivery system, method, electronic device and storage medium Pending CN113434194A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110529679.3A CN113434194A (en) 2021-05-14 2021-05-14 Continuous integration and delivery system, method, electronic device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110529679.3A CN113434194A (en) 2021-05-14 2021-05-14 Continuous integration and delivery system, method, electronic device and storage medium

Publications (1)

Publication Number Publication Date
CN113434194A true CN113434194A (en) 2021-09-24

Family

ID=77802461

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110529679.3A Pending CN113434194A (en) 2021-05-14 2021-05-14 Continuous integration and delivery system, method, electronic device and storage medium

Country Status (1)

Country Link
CN (1) CN113434194A (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109491662A (en) * 2018-10-31 2019-03-19 珠海市筑巢科技有限公司 Code packaging method, computer installation and computer readable storage medium, code packaging system
US20190303187A1 (en) * 2018-03-29 2019-10-03 The United States Of America As Represented By The Secretary Of The Navy Methods, devices, and systems for distributing software to and deploying software in a target environment
CN111443940A (en) * 2020-05-08 2020-07-24 南京大学 Complete software life cycle management method and platform based on DevOps
CN111901294A (en) * 2020-06-09 2020-11-06 北京迈格威科技有限公司 Method for constructing online machine learning project and machine learning system
CN111930388A (en) * 2020-07-13 2020-11-13 银盛支付服务股份有限公司 Continuous integration method and system based on k8s and docker container
WO2021035553A1 (en) * 2019-08-27 2021-03-04 西门子股份公司 Application program development and deployment method and apparatus, and computer readable medium
CN112579101A (en) * 2020-12-21 2021-03-30 广州博冠信息科技有限公司 Task script control method and device, electronic equipment and storage medium
US20210124576A1 (en) * 2019-10-24 2021-04-29 Next Pathway Inc. System and Method for Automated Microservice Source Code Generation and Deployment

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190303187A1 (en) * 2018-03-29 2019-10-03 The United States Of America As Represented By The Secretary Of The Navy Methods, devices, and systems for distributing software to and deploying software in a target environment
CN109491662A (en) * 2018-10-31 2019-03-19 珠海市筑巢科技有限公司 Code packaging method, computer installation and computer readable storage medium, code packaging system
WO2021035553A1 (en) * 2019-08-27 2021-03-04 西门子股份公司 Application program development and deployment method and apparatus, and computer readable medium
US20210124576A1 (en) * 2019-10-24 2021-04-29 Next Pathway Inc. System and Method for Automated Microservice Source Code Generation and Deployment
CN111443940A (en) * 2020-05-08 2020-07-24 南京大学 Complete software life cycle management method and platform based on DevOps
CN111901294A (en) * 2020-06-09 2020-11-06 北京迈格威科技有限公司 Method for constructing online machine learning project and machine learning system
CN111930388A (en) * 2020-07-13 2020-11-13 银盛支付服务股份有限公司 Continuous integration method and system based on k8s and docker container
CN112579101A (en) * 2020-12-21 2021-03-30 广州博冠信息科技有限公司 Task script control method and device, electronic equipment and storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
张琦: "基于Docker的CaaS管理平台架构研究与设计", 《计算机应用与软件》, vol. 35, no. 11, 12 November 2018 (2018-11-12), pages 33 - 41 *

Similar Documents

Publication Publication Date Title
US10990365B2 (en) Container image building using dependency container images
US9513938B2 (en) Virtual appliance integration with cloud management software
US8302093B2 (en) Automated deployment of defined topology in distributed computing environment
CN106681956A (en) Method and device for operating large-scale computer cluster
CN107885551A (en) A kind of service arrangement method, apparatus, computer-readable recording medium and storage control
CN112965785B (en) Container-based micro-service application development method and development platform
US10915551B2 (en) Change management for shared objects in multi-tenancy systems
JP6673359B2 (en) System construction support system, method and program
US11029943B1 (en) Processing framework for in-system programming in a containerized environment
US10691659B2 (en) Integrating copy data tokens with source code repositories
CN111857801B (en) Construction method of mobile application
US20210334149A1 (en) Api adapter creation device, api adapter creation method, and api adapter creation program
CN109344619A (en) The hot restorative procedure and device of application program
CN104423961A (en) Method and system for generating testing script
CN116860746A (en) Processing system for lightweight big data
CN113094028A (en) Windows desktop program development framework, method and related components
CN113835705A (en) Big data service product development method, device and system
CN113296891A (en) Multi-scene knowledge graph processing method and device based on platform
CN116450107B (en) Method and device for secondary development of software by low-code platform and electronic equipment
KR102455316B1 (en) Method for unifying tool and information coming from multiple information sources, and computer program product and device using same
US5963955A (en) Bridge for exporting and importing objects between object oriented programming environments
CN113434194A (en) Continuous integration and delivery system, method, electronic device and storage medium
CN114124739B (en) Method and device for deploying FreeWITCH service, electronic equipment and storage medium
US10802810B2 (en) Consuming persistent library bundles
CN115525396A (en) Application management method and device based on cloud protogenesis

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination