CN115167972A

CN115167972A - Cloud native platform integration method and system

Info

Publication number: CN115167972A
Application number: CN202210602491.1A
Authority: CN
Inventors: 李江涛
Original assignee: Inspur Communication Technology Co Ltd
Current assignee: Inspur Communication Technology Co Ltd
Priority date: 2022-05-30
Filing date: 2022-05-30
Publication date: 2022-10-11

Abstract

The invention provides a cloud native platform integration method and a system, which belong to the technical field of computers and comprise the following steps: deploying Gitlab in Kubernets, and creating Webhook in the Gitlab; and generating an event based on the Webhook, and triggering trigger unit by the event to generate Kubernetes identifiable resources. In a Kubernets environment, tasks are arranged through pipelines, and the pipelines and the tasks are predefined, so that trigger can be triggered to execute the operation of a full-flow code, a low-cost containerization integration scheme is realized, and a cloud native environment is healthier and more transparent.

Description

Cloud native platform integration method and system

Technical Field

The invention relates to the technical field of computers, in particular to a cloud native platform integration method and system.

Background

kubernets, K8s for short, is an open source for managing containerized applications on multiple hosts in a cloud platform, and aims to make deploying containerized applications simple and efficient and provide mechanisms for application deployment, planning, updating, and maintenance. The traditional application deployment mode is to install an application through a plug-in or a script, and the disadvantage of this is that the running, configuration, management, and all life cycles of the application are bound with the current operating system, which is not beneficial to the operations of upgrading, updating, rolling back, etc. of the application. The new mode is realized by deploying containers, each container is isolated from each other, each container has a file system, processes among the containers cannot influence each other, and computing resources can be distinguished. Compared with a virtual machine, the container can be rapidly deployed, and due to the fact that the container is decoupled from underlying facilities and a machine file system, the container can be migrated between different clouds and different versions of operating systems.

Particularly, kubernetes can realize rapid deployment and application, rapid expansion and application, seamless docking of new application functions, resource saving and optimized use of hardware resources; the Kubernetes functions also include: a plurality of pods working cooperatively; mounting a storage system; applying a health check; replication of the application instance; pod auto-scaling/expansion; registering and discovering; load balancing; updating in a rolling mode; monitoring resources; log access; debugging an application program; authentication and authorization are provided. In addition, a concept of Continuous Integration (CI)/Continuous Deployment (CD) is also proposed, systematic software necessarily involves scenarios of continuously integrating, integrating and continuously verifying and applying new functions in the development and delivery processes, the existing mode is that development and testing integrators continuously integrate around Jenkins, jenkins is an open source software project, and is a Continuous Integration tool developed based on Java, and is used for monitoring continuously repeated work, and an open easy-to-use software platform is provided, so that after the software project is continuously integrated, that is, jenkins is containerized and deployed, the business is migrated into a container for execution, and the containerization scheme brings inconvenience and difficulty to the Continuous Integration process, mainly as follows:

(1) All tool chains and code operating environments need to be subjected to containerization deployment around Jenkins, so that the problems of single-point overhigh load, single-point failure and the like are easily caused;

(2) The continuous integration process established based on the Jenkins container is closed and agnostic to the cloud native environment, and the replication process is complicated;

(3) The continuous integrated upgrade needs to be reconstructed to build a mirror image, and the learning and maintenance cost is high.

Therefore, a new cloud-native platform integration method needs to be provided to address the limitations in the current cloud-native container integration operation.

Disclosure of Invention

The invention provides a cloud native platform integration method and system, which are used for overcoming the defect caused by adopting Jenkins components for integrating software in Kubernets in the prior art and realizing containerization integration operation of sustainable integration and learning.

The invention provides a cloud native platform integration method, which comprises the following steps:

deploying Gitlab in Kubernets, and creating Webhook in the Gitlab;

and generating an event based on the Webhook, and triggering trigger unit by the event to generate Kubernetes identifiable resources.

According to the cloud native platform integration method provided by the invention, before deploying the Gitlab in Kubernets, the method further comprises the following steps:

deploying a Kubernetes cluster, wherein the Kubernetes cluster comprises a control node and a working node;

deploying a tekton component in the kubernets cluster by using a mark-up language yaml, wherein the tekton component comprises a task component task, a pipeline component pipeline and a dashboard component dashboard.

According to the cloud native platform integration method provided by the present invention, after deploying tekton components in the kubernets cluster by using yaml, the method further includes:

creating a pipeline template in the Kubernets cluster based on the pipeline, wherein the pipeline template comprises a task running task run template and a pipeline running pipeline run template;

deploying a trigger resource in the Kubernets cluster.

According to the cloud native platform integration method provided by the invention, the creating of the pipeline template in the kubernets cluster based on the pipeline comprises the following steps:

determining a plurality of tasks, and arranging the plurality of tasks based on the pipeline to obtain a plurality of resources and a plurality of corresponding resource references;

obtaining a pipeline resource PipelineResource based on the plurality of resources, and obtaining a pipeline reference PipelineRef and a task reference taskRef based on the plurality of resource references;

and respectively obtaining the taskrun template through the pipeline Ref, and obtaining the pipeline template through the taskRef.

According to the cloud native platform integration method provided by the invention, deploying trigger resources in the kubernets cluster comprises the following steps:

determining an event listener container Eventlistener Pod;

inputting the event to be triggered into the Eventlistener Pod, verifying the event to be triggered by using a verification component Triggerbinding, and extracting the field attribute of the event to be triggered;

and endowing the field attribute to the trigger element and outputting the trigger resource.

According to the cloud native platform integration method provided by the present invention, after generating kubernets identifiable resource, the method further comprises:

and acquiring a pipeline trigger state and a task execution state through the dashboard.

In a second aspect, the present invention further provides a cloud-native platform integration system, including:

the deployment module is used for deploying Gitlab in Kubernets and creating Webhook in the Gitlab;

and the generating module is used for generating vent based on the Webhook, triggering trigger unit by the event and generating Kubernetes identifiable resources.

In a third aspect, the present invention further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program to implement the cloud native platform integration method according to any one of the above methods.

In a fourth aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a cloud-native platform integration method as described in any one of the above.

In a fifth aspect, the present invention also provides a computer program product comprising a computer program which, when executed by a processor, implements the cloud-native platform integration method as described in any one of the above.

According to the cloud native platform integration method and system provided by the invention, the task is arranged through the pipeline, the pipeline template and the task template are predefined, and the trigger can be triggered to execute the whole-flow code operation, so that a low-cost containerization integration scheme is realized, and the cloud native environment is healthier and more transparent.

Drawings

In order to more clearly illustrate the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.

FIG. 1 is a schematic flow diagram of a cloud-native platform integration method provided by the present invention;

FIG. 2 is a schematic diagram of the implementation of pipeline and task provided by the present invention;

FIG. 3 is a trigger flow diagram provided by the present invention;

FIG. 4 is a schematic structural diagram of a cloud native platform integration system provided by the present invention;

fig. 5 is a schematic structural diagram of an electronic device provided in the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Aiming at the defects in cloud native container platform integration in the prior art, the invention provides a novel method for continuously integrating a cloud native container platform, which does not adopt the traditional Jenkins component for containerized deployment any more, but adopts a tekton component based on native resources to realize the basic function of continuous integration, continuously deploys the full-flow function, and avoids the defects of the original Jenkins containerized deployment.

Fig. 1 is a schematic flow diagram of a cloud native platform integration method provided in the present invention, as shown in fig. 1, including:

step 100: deploying Gitlab in Kubernets, and creating Webhook in the Gitlab;

step 200: and generating an event based on the Webhook, and triggering trigger unit by the event to generate Kubernetes identifiable resources.

It should be noted that cloud-native is a set of cloud technology product system established based on technologies such as container, micro-service, devOps, etc., based on distributed deployment and unified operation and management of distributed cloud.

Firstly, gitlab is deployed in Kubernets, the invention adopts the Gitlab, namely a code warehouse management system, for an open source project of the warehouse management system, uses Git as a code management tool, and builds Web service on the basis, thereby realizing batch processing of massive codes.

Secondly, creating Webhook in Gitlab, which is an API concept and one of the usage paradigms of microservice APIs, and also being a reverse API, i.e. the front end does not actively send a request and is completely pushed by the back end, e.g. a friend has sent a friend circle, the back end pushes this message to all other friends' clients, i.e. the typical scenario of Webhook, webhook is a URL that receives HTTP POST (or GET, PUT, DELETE), an API provider that implements Webhook sends a message to this configured URL when an event occurs, unlike the request-response method, the change can be received in real time using Webhook, the reversion to the client-server mode, in the conventional method, the client requests data from the server, and then the server provides the client data (the client is pulling data), in the request-response method, the server updates the resources that need to be provided and then automatically sends them to the client as updates (the server is pushing data, but not the receiver requests data); this reversal of the control relationship may be used to facilitate many communication requests that would otherwise require more complex requests and constant polling on the remote server.

And finally, generating an event by the created interface Webhook, triggering on the created trigger update, and generating Kubernets identifiable resources.

The invention relates to a conversion of a non-cloud-native continuous integration process into a native continuous integration process, uses a code warehouse to generate resources which can be identified by a cluster, and triggers a predefined production line to complete CI/CD work.

On the basis of the above embodiment, before deploying the Gitlab in kubernets, the method further includes:

In the invention, a kubernets cluster is required to be deployed firstly, and the cluster comprises a control node and a working node, namely a master node and a worker node. Also, it is desirable to deploy tekton components, including task, pipeline and dashboard, in kubernets clusters, where yaml, a markup language, is used.

Tekton is a kubernets native framework adapted to create google sources for continuous integration and continuous deployment/delivery (CI/CD) systems, defines Pipeline based on kubernets CRD, is powerful and easily scalable, including the following core components:

pipeline: the method is a basis of Tekton, defines a group of Kubernets self-defined resources which are used as building blocks and can assemble CI/CD pipelines;

triggers: allowing a pipeline to be instantiated upon an event, for example, instantiation and execution of the pipeline may be triggered each time a PR is incorporated into a GitHub repository, and a user interface may be built that launches a particular Tekton trigger;

cli: a command line interface is provided, named tkn, which builds on top of kubernets CLI, allowing interaction with Tekton;

dashboards: is a Web-based graphical interface to Tekton pipeline that displays information about pipeline execution;

catalog: a repository of high quality Tekton building blocks contributed by the community-Tasks, pipeline, etc., that can be used in their own pipeline;

hub: is a Web-based graphical interface for accessing Tekton Catalog;

operator: is a Kubernetes Operato model that allows Tekton projects to be installed, updated, and deleted on Kubernetes clusters.

It will be appreciated that due to the full and powerful functionality of the Tekton component, the full requirements of system integration can be met.

yaml is a high readability format for expressing data serialization, and references are made to other languages, including: c language, python, perl, and gets inspiration from XML, data format of email. Thus, yaml is similar to other high-level languages, and can simply express data forms such as lists, hash tables, scalars, etc., using space indentation and a number of appearance-dependent features, particularly suited for expressing or editing data structures, various profiles, typographically debugged content, and file schemas, etc.

According to the invention, by pre-deploying a Kubernets cluster architecture and adopting a Tekton component different from a Jenkins component, the defect of originally containerized Jenkins deployment is effectively avoided, all basic components which are native to cloud are fully utilized, and the difficulty brought by CI process construction containerization and tool deployment is reduced.

On the basis of the above embodiment, after deploying the tekton module in the kubernets cluster by using yaml, the method further includes:

creating a pipeline template in the Kubernets cluster based on the pipeline, wherein the pipeline template comprises a task running template and a pipeline running pipeline template;

deploying a trigger resource in the Kubernets cluster.

Wherein the creating a pipeline template in the Kubernets cluster based on the pipeline comprises:

Wherein the deploying of trigger resources in the kubernets cluster comprises:

determining an event listener container Eventlistener Pod;

Specifically, a task template is created in the kubernets cluster, and the pipeline template is organized based on the task template and according to a specific business form.

As shown in fig. 2, in the whole implementation method, task is implemented in kubernets of a specific task, pipeline is a service arrangement for multiple tasks, so as to ensure diversification of service implementation forms, resources and resources ref need to be distinguished in implementation logic of tekton, task pipeline is equivalent to a resource template, and task run and pipeline define resource parameters, and the template is described in detail, so that the template operates in a cluster in combination with specific resources.

As also shown in fig. 3, the trigger enables the user to map fields in an event into a resource template, in other words, this allows the event to be both modeled and instantiated as a tekton resource, for tektoned/pipeline, the trigger enables parameters in the event to be easily encapsulated into objects such as pipeline runs, pipeline resources, etc.

EventListener in fig. 3 is a custom resource of kubernets that allows users to handle HTTP incoming events with JSON data bodies in a declarative manner. EventListener needs to open services to the outside to let events come in. The user may declare TriggerBinding to extract fields from the event and apply them to TriggerTemplate to create a Tekton resource. Furthermore, eventListener allows lightweight event processing using event interceptors.

The method is characterized in that the method is customized for a specific matter by predefining triggeremplate, and the TriggerTemplate is a kubernets resource which can be templated. The Event specifies the type and the detailed information of the trigger through the interrupt, the invention uses the gitlab webhook to trigger, the definition comprises the gitlab login verification information and the corresponding webhook setting, and the triggerBinding verifies the Event and extracts the related field attribute.

The developer submits the code to GitLab, then Gitlab sends the pushing event to EventListener, then Trigger extracts information from the data body based on the configuration of Triggerbinding, loads the information in the parameters and transmits the parameters to Triggertemplate for analysis, and finally instantiates the resources (objects such as task, task run, pipeline and the like) of tekton. Triggering of the pipeline can be achieved if taskrun and pipeline are instantiated. The whole process is clearly divided into work, and each object can be decoupled and reused.

Here, the tekton trigger is an event-based trigger that can detect the occurrence of an event and extract information in the event, from which taskrun and pipeline are instantiated and executed.

the four most basic components in a tekton trigger are: triggermeplate, triggerBinding, eventListener and interpolator.

Triggertemplate: specifying resources, such as taskrun and pipeline, to instantiate and execute upon detection of an event;

triggerbinding: specifies a field to be extracted from the event information and passes the field information to triggermeplate so that the field can be used in the taskrun and the pipeline.

EventListener: and starting an event monitoring service in the Kubernets cluster, exposing the service, and binding TriggerTemplate and TriggerBinding.

An interpolator: and the event interceptor is used for filtering the events and runs before the trigger binding.

According to the invention, through creating a task template in a Kubernets cluster and deploying trigger resources, events can be modeled and instantiated into tekton resources. For tektoning/pipeline, triggers allow parameters in events to be easily encapsulated into the objects PipelineRuns, pipelineResources, etc.

On the basis of the foregoing embodiment, after the generating the kubernets identifiable resource, the method further includes:

Specifically, according to the method, a webhook of the trigger time is created on the gitlab by deploying the containerized gitlab, for example, the event trigger is generated by adopting code push and merge, a trigger template is deployed in a kubernets cluster in a yaml mode to receive the gitlab event trigger and generate kubernets related resources, and the wrapper code is used for manually monitoring the pipeline trigger and task execution by logging in the dashboards.

The dashboard is a general Web interface of tekton pipeline and tekton trigger, and can realize the work of creating, executing, checking and the like of tekton resources through the dashboard.

It can be understood that through the monitorable page, the progress and details of the executed task can be checked by triggering task and pipeline execution through the page, and the visualization of the kubernets cluster on the CI flow, the interactive operation of the cluster, the re-development of the cluster alarm and the expansion of the CI function are realized.

The application scene of the pipeline in the invention not only stays in the role of a string flow, but also can bear a solution, the solution can be converted into compiling yaml to obtain the pipeline by arranging task, a kubecultl application command is executed on a cluster, then parameters are filled to create a pipeline, and multiple service forms can be introduced into a corresponding scheme; in addition, the alarm in the service execution process can be defined and captured through the kubernets cluster, and more detailed development is achieved.

The following is a description of the complete technical scheme flow of the invention:

(1) Creating a kubernets cluster which comprises a control controller node and a working worker node;

(2) Tekton base components task, pipeline, dashboard are deployed in a kubernets cluster using the yaml approach.

(3) And (3) constructing a pipeline template containing a continuous integration task in the kubernets cluster, constructing a taskrun and pipeline template, and manually operating to operate the pipeline according to the definition content of the template so as to complete a complete continuous integration process.

(4) And deploying trigger related resources.

(5) Deploying the containerized gitlab, creating webhook of the trigger time on the gitlab, and generating the event trigger by adopting code push and merge.

(6) And deploying trigger templates in a kubernets cluster in a yaml mode to accept the gitlab event trigger and generate kubernets related resources.

(7) Login to dashboard, manually use merger code, monitor pipeline trigger and task execution.

The method is characterized in that a continuous integration process of a native code is realized based on Kubernets, a task is arranged through pipeline, a pipeline template and a task template are predefined, and the pipeline is triggered manually or trigger to activate the pipeline to run; constructing a gitlab container outside, monitoring a code warehouse in a webhook form, triggering a trigger automatically, triggering a pipeline through a predefined trigger template, providing conversion from jenkisnfile to pipeline, building an open ecology, easily deploying other services into an implementation scheme, and realizing alarm redevelopment, page visualization and command line interaction.

The cloud-native platform integration system provided by the present invention is described below, and the cloud-native platform integration system described below and the cloud-native platform integration method described above may be referred to in correspondence with each other.

Fig. 4 is a schematic structural diagram of the cloud-native platform integration system provided in the present invention, as shown in fig. 4, including: a deployment module 41 and a generation module 42, wherein:

the deployment module 41 is used for deploying Gitlab in Kubernets, and Webhook is created in the Gitlab; the generating module 42 is configured to generate an event based on the Webhook, and trigger the Triggertemplate by the event to generate a Kubernetes identifiable resource.

According to the invention, the task is arranged through the pipeline, the pipeline template and the task template are predefined, and trigger execution of the full-flow code operation by the trigger can be triggered, so that a low-cost containerization integration scheme is realized, and the cloud native environment is healthier and more transparent.

Fig. 5 illustrates a physical structure diagram of an electronic device, which may include, as shown in fig. 5: a processor (processor) 510, a communication Interface (Communications Interface) 520, a memory (memory) 530 and a communication bus 540, wherein the processor 510, the communication Interface 520 and the memory 530 communicate with each other via the communication bus 540. Processor 510 may invoke logic instructions in memory 530 to perform a cloud-native platform integration method comprising: deploying Gitlab in Kubernets, and creating Webhook in the Gitlab; and generating an event based on the Webhook, and triggering trigger unit by the event to generate Kubernetes identifiable resources.

Furthermore, the logic instructions in the memory 530 may be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, and various media capable of storing program codes.

In another aspect, the present invention also provides a computer program product, where the computer program product includes a computer program, the computer program may be stored on a non-transitory computer readable storage medium, and when the computer program is executed by a processor, a computer can execute the cloud native platform integration method provided by the above methods, where the method includes: deploying Gitlab in Kubernets, and creating Webhook in the Gitlab; and generating an event based on the Webhook, and triggering trigger unit by the event to generate Kubernetes identifiable resources.

In yet another aspect, the present invention also provides a non-transitory computer readable storage medium, on which a computer program is stored, the computer program, when being executed by a processor, implementing a cloud native platform integration method provided by the above methods, the method including: deploying Gitlab in Kubernets, and creating Webhook in the Gitlab; and generating an event based on the Webhook, triggering Triggertedplate by the event, and generating Kubernets identifiable resources.

The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment may be implemented by software plus a necessary general hardware platform, and may also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.

Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, and not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A cloud-native platform integration method, comprising:

deploying a code warehouse Gitlab in a container arranging engine Kubernets, and creating an application programming interface Webhook in the Gitlab;

and generating an event based on the Webhook, triggering a trigger template trigger by the event, and generating Kubernetes identifiable resources.

2. The cloud-native platform integration method according to claim 1, further comprising, prior to deploying Gitlab in Kubernets:

3. The cloud-native platform integration method according to claim 2, wherein said deploying yaml after deploying tekton components in said kubernets cluster further comprises:

deploying a trigger resource in the Kubernets cluster.

4. The cloud-native platform integration method according to claim 3, wherein said creating a pipeline template in said Kubernets cluster based on said pipeline comprises:

5. The cloud-native platform integration method according to claim 3, wherein said deploying a trigger resource in said Kubernets cluster comprises:

determining an event listener container Eventlistener Pod;

6. The cloud-native platform integration method according to claim 1, further comprising, after said generating kubernets identifiable resources:

7. A cloud-native platform integration system, comprising:

the system comprises a deployment module, a database module and a management module, wherein the deployment module is used for deploying Gitlab in Kubernets and creating Webhook in the Gitlab;

and the generating module is used for generating an event based on the Webhook, triggering the trigger unit by the event and generating the Kubernetes identifiable resource.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements the cloud-native platform integration method of any of claims 1 to 6.

9. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the cloud-native platform integration method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the cloud-native platform integration method of any of claims 1 to 6.