CN117873862A - Automatic regression testing system and method for container engine platform - Google Patents
Automatic regression testing system and method for container engine platform Download PDFInfo
- Publication number
- CN117873862A CN117873862A CN202311657517.3A CN202311657517A CN117873862A CN 117873862 A CN117873862 A CN 117873862A CN 202311657517 A CN202311657517 A CN 202311657517A CN 117873862 A CN117873862 A CN 117873862A
- Authority
- CN
- China
- Prior art keywords
- container
- container engine
- test
- task
- engine platform
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 147
- 238000000034 method Methods 0.000 title claims abstract description 25
- 238000012795 verification Methods 0.000 claims abstract description 22
- 230000008859 change Effects 0.000 claims abstract description 7
- 238000012544 monitoring process Methods 0.000 claims abstract description 6
- 230000001960 triggered effect Effects 0.000 claims description 4
- 230000008569 process Effects 0.000 abstract description 4
- 238000000275 quality assurance Methods 0.000 abstract 1
- 230000010354 integration Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000002085 persistent effect Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000013515 script Methods 0.000 description 2
- 238000002560 therapeutic procedure Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 1
- 239000000523 sample Substances 0.000 description 1
Landscapes
- Testing Or Calibration Of Command Recording Devices (AREA)
Abstract
The invention discloses an automatic regression testing system and method for a container engine platform, which belong to the technical field of cloud computing, in particular to the field of functional regression quality assurance of a cloud primary container engine platform, and are used for realizing versioning management of resource testing cases of the container engine platform, storing task testing cases and resource pool configuration information in a git code warehouse, continuously integrating configuration item change of a jenkins system monitoring code warehouse, and triggering corresponding regression testing verification task flows when new resource pool configuration information is increased; in jenkins's task, can start a container with the raply instrument on the tester node to draw latest task use case and resource pool configuration information, register the resource pool information to be tested in the raply system, then carry out task test use case, finally export the test result into html format file, the whole process does not need manual intervention, the time is greatly saved and the error of manual test is reduced.
Description
Technical Field
The invention belongs to the technical field of cloud computing, and particularly relates to an automatic regression testing system and method for a container engine platform.
Background
The application of cloud native technology in the field of cloud computing is increasingly widespread, and public cloud manufacturers provide container engine platforms in product matrixes. The container engine platform provides the user with the function of opening a container cluster, and creates various workloads and network services in the cluster for deploying applications.
As a result of the multiple resource types involved in the container engine platform, various workloads, services, configuration items, and the like are involved. In order to ensure that the functions are normal, the container engine platform needs to be subjected to sufficient functional regression verification before being on line. Functional regression verification of the container engine encompasses a variety of test case sets including common deployment, stateful copy sets, daemons sets, tasks, timed tasks, container groups, services, application routing, secret dictionaries, configuration dictionaries, persistent volume declarations, persistent volumes, labels, probes, affinities, namespaces, and so forth.
Corresponding test scripts are designed and written aiming at the various resource types, and the availability of the functions is verified by executing the scripts. Because the resources are more, each resource test case is time-consuming to manually execute, and in a public cloud scene, a plurality of resource pools are frequently changed, and regression verification tests need to be executed after each change. An automated tool is needed to improve efficiency and enable use case versioning.
Disclosure of Invention
Aiming at the problem of functional regression verification of the container engine platform in the background technology, the invention provides an automatic regression testing system and method for the container engine platform based on the rally.
The invention adopts the following technical scheme for solving the technical problems:
an automatic regression testing system of a container engine platform comprises a code warehouse, a testing machine, a container engine platform and a container mirror image warehouse;
the code warehouse is used for storing the container engine platform information including the access address and the certificate file of the container engine cluster, and also storing the automatic test case set;
the testing machine is used for executing nodes of regression testing tasks;
the container engine platform refers to a cloud primary container engine arrangement cluster system to be tested, and comprises three master nodes and a plurality of node nodes, wherein the master nodes are operated with container engine platform control service related components, and the node nodes are operated with life cycles for managing workload when a container is operated;
the container mirror image warehouse is used for storing container mirror image files, and the container mirror image files are used for starting the containers.
As a further preferred embodiment of the container engine platform automated regression testing system of the present invention, the code repository employs a gerit system or a gitlab system.
As a further preferred embodiment of the container engine platform automated regression testing system of the present invention, the testing machine is a virtual machine or physical server.
As a further preferable scheme of the automatic regression testing system of the container engine platform, when the dock container runs on the testing machine, jenkins continuously integrates services and a container provided with a raply tool;
the method comprises the steps that when a dock container runs, the dock container is used for running, a raply tool and a depending virtual environment are manufactured into a container mirror image, after a jenkins system triggers a test task, the container mirror image is pulled on a tester, and then a container for testing is started through the container mirror image;
jenkins continuous integrated service is used for monitoring code merging actions in a code warehouse so as to trigger a test task;
the container is provided with a rally tool and is used for expanding a rally test task set and executing regression verification of a container engine platform; the rally is a set of testing tools widely applied to cloud computing open source infrastructure platforms.
As a further preferred scheme of the container engine platform automatic regression testing system, a testing tool in an openstack platform is adopted by cloud computing open-source infrastructure.
As a further preferable scheme of the container engine platform automatic regression testing system, the container engine platform node is a physical machine or a cloud host.
As a further preferred scheme of the automated regression testing system of the container engine platform, the container mirror warehouse is realized by adopting a docker registry system or a harbor system.
A verification method based on a container engine platform automatic regression testing system specifically comprises the following steps;
step 1, when a certain new resource pool is online for container engine service or has function change operation, a manager submits newly created container engine cluster access authority information to a code warehouse, and when merging operation is performed, jenkins task flow is triggered;
step 2, the jenkins system issues a task to a testing machine, and registers a container engine cluster to be tested to a rack testing system;
step 3, starting to execute test cases, wherein the test case sets are pulled from a code warehouse, the raply receives the input yaml files of the test case sets, executes the test cases in the yaml files respectively, and stores the results in a database;
and step 4, after the test is finished, generating a test result report in an html format, and analyzing the execution condition of the test case set and the failed error stack information in detail from the test result report.
As a further preferred embodiment of the verification method based on the container engine platform automated regression testing system of the present invention, in step 2, the registration information includes an access address and a key of the container engine cluster.
As a further preferred scheme of the verification method based on the container engine platform automatic regression testing system, in step 3, the resources included in the test case set cover the life cycle management test of various resources in the container engine.
Compared with the prior art, the technical scheme provided by the invention has the following technical effects:
1. the invention provides an automatic regression testing system and method for a container engine platform, which solve the complex steps of manually executing a container engine test case set, and simultaneously introduce a code warehouse and a continuous integration system to realize the version management of the test case set and the automation of a test flow; the regression test time is saved, and the bug introduction is reduced; the continuous integrated system can trigger the pipeline in batches to carry out regression verification work of a plurality of container engine clusters;
2. the invention realizes the versioning management of the resource test cases of the container engine platform, the task test cases and the resource pool configuration information are stored in the git code warehouse, the jenkins system is continuously integrated to monitor the configuration item change of the code warehouse, and when new resource pool configuration information is increased, the corresponding regression test verification task flow is triggered; in jenkins' task, a container with a raply tool is started on a tester node, the latest task use case and resource pool configuration information are pulled, the resource pool information to be tested is registered in a raply system, then the task test use case is executed, and finally the test result is exported into an html format file, so that the whole process does not need manual intervention, the time is greatly saved, and the error of manual test is reduced;
drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a diagram of an automated regression verification system for a container engine of the present invention;
FIG. 2 is a flow chart of automated regression verification of the container engine of the present invention;
FIG. 3 is a set of container engine automated regression verification task use cases of the present invention;
FIG. 4 is an embodiment of the automated testing of the container engine of the present invention.
Detailed Description
The technical scheme of the invention is further described in detail below with reference to the accompanying drawings:
the following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention. The objects and effects of the present invention will become more apparent from the following detailed description of the preferred embodiments and the accompanying drawings, it being understood that the specific embodiments described herein are merely illustrative of the invention and not limiting thereof.
FIG. 1 is a diagram of an automated regression verification system for a container engine platform according to the present invention, comprising four components: code repository, tester, container engine platform and container mirror repository.
The code warehouse is used for storing the container engine platform information including the access address and the certificate file of the container engine cluster, and also storing the automatic test case set, and can be realized by adopting a gerrit, gitlab system and the like.
The testing machine refers to a node for executing regression testing tasks, and can be a virtual machine or a physical server, and when a dock container runs, jenkins continuous integrated service and a container provided with a rally tool run on the testing machine: the method comprises the steps that when a dock container runs, a raply tool and a depending virtual environment are manufactured into a container mirror image, after a jenkins system triggers a test task, the container mirror image is pulled on a tester, and then a container for testing is started through the container mirror image; the jenkins continuous integrated service component is used for monitoring code merging actions in a code warehouse so as to trigger a test task; the method is widely applied to a set of testing tools in cloud computing open-source Infrastructure As A Service (IAAS) such as an openstack platform, and extends a raply testing task set so as to execute regression verification of a container engine platform.
The container engine platform refers to a cloud primary container engine arrangement cluster system to be tested, and comprises three master nodes and a plurality of node nodes, wherein the container engine platform nodes can be physical machines or cloud hosts, container engine platform control service related components are operated on the master nodes, and life cycles for managing workload when containers are operated on the node nodes.
The container mirror image warehouse is used for storing container mirror image files, and the container mirror image files are used for starting the containers. The container mirror warehouse may be implemented using a docker registry or a harbor system. The container process used by the workload running in the container engine cluster is pulled from the container image repository.
The container engine automated regression verification flow chart is shown in FIG. 2:
1) When a new resource pool is online for container engine service or has function change operation, an administrator submits newly created container engine cluster access authority information to a code warehouse, and when the merging operation is performed, the task flow of jenkins is triggered.
2) The jenkins system issues tasks to the tester, firstly registers the container engine cluster to be tested to the rack test system, and the registration information comprises the access address and the secret key of the container engine cluster.
3) The test case is started to be executed, wherein the test case set is pulled from the code warehouse, and the resources included in the test case set are shown in fig. 3 and cover the life cycle management test of various resources in the container engine. The rally accepts the test case set input yaml file, executes the test cases therein respectively, and stores the results in a database.
4) After the test is finished, a test result report in an html format is generated, and the execution condition of the test case set and the failed error stack information can be analyzed in detail from the test result report. The error and time of manual test are saved.
As an extension of the present invention, the automated test system may be applied to private cloud, hybrid cloud, etc. scenarios, and may support cpu types of different architectures, including x86 and arm architecture servers. The container mirror image with the ram test tool is respectively constructed according to the CPU architecture.
Aiming at the problem of functional regression verification of the container engine platform in the background technology, the invention provides an automatic regression testing system, device and method for the container engine platform based on the rally. The invention realizes the versioning management of the resource test cases of the container engine platform, saves the task test cases and the resource pool configuration information in the git code warehouse, continuously integrates the configuration item change of the jenkins system monitoring code warehouse, and triggers the corresponding regression test verification task flow when new resource pool configuration information is increased. In jenkins's task, can start a container with the raply instrument on the tester node to draw latest task use case and resource pool configuration information, register the resource pool information to be tested in the raply system, then carry out task test use case, finally export the test result into html format file, the whole process does not need manual intervention, the time is greatly saved and the error of manual test is reduced.
Specific examples are as follows: the embodiment of the automatic regression verification system for the container engine platform provided by the invention comprises the following components: the system comprises a container engine cluster access information and authority configuration file, a regression test task use case set, a code warehouse management component, a continuous integration component, a raply test tool component, a tester node, a container engine platform, a container mirror warehouse and the like.
The container engine access information and the authority configuration file refer to an access address and an access key of a container engine cluster, and a rapy test tool on the tester accesses the container cluster address through the access key to execute operations such as creating a workload;
the test task case set refers to a test case comprising various resources in a container engine cluster, and as shown in fig. 3, the test case set comprises various resources such as workload and network type provided by a container engine; the code warehouse management component is used for versioning management of container engine cluster information and regression test task case sets, and the code warehouse can be realized by using a gerit or gitlab system; the persistent integration component is used for monitoring merging requests in a code warehouse and automatically triggering workflow tasks to carry out regression testing on a container engine cluster, and can be realized by adopting a jenkins system. The raply test tool component refers to a set of test tools capable of executing task sets, receives test case sets, accesses a container engine platform to be tested, executes test cases and stores test results in a database, and can execute an operation of generating a test report after the test case execution is finished, and pulls the test task results from the database and generates a test report in an html format, wherein the whole flow is shown in fig. 4;
the tester node refers to a node for running a rapy tool, and a continuous integrated component jenkins service is simultaneously run on the node, so that as an extension of the invention, a rapy test component on the tester can run on the tester directly or in a container on the tester node, and a dock program is installed on the tester node and can be used for starting to run the container with the rapy tool. The container engine platform refers to a container engine cluster with a test, the container engine cluster is communicated with a tester node network, a rapy component is operated on the tester node, the container engine cluster is registered into the rapy through the access information and the authority of the container engine cluster to be tested, and the life cycle management of resources such as workload and the like is realized based on the access information; the container mirror warehouse refers to a warehouse for storing public mirrors and private mirrors, the mirrors used by the workload running in the container engine are pulled from the container mirror warehouse, and the container mirror warehouse can be realized based on a docker registry or a harbor.
The invention relates to a system and a method for automatically testing regression of a container engine platform; the method comprises the steps of carrying out versioning management on package container cluster information and test task use cases, and triggering an automatic regression test pipeline task when new updates are submitted and combined; the container engine automatic testing tool can be operated on a tester node or in a container on the tester node, so that the problem of environmental dependence is reduced; the container engine automatic testing tool can be executed concurrently, namely, the continuous integrated pipeline task can be executed in a plurality of resource pools concurrently, so that errors and time of manual testing are saved to a certain extent. And the method is more valuable to public cloud platform applications with numerous resource pools.
It will be appreciated by persons skilled in the art that the foregoing description is a preferred embodiment of the invention, and is not intended to limit the invention, but rather to limit the invention to the specific embodiments described, and that modifications may be made to the technical solutions described in the foregoing embodiments, or equivalents may be substituted for elements thereof, for the purposes of those skilled in the art. All technical features of the present embodiment, which are included in the scope of the present invention, can be freely combined according to actual needs.
Finally, it should be noted that: the foregoing description is only illustrative of the preferred embodiments of the present invention, and although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications may be made to the embodiments described, or equivalents may be substituted for elements thereof, and any modifications, equivalents, improvements or changes may be made without departing from the spirit and principles of the present invention.
Claims (10)
1. An automated regression testing system for a container engine platform, comprising: the system comprises a code warehouse, a testing machine, a container engine platform and a container mirror image warehouse;
the code warehouse is used for storing the container engine platform information including the access address and the certificate file of the container engine cluster, and also storing the automatic test case set;
the testing machine is used for executing nodes of regression testing tasks;
the container engine platform refers to a cloud primary container engine arrangement cluster system to be tested, and comprises three master nodes and a plurality of node nodes, wherein the master nodes are operated with container engine platform control service related components, and the node nodes are operated with life cycles for managing workload when a container is operated;
the container mirror image warehouse is used for storing container mirror image files, and the container mirror image files are used for starting the containers.
2. The automated regression testing system of claim 1 wherein: the code warehouse adopts a gerit system or a gitlab system.
3. The automated regression testing system of claim 1 wherein: the test machine is a virtual machine or a physical server.
4. A container engine platform automated regression testing system according to claim 3, wherein: when the dock container runs, jenkins continuous integrated service and a container provided with a raply tool are operated on the tester;
the method comprises the steps that when a dock container runs, the dock container is used for running, a raply tool and a depending virtual environment are manufactured into a container mirror image, after a jenkins system triggers a test task, the container mirror image is pulled on a tester, and then a container for testing is started through the container mirror image;
jenkins continuous integrated service is used for monitoring code merging actions in a code warehouse so as to trigger a test task;
the container is provided with a rally tool and is used for expanding a rally test task set and executing regression verification of a container engine platform; the rally is a set of testing tools widely applied to cloud computing open source infrastructure platforms.
5. The automated regression testing system of claim 4 wherein: the cloud computing open source infrastructure employs test tools in the openstack platform.
6. The automated regression testing system of claim 4 wherein: the container engine platform node is a physical machine or a cloud host.
7. The automated regression testing system of claim 1 wherein: the container mirror warehouse is realized by adopting a docker registry system or a harbor system.
8. A method of verification based on the container engine platform automated regression testing system of any one of claims 1 to 7, characterized by: the method specifically comprises the following steps of;
step 1, when a certain new resource pool is online for container engine service or has function change operation, a manager submits newly created container engine cluster access authority information to a code warehouse, and when merging operation is performed, jenkins task flow is triggered;
step 2, the jenkins system issues a task to a testing machine, and registers a container engine cluster to be tested to a rack testing system;
step 3, starting to execute test cases, wherein the test case sets are pulled from a code warehouse, the raply receives the input yaml files of the test case sets, executes the test cases in the yaml files respectively, and stores the results in a database;
and step 4, after the test is finished, generating a test result report in an html format, and analyzing the execution condition of the test case set and the failed error stack information in detail from the test result report.
9. The method of claim 8, wherein in step 2, the registration information includes an access address and a key of the container engine cluster.
10. The method of claim 8, wherein in step 3, the set of test cases includes resources that cover lifecycle management tests of various resources in the container engine.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311657517.3A CN117873862A (en) | 2023-12-05 | 2023-12-05 | Automatic regression testing system and method for container engine platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311657517.3A CN117873862A (en) | 2023-12-05 | 2023-12-05 | Automatic regression testing system and method for container engine platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117873862A true CN117873862A (en) | 2024-04-12 |
Family
ID=90578024
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311657517.3A Pending CN117873862A (en) | 2023-12-05 | 2023-12-05 | Automatic regression testing system and method for container engine platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117873862A (en) |
-
2023
- 2023-12-05 CN CN202311657517.3A patent/CN117873862A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10983774B2 (en) | Extensions for deployment patterns | |
| US10394697B2 (en) | Focus area integration test heuristics | |
| US10360141B2 (en) | Automated application test system | |
| US7694181B2 (en) | Automated software testing framework | |
| US11385993B2 (en) | Dynamic integration of command line utilities | |
| US9218231B2 (en) | Diagnosing a problem of a software product running in a cloud environment | |
| US11762763B2 (en) | Orchestration for automated performance testing | |
| WO2019029160A1 (en) | Application construction method and system, computer device, and storage medium | |
| US10114861B2 (en) | Expandable ad hoc domain specific query for system management | |
| US11163668B2 (en) | Determine errors and anomalies in a flow process integration instance that integrates services | |
| US10146674B2 (en) | Plugin-based software verification system | |
| US9256509B1 (en) | Computing environment analyzer | |
| US11113186B1 (en) | Testing and publishing of resource handlers in a cloud environment | |
| WO2023094931A1 (en) | Detecting vulnerabilities in configuration code of a cloud environment utilizing infrastructure as code | |
| CN110494849B (en) | System and method for determining success of cross-platform application migration | |
| US20150339219A1 (en) | Resilient mock object creation for unit testing | |
| JP5400873B2 (en) | Method, system, and computer program for identifying software problems | |
| Huang et al. | Surrogate: A simulation apparatus for continuous integration testing in service oriented architecture | |
| CN117873862A (en) | Automatic regression testing system and method for container engine platform | |
| CN113220586A (en) | Automatic interface pressure test execution method, device and system | |
| US20240160559A1 (en) | Automated decoupling of unit tests | |
| US20230229581A1 (en) | Identifying regression test failures | |
| Benton et al. | Wallaby: A scalable semantic configuration service for grids and clouds | |
| Bernhardt | CI/CD Pipeline from Android to Embedded Devices with end-to-end testing based on Containers | |
| Schallnahs et al. | Challenges in Prototyping a Cloud-Native Billing Application for 5G with Stream Processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |