CN113157577A

CN113157577A - PaaS cloud platform fault testing method and device

Info

Publication number: CN113157577A
Application number: CN202110458534.9A
Authority: CN
Inventors: 张晓娜; 暨光耀; 张�浩; 傅媛媛
Original assignee: Industrial and Commercial Bank of China Ltd ICBC
Current assignee: Industrial and Commercial Bank of China Ltd ICBC
Priority date: 2021-04-27
Filing date: 2021-04-27
Publication date: 2021-07-23

Abstract

The application provides a failure testing method and device for a PaaS cloud platform, wherein the method comprises the following steps: acquiring architecture information and availability information of a tested application; selecting corresponding fault characteristic data from a pre-generated fault library according to the architecture information and the availability information, and generating a fault test case set according to the fault characteristic data; and injecting the fault test case set into the target fault point through an installation medium preset at the target fault point, and acquiring fault index data generated in the fault execution process to generate a fault test result. The method and the device for testing the faults of the PaaS cloud platform can accurately and efficiently generate the fault test cases and implement the fault test, obviously reduce the dependence of the fault test of the PaaS cloud platform on experience accumulation and technical capability of testers, and effectively reduce the labor input of the fault test of the PaaS cloud platform.

Description

PaaS cloud platform fault testing method and device

Technical Field

The application belongs to the technical field of system testing, and particularly relates to a method and a device for testing faults of a PaaS cloud platform.

Background

Paas (platform as a service), which is a cloud computing service model. The PaaS cloud platform is a cloud computing service platform constructed based on a PaaS mode, and delivers a server platform and a development environment to a user for use in a service form by flattening basic resources such as computing, network, storage, an operating system, database middleware and the like. In recent years, with the rapid development of PaaS cloud platform technology, the number of computing nodes of PaaS cloud has also increased rapidly, and some nodes of cloud service manufacturers have reached tens of thousands of scales. With the increase of computing nodes, the probability and randomness of fault occurrence become larger, and the demand for improving the reliability, high availability, robustness and the like of the system is more urgent.

At present, most of fault test cases of PaaS cloud platforms are manually designed, fault is manually injected, and index data is manually collected. Whether the design of the fault test case is reasonable and comprehensive and whether the fault injection form is correct depends on experience accumulation and technical capability of related testers to a great extent. In addition, the implementation of the fault case and the collection of the test indexes require enormous manpower.

Disclosure of Invention

Aiming at the problems in the prior art, the application provides a PaaS cloud platform fault testing method and device, so that the problem that a fault testing case in the prior art relates to and is injected depending on the experience of related testers is solved, and the fault testing case is designed more accurately and more efficiently and is implemented with fault testing.

According to a first aspect of the application, a method for testing a failure of a PaaS cloud platform is provided, which includes:

acquiring architecture information and availability information of a tested application;

selecting corresponding fault characteristic data from a pre-generated fault library according to the architecture information and the availability information to generate a fault test case set;

installing a mounting medium to a target fault point;

and injecting a mounting medium into the fault test case set, executing the fault, and acquiring fault index data generated in the fault execution process to generate a fault test result.

In one embodiment, the generating process of the fault library comprises:

collecting fault points and fault types of the application;

establishing a mapping relation between fault points and fault types, wherein one fault point corresponds to one or more fault types;

and generating a fault library according to the mapping relation.

In one embodiment, selecting corresponding fault feature data from a pre-generated fault library according to architecture information and availability information to generate a fault test case set, including:

determining fault points in fault characteristic data related to the tested application according to the architecture information;

determining the fault type in the fault characteristic data related to the application to be tested according to the availability information;

and generating a fault test case set according to the fault point and the fault type.

In one embodiment, inserting a mounting medium to a target failure point comprises:

sending an installation medium to a target fault point and installing;

and after the installation is finished, executing a high concurrent pressure test script to perform pressure test on the installation medium.

In one embodiment, injecting a mounting medium into a fault test case set and executing a fault, and collecting fault index data generated in the fault executing process to generate a fault test result, includes:

injecting all fault test cases in the fault test case set collection through the installation medium;

the execution operation of the fault test cases is cancelled after the fault test cases are sequentially executed for a preset time;

collecting fault index data when executing a fault test case and after canceling the fault test case;

and generating a fault test result according to the fault index data and sending the fault test result to the related terminal.

According to a second aspect of the present application, there is also provided a PaaS cloud platform fault testing apparatus, including:

the system comprises a characteristic information acquisition unit, a characteristic information acquisition unit and a characteristic information acquisition unit, wherein the characteristic information acquisition unit is used for acquiring the architecture information and the availability information of the application to be tested;

the fault test case set generating unit is used for selecting corresponding fault characteristic data from a pre-generated fault library according to the architecture information and the availability information to generate a fault test case set;

the medium inserting unit is used for inserting and installing a medium to a target fault point;

and the fault execution unit is used for injecting a fault test case set through the installation medium, executing the fault, and acquiring fault index data generated in the fault execution process to generate a fault test result.

In an embodiment, the apparatus for testing a failure of a PaaS cloud platform further includes a failure library generating apparatus, where the failure library generating apparatus includes:

the fault acquisition module is used for acquiring fault points and fault types of the applications;

the mapping relation establishing module is used for establishing a mapping relation between fault points and fault types, wherein one fault point corresponds to one or more fault types;

and the fault library generating module is used for generating a fault library according to the mapping relation.

In one embodiment, the failure test case set generating unit includes:

the fault point determining module is used for determining fault points in fault characteristic data related to the application to be tested according to the architecture information;

the fault type determining module is used for determining the fault type in the fault characteristic data related to the application to be tested according to the availability information;

and the test set generation module is used for generating a fault test case set according to the fault point and the fault type.

In one embodiment, the media insertion unit includes:

the sending module is used for sending the installation medium to a target fault point and installing the installation medium;

and the pressure test module is used for executing the high concurrent pressure test script after the installation so as to carry out pressure test on the installation medium.

In one embodiment, the fault execution unit includes:

the fault input module is used for injecting all fault test cases in the fault test case collection through the installation medium;

the fault execution module is used for sequentially executing the execution operation of canceling the fault test case after the fault test case reaches the preset time;

the data acquisition module is used for acquiring fault index data when executing the fault test case and after canceling the fault test case;

and the result generation module is used for generating a fault test result according to the fault index data and sending the fault test result to the related terminal.

According to a third aspect of the present application, there is provided an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the steps of the PaaS cloud platform fault testing method when executing the program.

According to a fourth aspect of the present application, there is also provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the PaaS cloud platform fault testing method.

According to the technical scheme, the method and the device for testing the faults of the PaaS cloud platform provided by the application comprise the following steps: acquiring architecture information and availability information of a tested application; selecting corresponding fault characteristic data from a pre-generated fault library according to the architecture information and the availability information to generate a fault test case set; installing a mounting medium to a target fault point; and injecting a mounting medium into the fault test case set, executing the fault, and acquiring fault index data generated in the fault execution process to generate a fault test result. A mapping relation is established between fault points and fault types to generate a fault library, generally, one fault point corresponds to multiple fault types, when a test case needs to be generated, corresponding or nearest fault characteristic data is directly searched from the fault library according to characteristic information of a tested application program to generate a fault test case set, the problem that most of the current fault test cases of the PaaS cloud platform are designed manually, the rationality of the design mainly depends on the experience of a designer is solved, the most reasonable fault test cases can be automatically generated without depending on the experience of technicians, time is saved, efficiency is improved, and the usability of the test cases is improved. According to the method and the device, the chaosblade installation medium is inserted into the target fault point, the fault test case set is injected into the target fault point and sequentially executed, the problem of artificial dependence of fault injection is solved, the injection execution effect is not changed due to artificial factors, the functions of accurately and efficiently generating the fault test case and implementing the fault test are realized, the dependence of PaaS cloud platform fault test on experience accumulation and technical capability of testers is remarkably reduced, and the manpower input of the PaaS cloud platform fault test is effectively reduced.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a flowchart of a failure testing method for a PaaS cloud platform according to the present application.

Fig. 2 is a flowchart of a generation process of a fault library in the embodiment of the present application.

Fig. 3 is a flowchart of a method for generating a fault test case set in the embodiment of the present application.

Fig. 4 is a flowchart of a method for inserting an installation medium to a target failure point in the embodiment of the present application.

FIG. 5 is a flowchart of a method for performing a fault and generating test results according to an embodiment of the present application.

Fig. 6 is a structural block diagram of a PaaS cloud platform fault testing apparatus provided in the present application.

Fig. 7 is a block diagram of a fault library generation apparatus according to an embodiment of the present application.

Fig. 8 is a block diagram of a structure of a failure test case set generating unit in the embodiment of the present application.

Fig. 9 is a block diagram of a medium insertion unit in the embodiment of the present application.

FIG. 10 is a block diagram of a fault execution unit according to an embodiment of the present application

Fig. 11 is a specific implementation of an electronic device in an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that the method and the device for testing the faults of the PaaS cloud platform disclosed by the application can be used in the field of financial technology, and can also be used in any field except the field of financial technology.

At present, most of fault test cases of PaaS cloud platforms are manually designed, fault is manually injected, and index data is manually collected. Whether the design of the case is reasonable and comprehensive and whether the fault injection form is correct depends on experience accumulation and technical capability of related testers to a great extent. In addition, the implementation of the case and the collection of the test indexes require enormous manpower.

Based on the above problems, in order to design a fault test case and implement a fault test more accurately and efficiently, the present application provides a PaaS cloud platform fault test method, a PaaS cloud platform fault test apparatus, an electronic device and a computer readable storage medium, respectively, a fault library is generated by establishing a mapping relationship between fault points and fault types, generally, one fault point corresponds to multiple fault types, when a test case needs to be generated, corresponding or most similar fault characteristic data is directly searched from the fault library according to characteristic information of an application program to be tested to generate a fault test case set, so as to replace the problem that most of the current fault test cases of the PaaS cloud platform are designed manually, the rationality of the design mainly depends on the experience of a designer, the most reasonable fault test cases can be automatically generated without depending on the experience of a technician, not only time is saved and efficiency is improved, the usability of the test case is also improved. According to the method and the device, the chaosblade installation medium is inserted into the target fault point, the fault test case set is injected into the target fault point and sequentially executed, the problem of artificial dependence of fault injection is solved, the injection execution effect is not changed due to artificial factors, the functions of accurately and efficiently generating the fault test case and implementing the fault test are realized, the dependence of PaaS cloud platform fault test on experience accumulation and technical capability of testers is remarkably reduced, and the manpower input of the PaaS cloud platform fault test is effectively reduced.

Based on the above content, the present application further provides a PaaS cloud platform fault testing apparatus for implementing the PaaS cloud platform fault testing method provided in one or more embodiments of the present application, where the PaaS cloud platform fault testing apparatus may be in communication connection with client devices by itself or through a third-party server, so as to receive fault testing requests respectively initiated by the client devices, and then trigger a program in the apparatus for implementing the PaaS cloud platform fault testing method provided in one or more embodiments of the present application to execute a fault testing step.

It is understood that the client devices may include smart phones, tablet electronic devices, network set-top boxes, portable computers, desktop computers, Personal Digital Assistants (PDAs), in-vehicle devices, smart wearable devices, and the like. Wherein, intelligence wearing equipment can include intelligent glasses, intelligent wrist-watch, intelligent bracelet etc..

In another practical application scenario, a part of the foregoing PaaS cloud platform fault testing apparatus performing the PaaS cloud platform fault test may be executed in the server as described above, or all operations may be completed in the client device. The selection may be specifically performed according to the processing capability of the client device, the limitation of the user usage scenario, and the like. This is not a limitation of the present application. If all the operations are completed in the client device, the client device may further include a processor for performing a specific process of automatic fault diagnosis in the batch service process.

The client device may have a communication module (i.e., a communication unit), and may be communicatively connected to a remote server to implement data transmission with the server. The server may include a server on the task scheduling center side, and in other implementation scenarios, the server may also include a server on an intermediate platform, for example, a server on a third-party server platform that is communicatively linked to the task scheduling center server. The server may include a single computer device, or may include a server cluster formed by a plurality of servers, or a server structure of a distributed apparatus.

The server and the client device may communicate using any suitable network protocol, including network protocols not yet developed at the filing date of this application. The network protocol may include, for example, a TCP/IP protocol, a UDP/IP protocol, an HTTP protocol, an HTTPS protocol, or the like. Of course, the network Protocol may also include, for example, an RPC Protocol (Remote Procedure Call Protocol), a REST Protocol (Representational State Transfer Protocol), and the like used above the above Protocol.

The following embodiments and application examples are specifically and individually described in detail.

In order to design a fault test case and implement a fault test more accurately and efficiently, the present application provides a method for testing a fault of a PaaS cloud platform, as shown in fig. 1, including:

s101: and acquiring the architecture information and the availability information of the application to be tested.

In an implementation manner of S101, a user may trigger a failure testing process of the PaaS cloud platform provided in the embodiment of the present application, which specifically may be: the method comprises the steps of receiving a fault test request sent by a client, wherein the fault test request comprises characteristic information of an application program to be tested, and the characteristic information comprises architecture characteristic information and availability characteristic information.

In another implementation of S101, the automated fault test may be initiated at a time point set by a user at which the fault test is automatically performed.

S102: and selecting corresponding fault characteristic data from a pre-generated fault library according to the architecture information and the availability information, and generating a fault test case set according to the fault characteristic data.

It can be understood that, appropriate data is searched from a fault library according to the architecture information and the availability information of the current application program to be tested to generate a fault test case set, and the fact in the fault library is the sum of the fault point and the mapping relation of all fault types which may occur at the fault point. The fault library is established in advance according to the recorded historical information.

According to the method, the fault base is generated by establishing the mapping relation between the fault points and the fault types, generally, one fault point corresponds to multiple fault types, when the test case needs to be generated, the corresponding or nearest fault characteristic data is directly searched from the fault base according to the characteristic information of the tested application program to generate the fault test case set, the problem that most of the current fault test cases of the PaaS cloud platform are designed manually, the rationality of the design mainly depends on the experience of a designer is solved, the most reasonable fault test cases can be automatically generated without depending on the experience of technical personnel, the time is saved, the efficiency is improved, and the usability of the test cases is improved.

S103: and injecting the fault test case set into a target fault point through an installation medium preset at the target fault point, and acquiring fault index data generated in the fault execution process to generate a fault test result.

As can be appreciated, performing the failure includes sending a chaosblade installation medium to the target failure point, installing the chaosblade medium, executing a high concurrency stress test script, executing a command to inject the failure, executing a command to undo the failure. The chaosblade installation medium is a tool for injecting faults, provides rich fault scenes, can be used for injecting the faults, helps a distributed system to improve the fault tolerance and high availability of the system, and avoids troubles caused by manual fault injection.

It can be understood that according to different requirements, the corresponding fault command is executed, and the corresponding fault is injected. The failure command is provided by the chaosbelde medium. And injecting the fault, after the observation system performs for a period of time, canceling the fault, and comparing the observation system performance so as to know whether the system can recover to normal performance after the fault is removed. In the process, fault index data are collected and sent to application testers, so that the application testers can analyze the fault tolerance and high availability of the system according to the index data. System indexes including the number of concurrent users, response time, throughput, success rate and the like are acquired through a LoadRunner and Jmeter pressure testing tool. And acquiring resource indexes including a CPU (central processing unit), a memory, the current thread number, a disk IO (input/output) and the like of a server or a container by using the Nmon resource monitoring tool.

By inserting the chaosblade installation medium at the target fault point, the fault test case set is injected into the target fault point and sequentially executed, the problem of manual dependence of fault injection is solved, the injection execution effect is not changed due to manual factors, the functions of accurately and efficiently generating the fault test case and implementing the fault test are realized, the dependence of the PaaS cloud platform fault test on experience accumulation and technical capability of testers is remarkably reduced, and the labor input of the PaaS cloud platform fault test is effectively reduced.

In an embodiment, the present application further provides a generation process of a fault library, as shown in fig. 2, including:

s201: and collecting fault points and fault types of the application.

S202: and establishing a mapping relation between the fault points and the fault types, wherein one fault point corresponds to one or more fault types.

S203: and generating a fault library according to the mapping relation.

In a specific embodiment, a failure expert library (i.e., failure library) is first formed by organically fusing the failure types and failure points. The types of faults include: the method comprises the following steps of abnormal CPU utilization rate, abnormal memory utilization rate, busy disk IO, low read-write rate of the disk IO, full disk space, network delay, network jitter, unavailable network port, network packet loss, network packet retransmission and the like. The fault point comprises: the system layer comprises a PaaS container, a database server and a file server; the cache layer comprises an NOS cache, a Redis cache and the like; the message middleware layer comprises KAFKA, MQ and the like; the routing layer includes F5, SLB, etc. And organically fusing the fault types and the fault points to form a fault expert library, wherein the fault expert library comprises the corresponding relation between the fault points and the fault types, and in practice, one fault point usually corresponds to a plurality of fault types. As shown in table 1 below:

TABLE 1

From the above description, it can be known that the PaaS cloud platform fault test method provided by the application generates the fault library by establishing the mapping relationship between the fault point and the fault type, generally, one fault point corresponds to multiple fault types, when a test case needs to be generated, the corresponding or most similar fault characteristic data is directly searched from the fault library according to the characteristic information of the tested application program to generate the fault test case set, and the problem that most of the current fault test cases of the PaaS cloud platform are designed manually, and the rationality of the design mainly depends on the experience of a designer is solved, the most reasonable fault test cases can be automatically generated without depending on the experience of technical staff is solved, time is saved, efficiency is improved, and the usability of the test cases is also improved.

In order to further implement how to generate the fault test case set, in an embodiment of the method provided in the present application, the method selects corresponding fault feature data from a pre-generated fault library according to the architecture information and the availability information to generate the fault test case set, as shown in fig. 3, includes:

s301: and determining a fault point in fault characteristic data related to the tested application according to the architecture information.

S302: and determining the fault type in the fault characteristic data related to the application to be tested according to the availability information.

S303: and generating a fault test case set according to the fault point and the fault type.

In one embodiment, the application tester enters the architectural features (architectural information), high availability features (availability information), and test scripts of the application system.

Preferably, the architectural features of the application include: container nodes, database servers, caches, timeout times, routing layer servers, middleware, etc. involved in the application.

Preferably, the highly available features of the application include: the method comprises the following steps of container abnormal restart self-healing, container expansion, container contraction, container disk self-isolation, database master and standby switching, slow SQL (structured query language) checking and killing of a database, a fusing mechanism for upstream and downstream service calling, an overtime mechanism for upstream and downstream calling, a current limiting mechanism, an NOS (nitric oxide synthase) cache fusing mechanism, a KAKFA fusing mechanism, a load balancing mechanism and the like.

In order to simulate the server pressure of the system in the actual production environment, a high-frequency transaction is selected, and a pressure test script of the high-frequency transaction is debugged through LoadRunner or Jemeter. The following takes the personal marketing system as an example:

the individual marketing system has the structural characteristics that:

(1) service access layer, 1 SET, 4 containers under each SET

(2) Service processing layer, the layer has two SET, 6 containers under each SET

(3) Cloud MYSQL database

(4) Will call NOS cache server in service access layer

(5) Upstream and downstream timeout time was set to 8 seconds

The personal marketing system has the characteristics of high availability:

(1) the service access layer performs load balancing through the SLB, and when the container is abnormal, the container can restart the self-healing

(2) The service processing layer performs load balancing through the SLB, and when the container is abnormal, the container can restart the self-healing

(3) Cloud MYSQL master and backup database

(4) When 70% of transactions calling the NOS cache fail to access the NOS cache, punishing that the NOS cache is fused and does not call the NOS cache any more

(5) And the upstream and downstream are provided with time-out settings, and when the calling downstream exceeds 8 seconds, the service time-out returns.

The fault testing device selects proper fault points and fault types from the fault expert database according to the applied architecture characteristics and the high availability characteristics, and generates a fault testing case set. The cases in the generated failure test case set include: fault point, high availability characteristics, fault type and the like.

According to the recorded architecture characteristics and high availability characteristics of the personal marketing system, appropriate fault points and fault types are selected from the expert database, and a fault test case set is generated as shown in the following table 2:

TABLE 2

Therefore, when a test case needs to be generated, the corresponding or most similar fault feature data is directly searched from the fault library according to the feature information of the tested application program to generate a fault test case set, and the problem that most of the current fault test cases of the PaaS cloud platform are designed manually, and the reasonability of the design mainly depends on the experience of a designer is solved.

In one embodiment, inserting the installation medium to the target failure point, as shown in fig. 4, includes:

s401: and sending the installation medium to the target failure point and installing.

S402: and after the installation is finished, executing a high concurrent pressure test script to perform pressure test on the installation medium.

In an embodiment, injecting a mounting medium into a fault test case set and executing a fault, and collecting fault index data generated during the execution of the fault to generate a fault test result, as shown in fig. 5, includes:

s501: all fault test cases in the collection of fault test cases are injected by the installation medium.

S502: and sequentially executing the fault test cases, and canceling the execution operation of the fault test cases after the preset time is reached.

S503: and collecting fault index data when executing the fault test case and after canceling the fault test case.

S504: and generating a fault test result according to the fault index data and sending the fault test result to the related terminal.

In a particular embodiment, performing the fault includes sending a chaosblade installation medium to the target failure point, installing the chaosblade medium, executing a high concurrent pressure test script, executing a command to inject the fault, and executing a command to undo the fault.

Preferably, in order to simulate the server pressure of the system in the actual production environment, a high-frequency transaction is selected, and a pressure test script of the high-frequency transaction is debugged through LoadRunner or meter.

Preferably, the relevant resource data is collected, including system indexes (number of concurrent users, response time, throughput, success rate, etc.), and resource indexes (CPU, memory, current thread number, disk IO, etc.).

And sending the fault test result to relevant developers and testers so that the developers and the testers can analyze whether the performance of the system for dealing with the fault is in accordance with expectation according to the test result.

Preferably, the test result content is more, the fault test result can be sent to the relevant developers and testers in the form of an email, and the identification result is composed of the following data information:

And setting the time for automatically executing the fault test.

Preferably, the fault testing system will implement the automated fault test case at a set point in time when the fault test is automatically performed.

In a specific embodiment, the installing the medium to the target failure point specifically includes:

s1: and sending the chaosblade installation medium to the target failure point.

Preferably, the chaosblade is an open-source fault injection tool, provides implementation of rich fault scenes, can be used for injecting faults, and helps a distributed system to improve fault tolerance and high availability of the system.

Preferably, the target failure point is to simulate a failed server or container.

Preferably, the service access layer container, the service processing layer container, the NOS cache server, and the database server in the personal marketing system of the embodiment described in this patent are all failure points.

And S2, installing the chaosblade medium.

Preferably, a chaosblade medium is installed at the target failure point to initiate the failure injection command.

S3: a high concurrent stress test script is executed.

Preferably, a high concurrent pressure test script is executed, mainly to simulate the production of high concurrent transaction traffic. The pressure test is typically initiated by a LoadRunner or meter pressure test tool.

S4: a command to inject a fault is executed.

Preferably, according to different requirements, corresponding fault commands are executed, and corresponding faults are injected. The failure command is provided by the chaosbelde medium.

The command format is as follows: blank create cpu/mem/network/disk/process

S5: a command to undo the failure is executed.

Preferably, after the fault is injected and the system performance is observed for a period of time, the fault needs to be cancelled, and the system performance is compared to see whether the system can recover to normal performance after the fault is resolved.

The command to undo the failure is as follows: blade-defined uid

And S6, collecting relevant index data.

Preferably, relevant index data is collected and sent to application testers, so that the application testers can analyze the fault tolerance and high availability of the system according to the index data.

Preferably, system indexes including the number of concurrent users, response time, throughput, success rate and the like are acquired through LoadRunner and Jmeter pressure testing tools. And acquiring resource indexes including a CPU (central processing unit), a memory, the current thread number, a disk IO (input/output) and the like of a server or a container by using the Nmon resource monitoring tool.

S7: the chaosblade installation medium is deleted from the target failure point.

After the failure test is finished, the chaosblade installation medium needs to be deleted from the target failure point, and the test environment is restored to the original state.

In the fault injection execution link, the method specifically includes: the fault testing system will start the automatic fault testing at the time point of the automatic fault testing execution set by the user. And checking the state of each fault point according to the fault test case set, wherein the state comprises a fault point communicable state and a fault point incommunicable state. When the state of the fault point is the state of no communication, the test case is removed from the fault test case set, and a record is recorded: failure point | non-communicable state. And after all fault points are checked, the case set with the abnormal fault point state removed is named as a feasible fault test case set, and the case set with the abnormal fault point state is named as an abnormal fault test case set. And implementing the fault cases according to the sequence of the feasible fault test case set, wherein each case comprises a fault, and implementing the faults in sequence according to the cases in the feasible fault test case set. Collecting resource indexes before fault implementation, during fault implementation and after fault cancellation, integrating collected index data to form a fault test result set, wherein each result consists of the following data information: fault point | high availability characteristics | fault type | transaction name | system indicators (number of concurrent users, response time, throughput, success rate, etc.) | resource indicators (CPU, memory, current thread number, disk IO, etc.). And sending a test result set to an application tester, wherein the test result set comprises a fault test result set and an abnormal fault test case set.

Based on the same inventive concept, the embodiment of the present application further provides a device for testing a failure of a PaaS cloud platform, which can be used to implement the method described in the above embodiments, as described in the following embodiments. The principle of solving the problems of the PaaS cloud platform fault testing device is similar to that of a PaaS cloud platform fault testing method, so that the implementation of the PaaS cloud platform fault testing device can refer to the implementation of the PaaS cloud platform fault testing method. As used hereinafter, the term "unit" or "module" may be a combination of software and/or hardware that implements a predetermined function. While the system described in the embodiments below is preferably implemented in software, implementations in hardware, or a combination of software and hardware are also possible and contemplated.

The application provides a PaaS cloud platform fault testing device, as shown in fig. 6, includes:

the characteristic information obtaining unit 601 is configured to obtain architecture information and availability information of the application under test.

In an implementation manner of the feature information obtaining unit 601, a user may trigger a failure testing process of the PaaS cloud platform provided in the embodiment of the present application, which specifically may be: the method comprises the steps of receiving a fault test request sent by a client, wherein the fault test request comprises characteristic information of an application program to be tested, and the characteristic information comprises architecture characteristic information and availability characteristic information.

In another implementation of the characteristic information obtaining unit 601, the automated failure test may be started at a time point set by a user at which the failure test is automatically performed.

The failure test case set generating unit 602 is configured to select corresponding failure feature data from a pre-generated failure library according to the architecture information and the availability information to generate a failure test case set.

It can be understood that the failure test case set generating unit 602 searches suitable data from a failure library according to the architecture information and the availability information of the current application to be tested to generate a failure test case set, where the failure library is actually the sum of the failure point and the mapping relationship of all the failure types that may occur at the failure point. The fault library is established in advance according to the recorded historical information.

And a medium inserting unit 603 for inserting the mounting medium to the target failure point.

It is understood that the media plugin unit 603 performs failures, including sending a chaosblade installation media to the target failure point, installing the chaosblade media, executing a high concurrent pressure test script, executing a command to inject a failure, executing a command to undo a failure. The chaosblade installation medium is a tool for injecting faults, provides rich fault scenes, can be used for injecting the faults, helps a distributed system to improve the fault tolerance and high availability of the system, and avoids troubles caused by manual fault injection.

And the fault execution unit 604 is configured to inject a fault test case set through the installation medium, execute a fault, and acquire fault index data generated in a fault execution process to generate a fault test result.

It is understood that the fault execution unit 604 executes the corresponding fault command according to different requirements, and injects the corresponding fault. The failure command is provided by the chaosbelde medium. And injecting the fault, after the observation system performs for a period of time, canceling the fault, and comparing the observation system performance so as to know whether the system can recover to normal performance after the fault is removed. In the process, fault index data are collected and sent to application testers, so that the application testers can analyze the fault tolerance and high availability of the system according to the index data. System indexes including the number of concurrent users, response time, throughput, success rate and the like are acquired through a LoadRunner and Jmeter pressure testing tool. And acquiring resource indexes including a CPU (central processing unit), a memory, the current thread number, a disk IO (input/output) and the like of a server or a container by using the Nmon resource monitoring tool.

In an embodiment, the apparatus for testing a PaaS cloud platform fault further includes a fault library generating apparatus, as shown in fig. 7, the fault library generating apparatus includes:

a fault collection module 701, configured to collect a fault point and a fault type of an application;

a mapping relationship establishing module 702, configured to establish a mapping relationship between fault points and fault types, where one fault point corresponds to one or more fault types;

and a fault library generating module 703, configured to generate a fault library according to the mapping relationship.

In a specific embodiment, a failure expert library (i.e., failure library) is first formed by organically fusing the failure types and failure points. The types of faults include: the method comprises the following steps of abnormal CPU utilization rate, abnormal memory utilization rate, busy disk IO, low read-write rate of the disk IO, full disk space, network delay, network jitter, unavailable network port, network packet loss, network packet retransmission and the like. The fault point comprises: the system layer comprises a PaaS container, a database server and a file server; the cache layer comprises an NOS cache, a Redis cache and the like; the message middleware layer comprises KAFKA, MQ and the like; the routing layer includes F5, SLB, etc. And organically fusing the fault types and the fault points to form a fault expert library, wherein the fault expert library comprises the corresponding relation between the fault points and the fault types, and in practice, one fault point usually corresponds to a plurality of fault types.

From the above description, the PaaS cloud platform fault testing device provided by the application generates the fault library by establishing the mapping relationship between the fault point and the fault type, generally, one fault point corresponds to multiple fault types, when a test case needs to be generated, the corresponding or most similar fault characteristic data is directly searched from the fault library according to the characteristic information of the tested application program to generate the fault test case set, the problem that most of the current fault test cases of the PaaS cloud platform are designed manually, the rationality of the design mainly depends on the experience of a designer is solved, the most reasonable fault test cases can be automatically generated without depending on the experience of technical staff is solved, time is saved, efficiency is improved, and the usability of the test cases is improved.

In an embodiment, as shown in fig. 8, the failure test case set generating unit 602 includes:

a fault point determining module 801, configured to determine a fault point in fault feature data related to the application to be tested according to the architecture information;

a fault type determining module 802, configured to determine a fault type in fault feature data related to the application to be tested according to the availability information;

and the test set generating module 803 is configured to generate a fault test case set according to the fault point and the fault type.

In one embodiment, an application tester inputs the architecture characteristics (architecture information), high availability characteristics (availability information) and a test script of an application system in a PaaS cloud platform fault testing device.

In order to simulate the server pressure of the system in the actual production environment, a high-frequency transaction is selected, and a pressure test script of the high-frequency transaction is debugged through LoadRunner or Jemeter.

In one embodiment, as shown in fig. 9, the medium inserting unit 603 includes:

a sending module 901, configured to send an installation medium to a target failure point and install the installation medium;

and the pressure test module 902 is used for executing a high concurrent pressure test script after the installation is finished so as to perform pressure test on the installation medium.

In a specific embodiment, the media insertion unit 603 is configured to perform the following steps:

s1: and sending the chaosblade installation medium to the target failure point.

And S2, installing the chaosblade medium.

S3: a high concurrent stress test script is executed.

In one embodiment, as shown in fig. 10, the fault performing unit 1001 includes:

a fault input module 1001 for injecting all fault test cases in the fault test case collection through the installation medium;

the fault execution module 1002 is configured to sequentially execute the execution operations of cancelling the fault test cases after the fault test cases reach a preset time;

the data acquisition module 1003 is used for acquiring fault index data during execution of the fault test case and after cancellation of the fault test case;

and the result generating module 1004 is configured to generate a fault test result according to the fault index data and send the fault test result to the relevant terminal.

In an embodiment, the fault performing unit 1001 is configured to perform the following steps:

s4: a command to inject a fault is executed.

The command format is as follows: blank create cpu/mem/network/disk/process

S5: a command to undo the failure is executed.

The command to undo the failure is as follows: blade-defined uid

And S6, collecting relevant index data.

The method and the device for testing the faults of the PaaS cloud platform can accurately and efficiently generate the fault test cases and implement the fault test, obviously reduce the dependence of the fault test of the PaaS cloud platform on experience accumulation and technical capability of testers, and effectively reduce the labor input of the fault test of the PaaS cloud platform.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

The principle and the implementation mode of the present application are explained by applying specific embodiments in the present application, and the description of the above embodiments is only used to help understanding the method and the core idea of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

An embodiment of the present application further provides a specific implementation manner of an electronic device, which is capable of implementing all steps in the method in the foregoing embodiment, and referring to fig. 11, the electronic device specifically includes the following contents:

a processor (processor)1101, a memory 1102, a Communications Interface 1103, a bus 1104, and a non-volatile memory 1105;

the processor 1101, the memory 1102 and the communication interface 1103 complete mutual communication through the bus 1104;

the processor 1101 is configured to call the computer programs in the memory 1102 and the nonvolatile memory 1105, and when the processor executes the computer programs, the processor implements all the steps in the method in the foregoing embodiments, for example, when the processor executes the computer programs, the processor implements the following steps:

Embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps of the method in the above embodiments, where the computer-readable storage medium stores thereon a computer program, and the computer program when executed by a processor implements all the steps of the method in the above embodiments, for example, the processor implements the following steps when executing the computer program:

The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the hardware + program class embodiment, since it is substantially similar to the method embodiment, the description is simple, and the relevant points can be referred to the partial description of the method embodiment. Although embodiments of the present description provide method steps as described in embodiments or flowcharts, more or fewer steps may be included based on conventional or non-inventive means. The order of steps recited in the embodiments is merely one manner of performing the steps in a multitude of orders and does not represent the only order of execution. When an actual apparatus or end product executes, it may execute sequentially or in parallel (e.g., parallel processors or multi-threaded environments, or even distributed data processing environments) according to the method shown in the embodiment or the figures. The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, the presence of additional identical or equivalent elements in a process, method, article, or apparatus that comprises the recited elements is not excluded. For convenience of description, the above devices are described as being divided into various modules by functions, and are described separately. Of course, in implementing the embodiments of the present description, the functions of each module may be implemented in one or more software and/or hardware, or a module implementing the same function may be implemented by a combination of multiple sub-modules or sub-units, and the like. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form. The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks. As will be appreciated by one skilled in the art, embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, embodiments of the present description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present description may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein. The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment. In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of an embodiment of the specification. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction. The above description is only an example of the embodiments of the present disclosure, and is not intended to limit the embodiments of the present disclosure. Various modifications and variations to the embodiments described herein will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present specification should be included in the scope of the claims of the embodiments of the present specification.

Claims

1. A failure testing method for a PaaS cloud platform is characterized by comprising the following steps:

selecting corresponding fault characteristic data from a pre-generated fault library according to the architecture information and the availability information, and generating a fault test case set according to the fault characteristic data;

and injecting the fault test case set into a target fault point through an installation medium preset at the target fault point, and acquiring fault index data generated in the fault execution process to generate a fault test result.

2. The PaaS cloud platform fault testing method according to claim 1, wherein the generation process of the fault library comprises:

collecting fault points and fault types of the application;

and generating a fault library according to the mapping relation.

3. The PaaS cloud platform fault testing method according to claim 2, wherein the selecting corresponding fault feature data from a pre-generated fault library according to the architecture information and availability information, and generating a fault testing case set according to the fault feature data comprises:

determining fault points in the fault characteristic data related to the tested application according to the architecture information;

4. The PaaS cloud platform fault testing method according to claim 3, wherein the step of setting an installation medium at a target fault point comprises:

sending the installation medium to a target fault point and installing;

5. The PaaS cloud platform fault testing method according to claim 4, wherein the step of injecting the fault testing case set into a target fault point through an installation medium preset at the target fault point and acquiring fault index data generated in a fault execution process to generate a fault testing result comprises the steps of:

injecting all fault test cases in the fault test case collection through the installation medium;

sequentially executing the fault test cases, and cancelling the execution operation of the fault test cases after the preset time is reached;

collecting fault index data when the fault test case is executed and after the fault test case is cancelled;

and generating a fault test result according to the fault index data and sending the fault test result to a related terminal.

6. The PaaS cloud platform fault testing method according to claim 1, wherein the step of injecting the fault testing case set into a target fault point through an installation medium preset at the target fault point and acquiring fault index data generated in a fault execution process to generate a fault testing result comprises:

7. The PaaS cloud platform fault testing method according to claim 1, wherein the selecting corresponding fault feature data from a pre-generated fault library according to the architecture information and availability information, and generating a fault testing case set according to the fault feature data comprises:

8. A PaaS cloud platform fault testing device is characterized by comprising:

and the fault execution unit is used for injecting the fault test case set through the installation medium, executing the fault, and acquiring fault index data generated in the fault execution process to generate a fault test result.

9. The apparatus for testing PaaS cloud platform failures according to claim 8, further comprising a failure library generating apparatus, wherein the failure library generating apparatus comprises:

10. The PaaS cloud platform fault testing apparatus of claim 9, wherein the media insertion unit comprises:

and the pressure test module is used for executing a high concurrent pressure test script after the installation so as to carry out pressure test on the installation medium.

11. The PaaS cloud platform fault testing apparatus of claim 10, wherein the fault execution unit comprises:

the data acquisition module is used for acquiring fault index data when the fault test case is executed and after the fault test case is cancelled;

12. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements the steps of the PaaS cloud platform fault testing method of any of claims 1 to 7.

13. A computer readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the PaaS cloud platform fault testing method according to any one of claims 1 to 7.