CN112463616A - Chaos testing method and device for Kubernetes container platform - Google Patents
Chaos testing method and device for Kubernetes container platform Download PDFInfo
- Publication number
- CN112463616A CN112463616A CN202011401478.7A CN202011401478A CN112463616A CN 112463616 A CN112463616 A CN 112463616A CN 202011401478 A CN202011401478 A CN 202011401478A CN 112463616 A CN112463616 A CN 112463616A
- Authority
- CN
- China
- Prior art keywords
- chaos
- application
- kill
- pod
- dropped
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 28
- 238000000034 method Methods 0.000 claims abstract description 42
- 230000008569 process Effects 0.000 claims abstract description 18
- 238000004590 computer program Methods 0.000 claims description 9
- 238000010586 diagram Methods 0.000 description 13
- 230000000739 chaotic effect Effects 0.000 description 8
- 238000004891 communication Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 230000008859 change Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000011084 recovery Methods 0.000 description 4
- 230000006399 behavior Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 239000008186 active pharmaceutical agent Substances 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000034994 death Effects 0.000 description 2
- 231100000517 death Toxicity 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000002347 injection Methods 0.000 description 1
- 239000007924 injection Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000009662 stress testing Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000010998 test method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3664—Environments for testing or debugging software
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3696—Methods or tools to render software testable
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a chaos testing method and device for a Kubernetes container platform, and relates to the technical field of chaos engineering testing. One embodiment of the method comprises: deploying a corresponding k8s-chaos config map configmap in a namespace in which k8s-chaos is to be run, so that k8s-chaos is run as a k8s application in a kubernets cluster; in the scheduling process, generating a k8s application program list meeting the conditions; for the k8s application in the k8s application list, it is determined whether the pod of the k8s application should be dropped by kill. The implementation method can solve the technical problem that the conventional test cannot comprehensively cope with the Kubernets cluster running on a complex network and architecture foundation and various applications on the cluster.
Description
Technical Field
The invention relates to the technical field of chaotic engineering testing, in particular to a chaotic testing method and device for a Kubernetes container platform.
Background
As the focus of cloud development has shifted to containers, kubernets et al container related technology has become popular. As a tester, our main goal is to ensure that the product gets a full quality test, ensuring a high quality delivery of the product, and in order to achieve these goals, unit testing, interface testing, integration testing, stress testing, etc. are needed in order to discover unpredictable behavior and ensure that the pattern of testing does not lead to errors.
Typically, the kubernets platform bottom contains many components, but unit and integration tests often do not cover them completely, and some servers and components may still pull the entire system into deep brillouin in the event of a failure. Meanwhile, various faults can occur anytime and anywhere, and many faults cannot be avoided, such as sudden writing of a disk, sudden network disconnection and power failure of a machine room, and the like. These failures can cause the kubernets system to collapse and be difficult to recover.
In the process of implementing the invention, the inventor finds that at least the following problems exist in the prior art:
conventional tests cannot fully address various applications running on and above kubernets clusters of complex networks and architectural infrastructures.
Disclosure of Invention
In view of this, embodiments of the present invention provide a chaos testing method and apparatus for a kubernets container platform, so as to solve the technical problem that conventional testing cannot fully cope with kubernets clusters operating on complex networks and architecture bases and various applications on the clusters.
In order to achieve the above object, according to an aspect of an embodiment of the present invention, there is provided a chaos testing method for a kubernets container platform, including:
deploying a corresponding k8s-chaos config map configmap in a namespace in which k8s-chaos is to be run, so that k8s-chaos is run as a k8s application in a kubernets cluster;
in the scheduling process, generating a k8s application program list meeting the conditions;
for the k8s application in the k8s application list, it is determined whether the pod of the k8s application should be dropped by kill.
Optionally, the namespace is kube-system.
Optionally, for the k8s application in the k8s application list, determining whether the pod of the k8s application should be dropped by kill includes:
for the k8s application in the k8s application list, a biased coin is flipped to determine if the pod of the k8s application should be dropped by kill.
Optionally, the coin is biased by k8 s-chaos/mtbf.
Optionally, to determine whether the pod of the k8s application should be dropped by kill, includes:
a random time is calculated to determine when the pod of the k8s application will be dropped by Kill.
Optionally, the method further comprises:
when the random time is reached, checking whether the k8s application still meets the condition, if yes, checking whether the k8s application updates the kill mode and kill value.
Optionally, the conditions include at least one of:
unselected, blacklisted, or deleted from the whitelist.
Optionally, after checking whether the k8s application program updates the kill mode and the kill value, the method further includes:
if yes, executing the Pod according to the kill mode and the kill value.
Optionally, the k8s-chaos is configured by an environment variable or a toml file located at/etc/k 8s-chaos/config.
Optionally, before deploying the corresponding k8s-chaos config map configmap in the namespace where k8s-chaos is to be run, the method further includes:
creating k8s-chaos config map configmap;
a Kubernetes cluster is deployed.
In addition, according to another aspect of the embodiments of the present invention, there is provided a chaos device facing a kubernets container platform, including:
a deployment module, configured to deploy a corresponding k8s-chaos config map in a namespace where k8s-chaos is to be run, so that k8s-chaos is run as a k8s application in a kubernets cluster;
the scheduling module is used for generating a k8s application program list meeting the conditions in the scheduling process;
a determination module to determine, for the k8s application in the k8s list of applications, whether the pod of the k8s application should be dropped by kill.
Optionally, the namespace is kube-system.
Optionally, the determining module is further configured to:
for the k8s application in the k8s application list, a biased coin is flipped to determine if the pod of the k8s application should be dropped by kill.
Optionally, the coin is biased by k8 s-chaos/mtbf.
Optionally, the determining module is further configured to:
a random time is calculated to determine when the pod of the k8s application will be dropped by Kill.
Optionally, the determining module is further configured to:
when the random time is reached, checking whether the k8s application still meets the condition, if yes, checking whether the k8s application updates the kill mode and kill value.
Optionally, the conditions include at least one of:
unselected, blacklisted, or deleted from the whitelist.
Optionally, the determining module is further configured to:
and after checking whether the k8s application program updates the kill mode and the kill value, if so, executing the Pod according to the kill mode and the kill value.
Optionally, the k8s-chaos is configured by an environment variable or a toml file located at/etc/k 8s-chaos/config.
Optionally, the deployment module is further configured to:
before deploying the corresponding k8s-chaos config map configmap in the namespace where k8s-chaos is to be run, creating k8s-chaos config map configmap;
a Kubernetes cluster is deployed.
According to another aspect of the embodiments of the present invention, there is also provided an electronic device, including:
one or more processors;
a storage device for storing one or more programs,
when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any of the embodiments described above.
According to another aspect of the embodiments of the present invention, there is also provided a computer readable medium, on which a computer program is stored, which when executed by a processor implements the method of any of the above embodiments.
One embodiment of the above invention has the following advantages or benefits: the technical means that the corresponding k8s-chaos config map is deployed in the namespace where k8s-chaos is to be run, so that k8s-chaos is run in a Kubernets cluster as a k8s application program, a k8s application program list meeting conditions is generated in the scheduling process, and whether the pod of the k8s application program should be dropped by kill is determined for the k8s application program in the k8s application program list is adopted, so that the technical problem that conventional tests in the prior art cannot fully cope with Kubernets cluster running on complex networks and architecture bases and various applications on the cluster is solved. The tester can directly and effectively obtain the resource change condition and the occurrence reason in the cluster operation from the front end. k8s-chaos is a tool following the principles of chaotic engineering that can randomly delete kubernets pod, check if the service is fail-resistant and help maintain the healthy operation of the relevant system. k8s-chaos could also complete the configuration via a yaml file that not only can terminate the specified application, but also can determine the execution time of the recovery policy.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of the main flow of a chaos testing method for a Kubernetes container platform according to an embodiment of the invention;
fig. 2 is a schematic diagram of config. toml file for k8s-chaos according to an embodiment of the present invention;
FIG. 3 is a schematic illustration of the environment variables of k8s-chaos according to an embodiment of the present invention;
FIG. 4 is a flow chart of a k8s-chaos scenario according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of the opt-in mode of k8s-chaos in accordance with an embodiment of the present invention;
FIG. 6 is a schematic diagram of the main modules of a chaos device oriented to a Kubernets container platform according to an embodiment of the invention;
FIG. 7 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
fig. 8 is a schematic structural diagram of a computer system suitable for implementing a terminal device or a server according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic diagram of a main flow of a chaos testing method for a kubernets container platform according to an embodiment of the present invention. As an embodiment of the present invention, as shown in fig. 1, the chaos testing method for a kubernets container platform may include:
step 101, deploying a corresponding k8s-chaos config map in a namespace where k8s-chaos is to be run, so that k8s-chaos is run as a k8s application in a kubernets cluster.
Optionally, the namespace is kube-system. First, a corresponding k8s-chaos config map is deployed in the namespace (e.g., kube-system) where k8s-chaos is to be run, so that k8s-chaos is run as a k8s application in a kubernets cluster. Note that in this step, it is ensured that the keyname is defined as config.
Kubernets is a container orchestration platform that is widely used in microservice deployments. The chaos engineering practice is helpful for the construction of an elastic system, and in order to more conveniently and comprehensively verify the tolerance capability of a Kubernets system platform to various faults, a Kubernets-based chaos test k8s-chaos is carried out.
Optionally, before step 101, the method further includes: creating k8s-chaos config map configmap; a Kubernetes cluster is deployed. Regarding the configuration of k8s-chaos, k8s-chaos is configured by environment variables or a toml file at/etc/k 8s-chaos/config. Configuration keys and descriptions can be found in config/param/param.go, an example config.toml file is shown in fig. 2, and an example environment variable is shown in fig. 3.
In an embodiment of the present invention, a user may create or update a Chaos object to kubernets API Server through a YAML file or a kubernets client. k8s-Chaos creates update or delete events through the Chaos object in the watch API Server, maintains the operation and life cycle of specific Chaos experiments, and in the process, the controller-manager, the Chaos-daemon and the sidecar container work together to provide the capability of error injection. The Admission-queries are HTTP callback services for receiving Admission requests, and when receiving a Pod creation request, dynamically modify the Pod object to be created, as shown in fig. 4.
k8s-chaos employs the opt-in mode, as shown in fig. 5, which schedules termination times for kubernets (k8s) applications that agree to terminate their pod by k8 s-chaos. Opt-in is accomplished by setting the following tags on the k8s application:
k8 s-chaos/enabled: set to "enabled" to opt in k8s-chaos
k8 s-chaos/mtbf: mean time between failures (days)
k8s-chaos/identifier: k8s unique identifier of the application. For identifying a pod belonging to the k8s application, since the pod inherits the tag from its k8s application, if k8s-chaos detects that app foo has registered as a victim, k8s-chaos will look for all podcasts tagged with k8s-chaos/identifier: foo to determine which podcasts are likely killed. It is recommended to set this value to be the same as the application name.
k8s-chaos/kill mode: the default behavior is that k8s-chaos kills only one pod of the application. This behavior can be overridden by setting the values to:
"kill-all" if you want k8s-chaso to kill all of the pods in the current kubernets cluster, regardless of pod status. There is no need to set kill-value, but care must be taken to use this tag
Fixed may kill a specific number of executing pods with a kill value. If over-specified, all of the running Pod will be killed and a warning issued.
random max percent specifies the maximum percentage, containing the termination value that can be terminated. Once the scheduled time is reached, the uniformly randomly specified% running Pod will terminate.
fixed percentage specifies a fixed percentage whose kill value may be terminated. At the scheduled time, the specified fixed% of the running Pod will be terminated.
k8s-chaos was run on weekdays in preconfigured hours (hours run, defaults to 8 am) and a deployment plan was constructed that would face random Pod deaths sometime on the same day. The time frame of random deaths during the day is configurable, defaulting to 10 am to 4 pm. k8s-chaos may configure a namespace list to blacklist (any deployment in the blacklist namespace is not touched) to disable blacklist, please provide [ "] in the namespace config.
In the scheduling process, a list of eligible k8s applications is generated, step 102.
During the weekday, schedules are scheduled once a day and a Termination schedule for the day is generated. During the scheduling process, k8s-chaos generates a list of eligible k8s applications, such as k8s applications that have been selected for inclusion and are not blacklisted.
In step 103, for the k8s application in the k8s application list, it is determined whether the pod of the k8s application should be dropped by kill.
Optionally, step 103 may comprise: for the k8s application in the k8s application list, a biased coin is flipped to determine if the pod of the k8s application should be dropped by kill. Optionally, the coin is biased by k8 s-chaos/mtbf. Optionally, to determine whether the pod of the k8s application should be dropped by kill, includes: a random time is calculated to determine when the pod of the k8s application will be dropped by Kill.
For eligible k8s applications, a biased coin is flipped (biased by k8 s-chaos/mtbf) to determine if the pod of the k8s application should be dropped by kill. For each target object, a random time is counted to determine when Pod will be dropped by Kill. Optionally, the conditions include at least one of: unselected, blacklisted, or deleted from the whitelist.
Optionally, the method further comprises: when the random time is reached, checking whether the k8s application still meets the condition, if yes, checking whether the k8s application updates the kill mode and kill value. Optionally, after checking whether the k8s application program updates the kill mode and the kill value, the method further includes: if yes, executing the Pod according to the kill mode and the kill value.
Termination time: this is a randomly generated time when there will be Pod dropped by Kill in the target k8s application on a certain day. By the time of Termination, k8s-chaos checks whether the k8s application is still eligible (after scheduling, not selected for exit or blacklisted or deleted from whitelist) to check whether the k8s application has updated kill mode and kill value. If yes, executing the Pod according to the kill mode and the kill value.
According to the various embodiments described above, it can be seen that the technical problem that conventional tests in the prior art cannot fully cope with various applications running on a complex network and architecture-based Kubernetes cluster and clusters can not be solved by deploying the corresponding k8s-chaos config map in the namespace where k8s-chaos is to be run, so that k8s-chaos is run in a Kubernetes cluster as a k8s application, generating a k8s application list meeting conditions in a scheduling process, and determining whether pod of the k8s application should be kill or not for the k8s application in the k8s application list. The tester can directly and effectively obtain the resource change condition and the occurrence reason in the cluster operation from the front end. k8s-chaos is a tool following the principles of chaotic engineering that can randomly delete kubernets pod, check if the service is fail-resistant and help maintain the healthy operation of the relevant system. k8s-chaos could also complete the configuration via a yaml file that not only can terminate the specified application, but also can determine the execution time of the recovery policy.
Fig. 6 is a schematic diagram of main modules of a kubernets container platform-oriented chaos device according to an embodiment of the present invention, and as shown in fig. 6, the kubernets container platform-oriented chaos device 600 includes a deployment module 601, a scheduling module 602, and a determination module 603; the deployment module 601 is used for deploying a corresponding k8s-chaos config map in a namespace in which k8s-chaos is to be run, so that k8s-chaos is run as a k8s application program in a kubernets cluster; the scheduling module 602 is configured to generate a k8s application list meeting the condition during the scheduling process; the determining module 603 is configured to determine whether the pod of the k8s application should be dropped by kill for the k8s application in the k8s application list.
Optionally, the namespace is kube-system.
Optionally, the determining module 603 is further configured to:
for the k8s application in the k8s application list, a biased coin is flipped to determine if the pod of the k8s application should be dropped by kill.
Optionally, the coin is biased by k8 s-chaos/mtbf.
Optionally, the determining module 603 is further configured to:
a random time is calculated to determine when the pod of the k8s application will be dropped by Kill.
Optionally, the determining module 603 is further configured to:
when the random time is reached, checking whether the k8s application still meets the condition, if yes, checking whether the k8s application updates the kill mode and kill value.
Optionally, the conditions include at least one of:
unselected, blacklisted, or deleted from the whitelist.
Optionally, the determining module 603 is further configured to:
and after checking whether the k8s application program updates the kill mode and the kill value, if so, executing the Pod according to the kill mode and the kill value.
Optionally, the k8s-chaos is configured by an environment variable or a toml file located at/etc/k 8s-chaos/config.
Optionally, the deployment module 601 is further configured to:
before deploying the corresponding k8s-chaos config map configmap in the namespace where k8s-chaos is to be run, creating k8s-chaos config map configmap;
a Kubernetes cluster is deployed.
According to the various embodiments described above, it can be seen that the technical problem that conventional tests in the prior art cannot fully cope with various applications running on a complex network and architecture-based Kubernetes cluster and clusters can not be solved by deploying the corresponding k8s-chaos config map in the namespace where k8s-chaos is to be run, so that k8s-chaos is run in a Kubernetes cluster as a k8s application, generating a k8s application list meeting conditions in a scheduling process, and determining whether pod of the k8s application should be kill or not for the k8s application in the k8s application list. The tester can directly and effectively obtain the resource change condition and the occurrence reason in the cluster operation from the front end. k8s-chaos is a tool following the principles of chaotic engineering that can randomly delete kubernets pod, check if the service is fail-resistant and help maintain the healthy operation of the relevant system. k8s-chaos could also complete the configuration via a yaml file that not only can terminate the specified application, but also can determine the execution time of the recovery policy.
It should be noted that, in the implementation contents of the chaos device for the kubernets container platform according to the present invention, the above chaos testing method for the kubernets container platform has been described in detail, and therefore, the repeated contents are not described again.
Fig. 7 illustrates an exemplary system architecture 700 of a kubernets container platform-oriented chaotic test method or a kubernets container platform-oriented chaotic device to which embodiments of the present invention may be applied.
As shown in fig. 7, the system architecture 700 may include terminal devices 701, 702, 703, a network 704, and a server 705. The network 704 serves to provide a medium for communication links between the terminal devices 701, 702, 703 and the server 705. Network 704 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
A user may use the terminal devices 701, 702, 703 to interact with a server 705 over a network 704, to receive or send messages or the like. The terminal devices 701, 702, 703 may have installed thereon various communication client applications, such as a shopping-like application, a web browser application, a search-like application, an instant messaging tool, a mailbox client, social platform software, etc. (by way of example only).
The terminal devices 701, 702, 703 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 705 may be a server providing various services, such as a background management server (for example only) providing support for shopping websites browsed by users using the terminal devices 701, 702, 703. The background management server can analyze and process the received data such as the article information query request and feed back the processing result to the terminal equipment.
It should be noted that the chaos testing method for the kubernets container platform provided by the embodiment of the present invention is generally executed by the server 705, and accordingly, the chaos device for the kubernets container platform is generally disposed in the server 705.
It should be understood that the number of terminal devices, networks, and servers in fig. 7 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 8, shown is a block diagram of a computer system 800 suitable for use with a terminal device implementing an embodiment of the present invention. The terminal device shown in fig. 8 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 8, the computer system 800 includes a Central Processing Unit (CPU)801 that can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)802 or a program loaded from a storage section 808 into a Random Access Memory (RAM) 803. In the RAM803, various programs and data necessary for the operation of the system 800 are also stored. The CPU 801, ROM 802, and RAM803 are connected to each other via a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.
The following components are connected to the I/O interface 805: an input portion 806 including a keyboard, a mouse, and the like; an output section 807 including a signal such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 808 including a hard disk and the like; and a communication section 809 including a network interface card such as a LAN card, a modem, or the like. The communication section 809 performs communication processing via a network such as the internet. A drive 810 is also connected to the I/O interface 805 as necessary. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 810 as necessary, so that a computer program read out therefrom is mounted on the storage section 808 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 809 and/or installed from the removable medium 811. The computer program executes the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 801.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer programs according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes a deployment module, a scheduling module, and a determination module, where the names of the modules do not in some cases constitute a limitation on the modules themselves.
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, implement the method of: deploying a corresponding k8s-chaos config map configmap in a namespace in which k8s-chaos is to be run, so that k8s-chaos is run as a k8s application in a kubernets cluster; in the scheduling process, generating a k8s application program list meeting the conditions; for the k8s application in the k8s application list, it is determined whether the pod of the k8s application should be dropped by kill.
According to the technical scheme of the embodiment of the invention, a corresponding k8s-chaos config map is deployed in a namespace in which k8s-chaos is to be operated, so that k8s-chaos is operated in a Kubernets cluster as a k8s application program, a k8s application program list meeting conditions is generated in a scheduling process, and a technical means that whether the pod of the k8s application program should be dropped by kill is determined for the k8s application program in the k8s application program list is adopted, so that the technical problem that conventional tests in the prior art cannot fully cope with various applications operated in Kubernets and on the basis of complex networks and architectures is solved. The tester can directly and effectively obtain the resource change condition and the occurrence reason in the cluster operation from the front end. k8s-chaos is a tool following the principles of chaotic engineering that can randomly delete kubernets pod, check if the service is fail-resistant and help maintain the healthy operation of the relevant system. k8s-chaos could also complete the configuration via a yaml file that not only can terminate the specified application, but also can determine the execution time of the recovery policy.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (13)
1. A chaos testing method for a Kubernetes container platform is characterized by comprising the following steps:
deploying a corresponding k8s-chaos config map configmap in a namespace in which k8s-chaos is to be run, so that k8s-chaos is run as a k8s application in a kubernets cluster;
in the scheduling process, generating a k8s application program list meeting the conditions;
for the k8s application in the k8s application list, it is determined whether the pod of the k8s application should be dropped by kill.
2. The method of claim 1, wherein the namespace is a kube-system.
3. The method of claim 1, wherein determining whether the pod of the k8s application should be dropped by kill for the k8s application from the list of k8s applications comprises:
for the k8s application in the k8s application list, a biased coin is flipped to determine if the pod of the k8s application should be dropped by kill.
4. The method of claim 3, wherein the coin is biased by k8 s-chaos/mtbf.
5. The method as claimed in claim 3, wherein determining whether the pod of the k8s application should be dropped by kill comprises:
a random time is calculated to determine when the pod of the k8s application will be dropped by Kill.
6. The method of claim 5, further comprising:
when the random time is reached, checking whether the k8s application still meets the condition, if yes, checking whether the k8s application updates the kill mode and kill value.
7. The method of claim 6, wherein the conditions comprise at least one of:
unselected, blacklisted, or deleted from the whitelist.
8. The method as claimed in claim 6, wherein after checking whether the k8s application has updated kill mode and kill value, further comprising:
if yes, executing the Pod according to the kill mode and the kill value.
9. The method of claim 1, wherein the k8s-chaos is configured by an environment variable or a toml file at/etc/k 8s-chaos/config.
10. The method of claim 1, wherein before deploying a corresponding k8s-chaos config map configmap in a namespace in which k8s-chaos is to be run, further comprising:
creating k8s-chaos config map configmap;
a Kubernetes cluster is deployed.
11. A chaos device for a Kubernetes container platform, comprising:
a deployment module, configured to deploy a corresponding k8s-chaos config map in a namespace where k8s-chaos is to be run, so that k8s-chaos is run as a k8s application in a kubernets cluster;
the scheduling module is used for generating a k8s application program list meeting the conditions in the scheduling process;
a determination module to determine, for the k8s application in the k8s list of applications, whether the pod of the k8s application should be dropped by kill.
12. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
the one or more programs, when executed by the one or more processors, implement the method of any of claims 1-10.
13. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011401478.7A CN112463616A (en) | 2020-12-02 | 2020-12-02 | Chaos testing method and device for Kubernetes container platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011401478.7A CN112463616A (en) | 2020-12-02 | 2020-12-02 | Chaos testing method and device for Kubernetes container platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112463616A true CN112463616A (en) | 2021-03-09 |
Family
ID=74805513
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011401478.7A Pending CN112463616A (en) | 2020-12-02 | 2020-12-02 | Chaos testing method and device for Kubernetes container platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112463616A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114791846A (en) * | 2022-05-23 | 2022-07-26 | 北京同创永益科技发展有限公司 | Method for realizing observability aiming at cloud native chaos engineering experiment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110535831A (en) * | 2019-07-30 | 2019-12-03 | 平安科技（深圳）有限公司 | Cluster safety management method, device and storage medium based on Kubernetes and network domains |
| CN111880738A (en) * | 2020-07-29 | 2020-11-03 | 浪潮云信息技术股份公司 | Method for automatically creating and mounting LVM (logical volume manager) volume in K8s environment |
| CN111901157A (en) * | 2020-07-10 | 2020-11-06 | 苏州浪潮智能科技有限公司 | Service deployment method, device, equipment and medium based on k8s |
-
2020
- 2020-12-02 CN CN202011401478.7A patent/CN112463616A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110535831A (en) * | 2019-07-30 | 2019-12-03 | 平安科技（深圳）有限公司 | Cluster safety management method, device and storage medium based on Kubernetes and network domains |
| CN111901157A (en) * | 2020-07-10 | 2020-11-06 | 苏州浪潮智能科技有限公司 | Service deployment method, device, equipment and medium based on k8s |
| CN111880738A (en) * | 2020-07-29 | 2020-11-03 | 浪潮云信息技术股份公司 | Method for automatically creating and mounting LVM (logical volume manager) volume in K8s environment |
Non-Patent Citations (3)
| Title |
|---|
| CRAIG MORTEN: "K8s Chaos Dive: Kube-Monkey", pages 1 - 6, Retrieved from the Internet <URL:https://dev.to/craigmorten/k8s-chaos-dive-1-kube-monkey-4431> * |
| JASONMINGHAO: "Kubernetes之ConfigMap", Retrieved from the Internet <URL:https://www.cnblogs.com/jasonminghao/p/12584561.html> * |
| 殷成文: "Chaos Mesh–让应用跟混沌在 Kubernetes 上共舞", pages 1 - 11, Retrieved from the Internet <URL:https://cn.pingcap.com/blog/chaos-mesh/> * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114791846A (en) * | 2022-05-23 | 2022-07-26 | 北京同创永益科技发展有限公司 | Method for realizing observability aiming at cloud native chaos engineering experiment |
| CN114791846B (en) * | 2022-05-23 | 2022-10-04 | 北京同创永益科技发展有限公司 | Method for realizing observability aiming at cloud-originated chaos engineering experiment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112860451A (en) | Multi-tenant data processing method and device based on SaaS | |
| CN109245908B (en) | Method and device for switching master cluster and slave cluster | |
| CN111090423B (en) | Webhook framework system and method for realizing active calling and event triggering | |
| CN112867988A (en) | Implementing compliance settings by a mobile device to follow a configuration scenario | |
| CN111897633A (en) | Task processing method and device | |
| CN110795741A (en) | Method and device for carrying out security processing on data | |
| CN109828830B (en) | Method and apparatus for managing containers | |
| CN111046371A (en) | Method, electronic device and computer-readable medium for generating device identification | |
| US11734057B2 (en) | Method and apparatus for processing a service of an abnormal server | |
| CN113296828A (en) | Method, server and system for issuing application | |
| CN113010238A (en) | Permission determination method, device and system for micro application call interface | |
| CN112463616A (en) | Chaos testing method and device for Kubernetes container platform | |
| CN112445860B (en) | Method and device for processing distributed transaction | |
| CN115442129A (en) | Method, device and system for managing cluster access authority | |
| US20230014233A1 (en) | Serverless Application Function Execution | |
| CN113296829A (en) | Method, device, equipment and computer readable medium for processing service | |
| CN112099841A (en) | Method and system for generating configuration file | |
| CN113283891A (en) | Information processing method and device and electronic equipment | |
| CN112925623A (en) | Task processing method and device, electronic equipment and medium | |
| CN112579447A (en) | Browser testing method and device | |
| CN112559001A (en) | Method and device for updating application | |
| CN111290873A (en) | Fault processing method and device | |
| CN111949472A (en) | Method and device for recording application logs | |
| CN112241332A (en) | Interface compensation method and device | |
| CN110262756B (en) | Method and device for caching data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |