CN114924898A

CN114924898A - Method, device, equipment and medium for implementing automatic chaos experiment

Info

Publication number: CN114924898A
Application number: CN202210492672.3A
Authority: CN
Inventors: 盛旺强
Original assignee: Hainan Chezhiyi Communication Information Technology Co ltd
Current assignee: Hainan Chezhiyi Communication Information Technology Co ltd
Priority date: 2022-05-07
Filing date: 2022-05-07
Publication date: 2022-08-19

Abstract

The invention discloses a method for implementing an automatic chaos experiment, which comprises the following steps: acquiring initial information of a chaos experiment; executing an automatic chaotic experiment according to the initial information of the chaotic experiment to obtain experiment data and an experiment result of the chaotic experiment; and according to the experiment data and the experiment result of the chaotic experiment, acquiring the experiment destruction of the abnormal data of the mixed-action experiment and restarting the chaotic experiment, or acquiring an experiment ending instruction, collecting the experiment data and generating an experiment report. According to the method and the device, expected results of experimental projects can be achieved, the interface security mechanism is improved through the interface encryption technology to request experiments, adaptation of production environment in network scene experiments is supported, experiment destruction can be conducted on abnormal experimental data, the experiments can be restarted, and an experimental result evaluation report can be automatically generated.

Description

Method, device, equipment and medium for implementing automatic chaos experiment

Technical Field

The invention relates to the technical field of data processing, in particular to a method and a device for implementing an automatic chaotic experiment, electronic equipment and a storage medium.

Background

In order to support the development of data business more efficiently, more and more enterprises choose to build a platform based on the concept of cloud service or cloud-native. The distributed system is operated in various fields of business, but the complexity is increased sharply, the consequences of faults are difficult to predict, challenges such as avoidance and verification are difficult to achieve, various unexpected emergencies are difficult to occur when the distributed system is actually operated in a production environment, and the distributed system naturally has various interdependencies. It has become a challenge in the art how to prove through experiments that a distributed system in a production environment still has strong "observability" and "fault recovery" capabilities in the face of an out-of-control condition.

Therefore, there is a need for an active or passive experiment that can predict various unpredictable emergency events in the operation of a distributed system by using an automatic chaotic experiment and a failure mode as an entry point, so as to achieve different scenarios.

Disclosure of Invention

To this end, the present invention provides a method, apparatus, electronic device and storage medium for automated chaotic experiment implementation in an effort to solve, or at least alleviate, at least one of the problems identified above.

According to one aspect of the invention, a method for implementing an automatic chaotic experiment is provided, the method starts chaotic experiment items according to business observation indexes and experiment targets, carries out interface encryption verification, experiment abnormal data destruction and data acquisition and reporting, and completes active or passive experiments in different scenes, and comprises the following steps: acquiring initial information of a chaotic experiment, wherein the initial information of the chaotic experiment comprises experiment target determination information, experiment environment inspection information and experiment type judgment information; executing an automatic chaotic experiment according to the initial information of the chaotic experiment to obtain experiment data and an experiment result of the chaotic experiment; and according to the experiment data and the experiment result of the chaotic experiment, acquiring the experiment destruction of the abnormal data of the mixed-action experiment and restarting the chaotic experiment, or acquiring an experiment ending instruction, collecting the experiment data and generating an experiment report.

Optionally, the step of obtaining initial information of the chaos experiment includes: obtaining a creating result of the chaotic experimental project according to the determination of the chaotic experimental task and the experimental target; acquiring an environment inspection result of the chaotic experimental project according to the creating result of the chaotic experimental project, wherein the environment inspection result is a hardware equipment condition inspection result of the chaotic experimental project; acquiring the executed categories of the chaotic experimental project according to the environmental inspection result of the chaotic experimental project, wherein the executed categories of the chaotic experimental project comprise a classical environment and a container environment; and acquiring an experiment environment initialization result of the chaotic experiment project according to the executed category of the chaotic experiment project.

Optionally, the step of executing an automatic chaotic experiment according to the initial information of the chaotic experiment to obtain experiment data and experiment results of the chaotic experiment includes: according to the experiment environment initialization result of the chaotic experiment project, the executed category of which is the classic environment, obtaining the safety legitimacy authentication result of the chaotic experiment project of the classic environment type or the pod survival probe information of the chaotic experiment project of the container environment type; and acquiring an execution result of the chaotic experimental project according to a safety and legality certification result of the chaotic experimental project of the classical environment type or pod survival probe information of the chaotic experimental project of the container environment type, and acquiring and reporting experimental data.

Optionally, the step of obtaining the experiment destruction of the abnormal data of the mixed-action experiment and restarting the chaotic experiment according to the experiment data and the experiment result of the chaotic experiment, or the step of obtaining the experiment ending instruction, collecting the experiment data, and generating the experiment report includes: if the chaos experiment item of the classical environment type is abnormal in the execution process, actively destroying the chaos experiment item and carrying out interface encryption verification; if the chaos experiment item of the container environment type is abnormal in the execution process, actively destroying the chaos experiment item and restarting the pod survival probe detection; and if the chaotic experiment project is normally executed, acquiring an experiment ending instruction, collecting experiment data and generating an experiment report.

Optionally, the step of obtaining the experiment destruction of the abnormal data of the hybrid experiment and restarting the chaotic experiment includes: acquiring destruction request information of an actively destroyed chaotic experimental project; according to the destruction request information of the active destruction chaotic experimental project, acquiring information whether the result of the physical machine or the virtual machine or the container is successful; if the result of the physical machine, the virtual machine or the container fails, calling an interface to retry to close the task, and waiting for the application program interface of the chaotic experimental project to return the result; if the application program interface of the chaotic experimental project fails to return the result, forcibly stopping the application program process of the chaotic experimental project; if the application program process of the chaos experiment project is stopped forcibly, calling an application program interface of the forced restart host to restart the host forcibly; and verifying the recovery communication of the physical machine or the virtual machine through ping, summarizing data and generating an observability report.

Optionally, the step of encrypting and verifying the interface includes: acquiring key information carried by the chaotic experimental project when an experimental request is sent; acquiring an encrypted character string generated by the host according to the time stamp when the host receives the experiment request according to key information carried by the chaotic experiment project when the experiment request is sent; acquiring decryption information of the chaotic experimental project after receiving the encrypted character string according to the encrypted character string generated by the host according to the timestamp when the host receives the experimental request; acquiring experiment request processing information for completing the chaotic experiment project according to decryption success information of the chaotic experiment project after receiving the encrypted character string; if the decryption time of the chaos experimental project after receiving the encrypted character string exceeds a set threshold, generating decryption abnormal information of the encrypted character string; and acquiring experiment failure information of the chaotic experimental project according to the decryption abnormal information of the encrypted character string.

Optionally, the step of obtaining an experiment end instruction, collecting experiment data, and generating an experiment report includes: acquiring pressure measurement performance index data of the chaotic experimental project according to an experimental data acquisition task of the chaotic experimental project; according to the pressure measurement performance index data of the chaotic experiment item, acquiring the pressure measurement performance index data, calling an inflxdb SDK assembly data format, and writing the inflxdb SDK assembly data format into time sequence storage database cluster information; calling an inflxdb SDK (software development kit) to assemble a data format according to the pressure measurement performance index data, writing the data format into time sequence storage database cluster information, and chaotic experimental project task stop information to obtain request information for reading the data content of the time sequence storage database cluster through the inflxdb SDK; according to the request information of the data content of the time sequence storage database cluster read by the inflxdb SDK, acquiring the required data content information returned by the time sequence storage database cluster according to the request; returning required data content information according to a request according to the time sequence storage database cluster, obtaining the data content returned by the time sequence storage database cluster, performing chaotic formatting, and writing the data content into the relational database information; writing the data content into the relational database information after chaotically formatting according to the data content returned by the time sequence storage database cluster, and acquiring chaos relational data insertion information of the returned data content; and performing chaotic relation data insertion information according to the returned data content, and acquiring and displaying an experimental data acquisition observation report of the chaotic experimental project.

According to another aspect of the invention, a device for implementing an automatic chaotic experiment is disclosed, the device starts a chaotic experiment project according to a service observation index and an experiment target, performs interface encryption verification, experiment abnormal data destruction and data acquisition and reporting, and completes active or passive experiments in different scenes, the device comprises: the chaotic experiment system comprises an acquisition module, a processing module and a control module, wherein the acquisition module is used for acquiring initial information of a chaotic experiment, and the initial information of the chaotic experiment comprises experiment target determination information, experiment environment inspection information and experiment type judgment information; the execution module is used for executing the automatic chaotic experiment according to the initial information of the chaotic experiment and acquiring experiment data and an experiment result of the chaotic experiment; and the handling module is used for acquiring the experiment destruction of the abnormal data of the mixed-action experiment and restarting the chaotic experiment according to the experiment data and the experiment result of the chaotic experiment, or acquiring an experiment ending instruction, collecting the experiment data and generating an experiment report.

According to yet another aspect of the present invention, there is provided a computing device comprising: one or more processors; and a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods of automated chaotic experiment implementation described above.

According to yet another aspect of the invention, there is provided a computer readable storage medium storing one or more programs, the one or more programs including instructions, which when executed by a computing device, cause the computing device to perform any of the methods of automated chaotic experiment implementation described above.

According to the implementation scheme of the automatic chaotic experiment, the initial information of the chaotic experiment is acquired; executing an automatic chaotic experiment according to the initial information of the chaotic experiment to obtain experiment data and an experiment result of the chaotic experiment; according to the experiment data and the experiment result of the chaotic experiment, the experiment destruction of the abnormal data of the hybrid experiment is obtained, the chaotic experiment is restarted, or an experiment ending instruction is obtained, the experiment data is collected, and an experiment report is generated. According to the method and the device, expected results of experimental projects can be achieved, the interface security mechanism is improved through an interface encryption technology to request experiments, adaptation of production environments in network scene experiments is supported, experiment destruction can be carried out on abnormal experimental data, the experiments can be restarted, and an experimental result evaluation report can be automatically generated.

Drawings

To the accomplishment of the foregoing and related ends, certain illustrative aspects are described herein in connection with the following description and the annexed drawings, which are indicative of various ways in which the principles disclosed herein may be practiced, and all aspects and equivalents thereof are intended to be within the scope of the claimed subject matter. The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description read in conjunction with the accompanying drawings. Throughout this disclosure, like reference numerals generally refer to like parts or elements.

FIG. 1 shows a schematic diagram of an application scenario according to an embodiment of the invention; and

FIG. 2 shows a schematic block diagram of a computing device 100, according to an embodiment of the invention; and

FIG. 3 shows a flow diagram of a method 200 of automated chaotic experiment implementation according to one embodiment of the present invention; and

fig. 4 shows a schematic structural diagram of an apparatus 300 for automated chaos experiment implementation according to an embodiment of the invention.

Detailed Description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

Fig. 1 is a schematic view of an application scenario according to an embodiment of the present invention, and the method for implementing an automated chaos experiment provided by the present application may be applied to the application environment shown in fig. 1. The method for implementing the automatic chaos experiment is applied to a device for implementing the automatic chaos experiment. The device for implementing the automatic chaotic experiment is configured in the server 010, or partially configured in the terminal 020 and partially configured in the server 010, and the method for implementing the automatic chaotic experiment is completed by the interaction of the terminal 020 and the server 010.

Wherein the terminal 020 and the server 010 can communicate through a network.

The terminal 020 can be but not limited to various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices, the automatic chaotic experiment starts chaotic experiment projects according to business observation indexes and experiment targets, interface encryption verification, experiment abnormal data destruction and data acquisition reporting are carried out, and active or passive experiments in different scenes are completed. The server 010 of the present application may be implemented by an independent server or a server cluster formed by multiple servers.

Fig. 2 is a block diagram of an example computing device 100. In a basic configuration 102, computing device 100 typically includes system memory 106 and one or more processors 104. A memory bus 108 may be used for communication between the processor 104 and the system memory 106.

Depending on the desired configuration, the processor 104 may be any type of processing, including but not limited to: a microprocessor (μ P), a microcontroller (μ C), a digital information processor (DSP), or any combination thereof. The processor 104 may include one or more levels of cache, such as a level one cache 110 and a level two cache 112, a processor core 114, and registers 116. The example processor core 114 may include an Arithmetic Logic Unit (ALU), a Floating Point Unit (FPU), a digital signal processing core (DSP core), or any combination thereof. The example memory controller 118 may be used with the processor 104, or in some implementations the memory controller 118 may be an internal part of the processor 104.

Depending on the desired configuration, system memory 106 may be any type of memory, including but not limited to: volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. System memory 106 may include an operating system 120, one or more applications 122, and program data 124. In some embodiments, application 122 may be arranged to operate with program data 124 on an operating system. In some embodiments, the computing device 100 is configured to execute a method 200 for implementing an automated chaotic experiment, where the method 200 is capable of starting a chaotic experiment project, performing interface encryption verification, experiment abnormal data destruction, and data collection and reporting according to a business observation index and an experiment target, and completing active or passive experiments in different scenarios, and the program data 124 includes instructions for executing the method 200.

Computing device 100 may also include an interface bus 140 that facilitates communication from various interface devices (e.g., output devices 142, peripheral interfaces 144, and communication devices 146) to the basic configuration 102 via the bus/interface controller 130. The example output device 142 includes a graphics processing unit 148 and an audio processing unit 150. They may be configured to facilitate communication with various external devices, such as a display or speakers, via one or more a/V ports 152. Example peripheral interfaces 144 may include a serial interface controller 154 and a parallel interface controller 156, which may be configured to facilitate communication with external devices such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device) or other peripherals (e.g., printer, scanner, etc.) via one or more I/O ports 158. An example communication device 146 may include a network controller 160, which may be arranged to facilitate communications with one or more other computing devices 162 over a network communication link via one or more communication ports 164.

The network communication link may be one example of a communication medium. Communication media may typically be embodied by computer readable instructions, data structures, program modules, in a modulated data signal, such as a carrier wave or other transport mechanism, and may include any information delivery media. A "modulated data signal" may be a signal that has one or more of its data set or its changes made in such a manner as to encode information in the signal. By way of non-limiting example, communication media may include wired media such as a wired network or private-wired network, and various wireless media such as acoustic, Radio Frequency (RF), microwave, Infrared (IR), or other wireless media. The term computer readable media as used herein may include both storage media and communication media. In some embodiments, one or more programs are stored in the computer readable medium, the one or more programs including instructions for performing certain methods, such as the method 200 implemented by the automated chaotic experiment, performed by the computing device 100 according to embodiments of the present invention.

Computing device 100 may be implemented as part of a small-form factor portable (or mobile) electronic device such as a cellular telephone, a Personal Digital Assistant (PDA), a personal media player device, a wireless web-watch device, a personal headset device, an application specific device, or a hybrid device that include any of the above functions. Computing device 100 may also be implemented as a personal computer including both desktop and notebook computer configurations.

Specifically, since unpredictable emergency events are often encountered in typical scenes such as a distributed system, cloud-native and the like, there are many places where errors occur, and if the processing is not proper, serious service damage may occur, and the implementation method of the automated chaotic experiment of the application can be applied to basic resource scenes, including:

the CPU utilization rate load and the designated nuclear courage rate are fully loaded, the service quality is verified under the specific load, and the capacities of monitoring alarm, flow scheduling and the like are achieved;

network fault scenes comprise network fault experiments such as network delay, network packet loss, tampering analysis and the like, network faults are frequently encountered problems in system operation, network bottleneck can be found immediately based on the network fault scenes, and fault tolerance of the system in the network is improved

The method comprises the following steps that a hardware environment scene, internal memories, a CPU, a disk, raid and other faults influence abnormal downtime of a server, and fault tolerance of an LB layer is improved based on sudden hardware faults;

it can also be applied to cloud native service class scenarios:

node resource fault scenes in the Kubernetes cluster comprise the stability of a CPU, a network, a process and an experimental cloud native service platform in basic resources;

the method comprises the following steps that Pod resource fault scenes in a Kubernets cluster comprise Pod killing, stopping, Pod network abnormity, kubelet, Etcd and other performance experiments of dependent component basic resources;

in the experimental process, a standard drilling process is established by opening an internal subsystem, the drilling process comprises four processes of a preparation stage, an execution stage, an inspection stage and a recovery stage, the complete drilling process from planning to restoration is covered, the drilling process is clearly displayed on a platform in a visual mode, and the closure of the process of a true experiment is perfected.

FIG. 3 shows a flow diagram of a method 200 of automated chaotic experiment implementation according to one embodiment of the present invention. As shown in fig. 2, the method 200 starts chaotic experiment items according to the service observation indexes and the experiment targets, performs interface encryption verification, experiment abnormal data destruction, and data acquisition and reporting, and completes active or passive experiments in different scenes, and the method 200 starts with step S210, and obtains initial information of the chaotic experiment, where the initial information of the chaotic experiment includes experiment target determination, experiment environment inspection, and experiment type judgment information.

Specifically, a user in the embodiment of the application can fill in an experiment scene according to a form, reduce the cost of experiment flow and service gray scale online in an experiment process, depend on an a-one system of an automobile, preferentially select a gray scale host to perform an experiment when the application is online so as to minimize the abnormal cost of explosion radius control, randomly select an example from the application as an experiment target, calculate the deviation between the flow of the example and the expected flow, readjust the authority of the example, select according to the experiment type, input a service observation index, and create the experiment example according to different experiment types.

Specifically, the initial information of the chaotic experiment comprises the establishment of an experiment environment, the examination of the experiment environment, the judgment of the type of the chaotic experiment and the monitoring of the survival of an experiment probe.

Specifically, in an embodiment of the present application, the step of acquiring initial information of the chaos experiment includes:

obtaining a creating result of the chaotic experimental project according to the determination of the chaotic experimental task and the experimental target; specifically, the experiment task and the experiment target are basic bases for performing the chaotic experiment, and the chaotic experiment project can be created according to the chaotic experiment task and the target.

Acquiring an environment inspection result of the chaotic experimental project according to the creating result of the chaotic experimental project, wherein the environment inspection result is a hardware equipment condition inspection result of the chaotic experimental project;

acquiring the executed categories of the chaotic experimental project according to the environmental inspection result of the chaotic experimental project, wherein the executed categories of the chaotic experimental project comprise a classical environment and a container environment;

and acquiring an experiment environment initialization result of the chaotic experiment project according to the executed category of the chaotic experiment project.

Specifically, after the chaos experiment project is initialized, the survival of the experiment probe needs to be monitored, if the experiment probe is not installed, the experiment probe can be issued through salt/puppet for repairing, and the chaos experiment project is triggered manually to operate the agent probe.

In step S220, an automatic chaotic experiment is performed according to the initial information of the chaotic experiment, and experimental data and experimental results of the chaotic experiment are obtained.

Specifically, since the environment types of the chaotic experiment include a classical environment and a container environment, when the automatic chaotic experiment is performed, the chaotic experiment is performed according to the classical environment and the container environment.

Specifically, in an embodiment of the present application, the step of executing an automatic chaotic experiment according to the initial information of the chaotic experiment to obtain experiment data and an experiment result of the chaotic experiment includes:

according to the experiment environment initialization result of the chaotic experiment project, the executed category of which is the classic environment, obtaining the safety legitimacy authentication result of the chaotic experiment project of the classic environment type or the pod survival probe information of the chaotic experiment project of the container environment type;

and acquiring an execution result of the chaotic experimental project according to a safety and legality certification result of the chaotic experimental project of the classical environment type or pod survival probe information of the chaotic experimental project of the container environment type, and acquiring and reporting experimental data.

Specifically, the execution process of the classical environment type chaotic experiment project includes:

the host machine acquires request information of the chaotic experimental project on line through HTTP (hyper text transport protocol), and performs security and legitimacy authentication when taking over a task;

after the safety legitimacy authentication is verified, the host computer starts a chaotic experiment, and executes the issuing of a chaotic flow task according to a service observation index and an experiment target submitted by a user;

in the experiment process, the host computer records and observes experiment data and waits for an experiment result;

if the abnormality is found in the task execution process, the chaotic experimental project can be actively destroyed, and interface safety certification is carried out;

if the chaotic experimental project is failed to be destroyed and retried for 2 times, the task of the chaotic experimental project is closed, and the host checks whether the return state is successful or not;

if not, the host calls a saltAPI program to issue process interrupt operation and checks whether the return state is successful or not;

if the operation is unsuccessful, the host computer restarts the host computer for recovery, the physical computer calls an IDrac program, and the cloud host computer calls an OpenStack API program;

if the host is not restarted successfully, then the Ping detection is carried out after the host is accessed manually and restarted successfully;

the host informs the user of the end of the experiment and observes index data collection;

the host automatically generates and stores the experiment report, and sends the experiment report to related users.

In this embodiment, the host is a hardware device for executing the chaotic experimental project, and may be a terminal or a server.

Specifically, the execution process of the chaotic experimental project of the container environment type includes:

the host preselects and issues an operator program, judges the survival condition of the pod probe, if the survival is abnormal, updates the experimental configuration and preselects again;

calling an application program interface detection agent operator program of the k8s by the host computer, creating an object of the chaotic experimental project for experimental operation, and executing flow tasks issuing of the chaotic experimental project according to a business observation index and an experimental target submitted by a user; specifically, k8s is fully called kubernets, which is an open source and is used for managing containerized applications on multiple hosts in a cloud platform, the goal of kubernets is to make deploying containerized applications simple and efficient, kubernets provides a mechanism for application deployment, planning, updating and maintenance, and is the most mainstream system for constructing cloud originality;

the host computer records and observes experimental data, and waits for experimental results;

when the host machine finds abnormality in the execution process, the host machine can actively destroy chaotic experimental items in the experimental process and wait for returning results;

if the destruction fails, retrying the destruction of the chaotic experimental project for 2 times, and closing the task;

if the destroy retry of the chaos experiment project fails, the host restarts the service Pod program and waits for the return of a success result

If the pod program is unsuccessfully restarted, the manual access mode is adopted until the pod program is recovered;

after the experiment is finished, the host automatically generates and stores an experiment report and sends the experiment report to related users.

In step S230, according to the experiment data and the experiment result of the chaotic experiment, the experiment destruction of the abnormal data of the hybrid experiment is obtained and the chaotic experiment is restarted, or an experiment end instruction is obtained, the experiment data is collected, and an experiment report is generated.

Specifically, the chaos experiment is performed for the purposes of performing business row evaluation, predicting expectability of an experimental object, initiating an experiment request through an interface security verification mechanism, checking destruction technology of chaos experiment projects, realizing interface encryption technology, and collecting and reporting experimental data.

Specifically, in an embodiment of the present application, the step of obtaining the experiment destruction of the abnormal data of the mixed-action experiment and restarting the chaotic experiment according to the experiment data and the experiment result of the chaotic experiment, or obtaining the experiment end instruction, collecting the experiment data, and generating the experiment report includes:

if the chaos experiment item of the classic environment type is abnormal in the execution process, actively destroying the chaos experiment item and carrying out interface encryption verification; specifically, for the abnormal situation of the chaotic experimental project of the classical environment type during the execution, the experimental project can be actively destroyed to process the interface encryption verification.

If the chaos experiment item of the container environment type is abnormal in the execution process, actively destroying the chaos experiment item and restarting the pod survival probe detection; specifically, for the abnormal situation of the chaos experiment project of the container environment type in the execution process, the chaos experiment project can be processed by actively destroying the solid eye project and restarting the pod survival probe detection.

And if the chaotic experiment project is normally executed, acquiring an experiment ending instruction, collecting experiment data and generating an experiment report. Specifically, for a chaos experiment project which normally runs, corresponding experiment steps are executed according to an experiment flow, corresponding experiment targets and tasks are completed, experiment data are collected, and an experiment report is generated.

Specifically, in an embodiment of the present application, the step of destroying the experiment for obtaining the abnormal data of the mixing experiment and restarting the chaotic experiment includes:

obtaining destruction request information of actively destroying the chaotic experimental project; specifically, the precondition of the destruction request is that the mixed-action experiment project is stopped and the data acquisition is finished;

according to the destruction request information of the active destruction chaotic experimental project, acquiring information whether the result of the physical machine or the virtual machine or the container is successful; specifically, whether the result of the physical machine or the virtual machine or the container succeeds or not is judged, whether the physical machine or the virtual machine or the container correctly obtains the destruction request information or not is judged, and the initialization condition for actively destroying the chaotic experimental project is made;

if the result of the physical machine or the virtual machine or the container fails, calling the interface to retry to close the task, and waiting for the application program interface of the chaotic experimental project to return the result; specifically, in actual application, after the result of the physical machine, the virtual machine or the container fails, if the retry of the interface is not successful after two times of calling, the task is closed, and an application program interface of the chaotic experimental project returns a corresponding instruction;

if the returned result of the application program interface of the chaotic experimental project fails, the application program process of the chaotic experimental project is forcibly stopped; specifically, if the application program interface of the chaotic experimental project returns a failure result, calling the application program interface of salt to forcibly close the application program process of the chaotic experimental project;

if the application program process of the chaos experiment project is stopped forcibly, calling an application program interface of the forced restart host to restart the host forcibly;

and verifying the recovery communication of the physical machine or the virtual machine through ping, summarizing data and generating an observability report. Specifically, after the host is forcibly restarted, the recovery communication condition of the physical machine or the virtual machine is verified through ping, if the communication fails, the communication is processed through a manual access mode, if the communication succeeds, a destruction request is completed, at the moment, the data of the completed chaotic experimental project are summarized, and an observability report is generated and sent to a corresponding user.

Specifically, in an embodiment of the present application, the step of encrypting and verifying the interface includes:

acquiring key information carried by the chaotic experiment project when an experiment request is sent; specifically, when a user carries out a chaotic experiment project, if corresponding requirements of interface encryption verification are required, a secret key is required to be carried when an experiment request is sent;

acquiring an encrypted character string generated by the host according to the time stamp when the host receives the experiment request according to key information carried by the chaotic experiment project when the experiment request is sent; specifically, after receiving a key sent by a user, a host generates an encryption character string according to a timestamp of a receiving event, wherein the encryption character string comprises timestamp information and key related information;

acquiring decryption information of the chaotic experimental project after the encrypted character string is received according to the encrypted character string generated by the host according to the time stamp when the host receives the experimental request;

acquiring experiment request processing information for completing the chaotic experiment project according to decryption success information of the chaotic experiment project after the encrypted character string is received;

if the decryption time of the chaos experimental project after receiving the encryption character string exceeds a set threshold, generating decryption abnormal information of the encryption character string;

and acquiring experiment failure information of the chaotic experimental project according to the decryption abnormal information of the encrypted character string.

Specifically, after the host receives the decryption information for analyzing the encrypted character string, the host analyzes the chaotic experiment task and the specific step flow in the chaotic experiment project request sent by the user, if the host receives the information for analyzing the encrypted character string failure, the information for decrypting the failure is returned to the user, and the user initiates the request information of the chaotic experiment project again.

Specifically, in an embodiment of the present application, the step of obtaining an experiment ending instruction, collecting experiment data, and generating an experiment report includes:

acquiring pressure measurement performance index data of the chaotic experimental project according to an experimental data acquisition task of the chaotic experimental project;

according to the pressure measurement performance index data of the chaotic experiment item, acquiring the pressure measurement performance index data, calling an inflixdb SDK assembly data format, and writing the pressure measurement performance index data into time sequence storage database cluster information; specifically, at this time, the writing of the experimental data into the time sequence storage database is completed, if the writing into the time sequence storage database fails, retry is performed according to the set retry times, if retry also fails, a user is notified that the writing of the experimental data into the time sequence storage database fails, the user can initiate a corresponding new task again, and the experimental data is written into the time sequence storage database again;

calling an inflixdb SDK (software development kit) to assemble a data format according to the pressure measurement performance index data, writing the data format into time sequence storage database cluster information and chaotic experimental project task stop information, and acquiring request information for reading the data content of the time sequence storage database cluster through the inflixdb SDK;

according to the request information of the data content of the time sequence storage database cluster read by the influxdb SDK, acquiring the required data content information returned by the time sequence storage database cluster according to the request;

returning required data content information according to a request according to the time sequence storage database cluster, obtaining the data content returned by the time sequence storage database cluster, performing chaotic formatting, and writing the data content into the relational database information;

writing the data content returned by the time sequence storage database cluster into relational database information after being subjected to chaotic formatting, and acquiring chaotic relational data insertion information of the returned data content;

and performing chaotic relation data insertion information according to the returned data content, and acquiring and displaying an experimental data acquisition observation report of the chaotic experimental project. Specifically, the above steps complete the reading of the experimental data acquisition.

According to the implementation scheme of the automatic chaotic experiment, the initial information of the chaotic experiment is acquired; executing an automatic chaotic experiment according to the initial information of the chaotic experiment to obtain experiment data and an experiment result of the chaotic experiment; and according to the experiment data and the experiment result of the chaotic experiment, acquiring the experiment destruction of the abnormal data of the mixed-action experiment and restarting the chaotic experiment, or acquiring an experiment ending instruction, collecting the experiment data and generating an experiment report. According to the method and the device, expected results of experimental projects can be achieved, the interface security mechanism is improved through an interface encryption technology to request experiments, adaptation of production environments in network scene experiments is supported, experiment destruction can be carried out on abnormal experimental data, the experiments can be restarted, and an experimental result evaluation report can be automatically generated.

It should be understood that, although the steps in the flowchart of fig. 3 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in fig. 3 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

In an embodiment, as shown in fig. 4, a device 300 for implementing an automatic chaotic experiment is provided, where the device 300 starts a chaotic experiment project according to a service observation index and an experiment target, performs interface encryption verification, experiment abnormal data destruction, and data acquisition and reporting, and completes active or passive experiments in different scenarios, and the device 300 includes: the device comprises an acquisition module, an execution module and a treatment module.

The chaotic experiment system comprises an acquisition module, a processing module and a control module, wherein the acquisition module is used for acquiring initial information of a chaotic experiment, and the initial information of the chaotic experiment comprises experiment target determination information, experiment environment inspection information and experiment type judgment information; the execution module is used for executing the automatic chaotic experiment according to the initial information of the chaotic experiment and acquiring experiment data and an experiment result of the chaotic experiment; and the disposal module is used for acquiring experiment destruction of the abnormal data of the mixed-action experiment and restarting the chaotic experiment according to the experiment data and the experiment result of the chaotic experiment, or acquiring an experiment ending instruction, collecting the experiment data and generating an experiment report.

Specifically, in another embodiment of the present application, the obtaining module is configured to obtain a creating result of the chaotic experimental project according to the determination of the chaotic experimental task and the experimental target; acquiring an environment inspection result of the chaotic experimental project according to the creating result of the chaotic experimental project, wherein the environment inspection result is a hardware equipment condition inspection result of the chaotic experimental project; acquiring the executed categories of the chaotic experimental project according to the environmental inspection result of the chaotic experimental project, wherein the executed categories of the chaotic experimental project comprise a classical environment and a container environment; and acquiring an experiment environment initialization result of the chaotic experiment project according to the executed category of the chaotic experiment project.

Specifically, in another embodiment of the present application, the execution module is configured to obtain a safety legitimacy authentication result of a chaotic experimental project of a classical environment type or pod survival probe information of the chaotic experimental project of a container environment type according to an experimental environment initialization result of the chaotic experimental project with a classical environment as a class; and acquiring an execution result of the chaotic experimental project according to a safety and legality certification result of the chaotic experimental project of the classical environment type or pod survival probe information of the chaotic experimental project of the container environment type, and acquiring and reporting experimental data.

Specifically, in another embodiment of the present application, the handling module is configured to actively destroy a chaotic experimental item and perform interface encryption verification if an exception is found in an execution process of the chaotic experimental item of a classical environment type; if the chaos experiment item of the container environment type is abnormal in the execution process, actively destroying the chaos experiment item and restarting the pod survival probe detection; and if the chaotic experiment project is normally executed, acquiring an experiment ending instruction, collecting experiment data and generating an experiment report.

Specifically, in another embodiment of the present application, the disposal module is configured to acquire destruction request information for actively destroying the chaotic experimental project; according to the destruction request information of the active destruction chaotic experimental project, acquiring information about whether the result of the physical machine or the virtual machine or the container is successful; if the result of the physical machine or the virtual machine or the container fails, calling the interface to retry to close the task, and waiting for the application program interface of the chaotic experimental project to return the result; if the returned result of the application program interface of the chaotic experimental project fails, the application program process of the chaotic experimental project is forcibly stopped; if the application program process of the chaos experiment project is stopped forcibly, calling an application program interface of the forced restart host to restart the host forcibly; and verifying the recovery communication of the physical machine or the virtual machine through ping, summarizing data and generating an observability report.

Specifically, in another embodiment of the present application, the handling module is configured to obtain key information that is carried by the chaotic experimental project when an experimental request is sent; acquiring an encrypted character string generated by the host according to the time stamp when the host receives the experiment request according to key information carried by the chaotic experiment project when the experiment request is sent; acquiring decryption information of the chaotic experimental project after receiving the encrypted character string according to the encrypted character string generated by the host according to the timestamp when the host receives the experimental request; acquiring experiment request processing information for completing the chaotic experiment project according to decryption success information of the chaotic experiment project after receiving the encrypted character string; if the decryption time of the chaos experimental project after receiving the encryption character string exceeds a set threshold, generating decryption abnormal information of the encryption character string; and acquiring experiment failure information of the chaotic experimental project according to the decryption abnormal information of the encrypted character string.

Specifically, in another embodiment of the present application, the processing module is configured to obtain pressure measurement performance index data of a chaotic experimental project according to an experimental data acquisition task of the chaotic experimental project; according to the pressure measurement performance index data of the chaotic experiment item, acquiring the pressure measurement performance index data, calling an inflixdb SDK assembly data format, and writing the pressure measurement performance index data into time sequence storage database cluster information; calling an inflixdb SDK (software development kit) to assemble a data format according to the pressure measurement performance index data, writing the data format into time sequence storage database cluster information and chaotic experimental project task stop information, and acquiring request information for reading the data content of the time sequence storage database cluster through the inflixdb SDK; reading the request information of the data content of the time sequence storage database cluster according to the influxdb SDK, and acquiring the required data content information returned by the time sequence storage database cluster according to the request; returning required data content information according to a request according to the time sequence storage database cluster, obtaining the data content returned by the time sequence storage database cluster, performing chaotic formatting, and writing the data content into the relational database information; writing the data content returned by the time sequence storage database cluster into relational database information after being subjected to chaotic formatting, and acquiring chaotic relational data insertion information of the returned data content; and performing chaotic relation data insertion information according to the returned data content, and acquiring and displaying an experimental data acquisition observation report of the chaotic experimental project.

According to the implementation scheme of the automatic chaotic experiment, the initial information of the chaotic experiment is acquired through the acquisition module, and the initial information of the chaotic experiment comprises experiment target determination information, experiment environment inspection information and experiment type judgment information; the execution module executes an automatic chaotic experiment according to the initial information of the chaotic experiment to obtain experiment data and an experiment result of the chaotic experiment; and the disposal module acquires experiment destruction of abnormal data of the mixed-action experiment and restarts the chaotic experiment according to the experiment data and the experiment result of the chaotic experiment, or acquires an experiment ending instruction, collects the experiment data and generates an experiment report. According to the method and the device, expected results of experimental projects can be achieved, the interface security mechanism is improved through the interface encryption technology to request experiments, adaptation of production environment in network scene experiments is supported, experiment destruction can be conducted on abnormal experimental data, the experiments can be restarted, and an experimental result evaluation report can be automatically generated.

It should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: rather, the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.

Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in a device as described in this embodiment, or alternatively may be located in one or more devices different from the device in this example. The modules in the foregoing examples may be combined into one module or may be further divided into multiple sub-modules.

Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.

Furthermore, some of the described embodiments are described herein as a method or combination of method elements that can be performed by a processor of a computer system or by other means of performing the described functions. A processor having the necessary instructions for carrying out the method or method elements thus forms a means for carrying out the method or method elements. Further, the elements of the apparatus embodiments described herein are examples of the following apparatus: the apparatus is used to implement the functions performed by the elements for the purpose of carrying out the invention.

As used herein, unless otherwise specified the use of the ordinal adjectives "first", "second", "third", etc., to describe a common object, merely indicate that different instances of like objects are being referred to, and are not intended to imply that the objects so described must be in a given sequence, either temporally, spatially, in ranking, or in any other manner.

While the invention has been described with respect to a limited number of embodiments, those skilled in the art, having benefit of this description, will appreciate that other embodiments can be devised which do not depart from the scope of the invention as disclosed herein. Furthermore, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter. Accordingly, many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the appended claims. The present invention has been disclosed with respect to the scope of the invention, which is to be considered as illustrative and not restrictive, and the scope of the invention is defined by the appended claims.

Claims

1. A method for implementing an automatic chaotic experiment starts chaotic experiment projects according to a service observation index and an experiment target, carries out interface encryption verification, experiment abnormal data destruction and data acquisition and reporting, and completes active or passive experiments in different scenes, and comprises the following steps:

acquiring initial information of a chaotic experiment, wherein the initial information of the chaotic experiment comprises experiment target determination information, experiment environment inspection information and experiment type judgment information;

executing an automatic chaotic experiment according to the initial information of the chaotic experiment to obtain experiment data and an experiment result of the chaotic experiment;

according to the experiment data and the experiment result of the chaotic experiment, the experiment destruction of the abnormal data of the hybrid experiment is obtained, the chaotic experiment is restarted, or an experiment ending instruction is obtained, the experiment data is collected, and an experiment report is generated.

2. The method of claim 1, wherein the step of acquiring initial information of the chaotic experiment comprises:

obtaining a creating result of the chaotic experimental project according to the determination of the chaotic experimental task and the experimental target;

3. The method of claim 2, wherein the step of executing the chaotic experiment according to the initial information of the chaotic experiment to obtain the experiment data and the experiment result of the chaotic experiment comprises:

4. The method according to claim 3, wherein the step of obtaining experiment destruction of abnormal data of the mixed-action experiment and restarting the chaotic experiment according to the experiment data and the experiment result of the chaotic experiment, or obtaining an experiment ending instruction, collecting the experiment data and generating an experiment report comprises:

if the chaos experiment item of the classical environment type is abnormal in the execution process, actively destroying the chaos experiment item and carrying out interface encryption verification;

if the chaos experiment item of the container environment type is abnormal in the execution process, actively destroying the chaos experiment item and restarting the pod survival probe detection;

and if the chaotic experiment project is normally executed, acquiring an experiment ending instruction, collecting experiment data and generating an experiment report.

5. The method of claim 4, wherein the step of obtaining experimental destruction of abnormal data of the mixing experiment and restarting the chaotic experiment comprises:

acquiring destruction request information of an actively destroyed chaotic experimental project;

according to the destruction request information of the active destruction chaotic experimental project, acquiring information about whether the result of the physical machine or the virtual machine or the container is successful;

if the result of the physical machine, the virtual machine or the container fails, calling an interface to retry to close the task, and waiting for the application program interface of the chaotic experimental project to return the result;

if the returned result of the application program interface of the chaotic experimental project fails, the application program process of the chaotic experimental project is forcibly stopped;

if the application program process of the forced stopping chaotic experimental project fails, calling an application program interface of the forced restarting host to forcibly restart the host;

and verifying the recovery communication of the physical machine or the virtual machine through ping, summarizing data and generating an observability report.

6. The method of claim 4, wherein the interface cryptographic validation step comprises:

acquiring key information carried by the chaotic experiment project when an experiment request is sent;

acquiring an encrypted character string generated by the host according to the time stamp when the host receives the experiment request according to key information carried by the chaotic experiment project when the experiment request is sent;

acquiring decryption information of the chaotic experimental project after receiving the encrypted character string according to the encrypted character string generated by the host according to the timestamp when the host receives the experimental request;

acquiring experiment request processing information for completing the chaotic experiment project according to decryption success information of the chaotic experiment project after receiving the encrypted character string;

7. The method of claim 4, wherein the steps of obtaining experiment end instructions and collecting experiment data and generating an experiment report comprise:

according to the pressure measurement performance index data of the chaos experiment project, acquiring the pressure measurement performance index data, calling an inflxdb SDK assembly data format, and writing the inflxdb SDK assembly data format into time sequence storage database cluster information;

calling an inflxdb SDK (software development kit) to assemble a data format according to the pressure measurement performance index data, writing the data format into time sequence storage database cluster information and chaotic experimental project task stop information, and acquiring request information for reading the data content of the time sequence storage database cluster through the inflxdb SDK;

according to the request information of the data content of the time sequence storage database cluster read by the inflxdb SDK, acquiring the required data content information returned by the time sequence storage database cluster according to the request;

and performing chaotic relation data insertion information according to the returned data content, and acquiring and displaying an experimental data acquisition observation report of the chaotic experimental project.

8. A device for implementing an automatic chaotic experiment starts chaotic experiment projects according to a service observation index and an experiment target, carries out interface encryption verification, experiment abnormal data destruction and data acquisition reporting, and completes active or passive experiments in different scenes, and comprises:

the chaotic experiment system comprises an acquisition module, a processing module and a control module, wherein the acquisition module is used for acquiring initial information of a chaotic experiment, and the initial information of the chaotic experiment comprises experiment target determination information, experiment environment inspection information and experiment type judgment information;

the execution module is used for executing an automatic chaotic experiment according to the initial information of the chaotic experiment to obtain experiment data and an experiment result of the chaotic experiment;

and the disposal module is used for acquiring experiment destruction of the abnormal data of the mixed-action experiment and restarting the chaotic experiment according to the experiment data and the experiment result of the chaotic experiment, or acquiring an experiment ending instruction, collecting the experiment data and generating an experiment report.

9. An electronic device, comprising:

one or more processors; and

a memory;

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for performing any of the methods of claims 1-7.

10. A computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing device, cause the computing device to perform any of the methods of claims 1-7.