CN117743153A - Fault injection method and device, electronic equipment and storage medium - Google Patents

Fault injection method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN117743153A
CN117743153A CN202311640859.4A CN202311640859A CN117743153A CN 117743153 A CN117743153 A CN 117743153A CN 202311640859 A CN202311640859 A CN 202311640859A CN 117743153 A CN117743153 A CN 117743153A
Authority
CN
China
Prior art keywords
event
fault
injection
target
fault event
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311640859.4A
Other languages
Chinese (zh)
Inventor
吴玲玲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Construction Bank Corp
CCB Finetech Co Ltd
Original Assignee
China Construction Bank Corp
CCB Finetech Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Construction Bank Corp, CCB Finetech Co Ltd filed Critical China Construction Bank Corp
Priority to CN202311640859.4A priority Critical patent/CN117743153A/en
Publication of CN117743153A publication Critical patent/CN117743153A/en
Pending legal-status Critical Current

Links

Landscapes

  • Debugging And Monitoring (AREA)

Abstract

The application provides a fault injection method, a fault injection device, electronic equipment and a storage medium, wherein the fault injection method comprises the following steps: and determining an evaluation value of each fault event by monitoring each fault event in the fault event pool, wherein the evaluation value is used for indicating the severity of the influence generated by the corresponding fault event. And selecting a target event to be injected from the fault event pool according to the set dimension evaluation value, responding to an input parameter received by an application program interface and carrying out fault injection, determining an object to be subjected to fault injection according to the input parameter, and injecting an executable instruction corresponding to the target event into the object according to the mapping relation between the event and the executable instruction, thereby realizing fault injection. The fault event in the fault event pool is dynamically monitored, and the target event needing to be injected is selected from the fault event pool based on the severity of the influence generated by the fault event, so that manual selection is avoided, the object of fault injection is appointed during injection, the test range is limited, and the pertinence and the effectiveness of fault injection are improved.

Description

Fault injection method and device, electronic equipment and storage medium
Technical Field
The present disclosure relates to the field of software testing technologies, and in particular, to a fault injection method, a fault injection device, an electronic device, and a storage medium.
Background
The micro-service architecture is a distributed system, each service runs in its independent process, and the micro-service communicate with each other by adopting a lightweight communication mechanism. Each micro-service may be built around a specific business and can be deployed independently to a production environment, class production environment, etc.
In the context of such complex distributed systems, the types of faults are numerous and once they occur, the consequences are difficult to predict. In the related art, the weak links of the distributed system are discovered through a fault injection mode test, so that the weak links can be processed early, and a more flexible high-availability system is obtained.
In the testing process, fault events occurring at different levels based on network faults, operating System (OS) faults, downstream dependent faults, middleware faults and the like directly or indirectly affect a certain micro service, so that the recovery capability of the service itself needs to be observed when the service depends on abnormality externally. In the related art, a manual selection manner is generally adopted to select a fault event which is likely to cause a fault to the system for fault injection because of a plurality of fault events.
Disclosure of Invention
The present application aims to solve, at least to some extent, one of the technical problems in the related art.
To this end, a first object of the present application is to propose a fault injection method to select fault injection for fault events that are likely to cause a fault to the system.
A second object of the present application is to propose a fault injection device.
A third object of the present application is to propose an electronic device.
A fourth object of the present application is to propose a computer readable storage medium.
A fifth object of the present application is to propose a computer programme product.
To achieve the above object, an embodiment of a first aspect of the present application provides a fault injection method, including:
monitoring each fault event in a fault event pool, and determining an evaluation value of each fault event, wherein the evaluation value is used for indicating the severity of influence generated by the corresponding fault event;
selecting a target event to be injected from the fault event pool according to the evaluation value of the set dimension;
responding to input parameters of fault injection received by an application program interface, and determining an object to be subjected to fault injection according to the input parameters;
and injecting the executable instruction corresponding to the target event into the object according to the mapping relation between the event and the executable instruction.
To achieve the above object, an embodiment of a second aspect of the present application provides a fault injection apparatus, including:
the monitoring module is used for monitoring each fault event in the fault event pool and determining an evaluation value of each fault event, wherein the evaluation value is used for indicating the severity of the influence generated by the corresponding fault event;
the selection module is used for selecting target events to be injected from the fault event pool according to the evaluation value of the set dimension;
the determining module is used for responding to the input parameters of fault injection received by the application program interface and determining an object to be subjected to fault injection according to the input parameters;
and the injection module is used for injecting the executable instruction corresponding to the target event into the object according to the mapping relation between the event and the executable instruction.
To achieve the above object, an embodiment of a third aspect of the present application provides an electronic device, including: a processor, and a memory communicatively coupled to the processor;
the memory stores computer-executable instructions;
the processor executes computer-executable instructions stored by the memory to implement the method as described in the first aspect.
To achieve the above object, an embodiment of a fourth aspect of the present application proposes a computer readable storage medium having stored therein computer executable instructions for implementing the method according to the first aspect when being executed by a processor.
To achieve the above object, an embodiment of a fifth aspect of the present application proposes a computer program product for implementing the method of the first aspect when the computer program is executed by a processor.
According to the fault injection method, the fault injection device, the electronic equipment and the storage medium, through monitoring each fault event in the fault event pool, the evaluation value of each fault event is determined, wherein the evaluation value is used for indicating the severity of influence caused by the corresponding fault event. And selecting a target event to be injected from the fault event pool according to the set dimension evaluation value, responding to an input parameter received by an application program interface and carrying out fault injection, determining an object to be subjected to fault injection according to the input parameter, and injecting an executable instruction corresponding to the target event into the object according to the mapping relation between the event and the executable instruction, thereby realizing fault injection. The fault event in the fault event pool is dynamically monitored, and the target event needing to be injected is selected from the fault event pool based on the severity of the influence generated by the fault event, so that manual selection is avoided, the object of fault injection is appointed during injection, the test range is limited, and the pertinence and the effectiveness of fault injection are improved.
Additional aspects and advantages of the application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the application.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
fig. 1 is a schematic flow chart of a fault injection method according to an embodiment of the present application;
FIG. 2 is a flow chart of another fault injection method according to an embodiment of the present disclosure;
FIG. 3 is a schematic diagram of a failure center architecture;
fig. 4 is a schematic structural diagram of a fault injection apparatus according to an embodiment of the present application.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and intended for the purpose of explaining the present application and are not to be construed as limiting the present application.
The fault injection method and device of the embodiment of the application are described below with reference to the accompanying drawings.
Fig. 1 is a flow chart of a fault injection method according to an embodiment of the present application.
In the related art, in the process of testing a distributed system, fault events occurring at different levels based on network faults, OS faults, downstream dependent faults, middleware faults and the like directly or indirectly affect a certain micro service in the distributed system. It is necessary to observe the restoration capabilities of the micro-service itself when each external dependency is abnormal. In the related art, a manual selection manner is generally adopted to select a fault event which is likely to cause a fault to the system for fault injection because of a plurality of fault events.
In view of this problem, the embodiment of the present application provides a fault injection method to improve the fault injection efficiency, as shown in fig. 1, the method includes the following steps:
and 101, monitoring each fault event in the fault event pool, and determining an evaluation value of each fault event, wherein the evaluation value is used for indicating the severity of the influence generated by the corresponding fault event.
Optionally, a fault event pool for fault injection is predefined, each fault event in the fault event pool has a corresponding executable instruction, and a correspondence between the fault event and the executable instruction is stored in the core storage database. The executable instructions are instructions for triggering the object of fault injection to generate corresponding faults.
The fault events in the fault event pool may include, for example: the downstream service is not limited in this embodiment, such as a rejection of the return, packet loss or delay, or write abnormality of the disk, and busy reading.
And 102, selecting a target event to be injected from the fault event pool according to the set dimension evaluation value.
Optionally, the fault event pool includes various monitored fault events, but the frequency and the influence range of different fault events can have some differences with time. In order to perform more effective test on the distributed system, fault events with higher occurrence frequency and larger influence range are preferentially selected for fault injection, and higher evaluation values can be configured in two dimensions of the event occurrence frequency and the event influence range aiming at the fault events needing to be preferentially selected.
Further, in some scenarios, fault injection may cause some loss to the distributed system itself, i.e., some test costs may result. In other scenarios, the process of fault event introduction is complex. Taking these factors into consideration, the probability of injection of the fault events is reduced appropriately, and lower evaluation values are configured in two dimensions of event introduction cost and event introduction complexity for the fault events needing to reduce the injection probability.
And step 103, responding to the input parameters of fault injection received by the application program interface, and determining the object to be subjected to fault injection according to the input parameters.
The object to be subjected to fault injection may be an instance, middleware or physical device, which is not limited in this embodiment.
And 104, injecting executable instructions corresponding to the target events into the object according to the mapping relation between the events and the executable instructions.
The executable instruction is an instruction for triggering the object of fault injection to generate a target event, the object is triggered to generate a corresponding fault event by injecting the executable instruction into the object, and the running condition of the tested target micro-service in the process is observed to determine the running performance of the target micro-service when the fault event is corresponding.
In this embodiment, by monitoring each fault event in the fault event pool, an evaluation value of each fault event is determined, where the evaluation value is used to indicate a severity of an impact generated by a corresponding fault event. And selecting a target event to be injected from the fault event pool according to the set dimension evaluation value, responding to an input parameter received by an application program interface and carrying out fault injection, determining an object to be subjected to fault injection according to the input parameter, and injecting an executable instruction corresponding to the target event into the object according to the mapping relation between the event and the executable instruction, thereby realizing fault injection. The fault event in the fault event pool is dynamically monitored, and the target event needing to be injected is selected from the fault event pool based on the severity of the influence generated by the fault event, so that manual selection is avoided, the object of fault injection is appointed during injection, the test range is limited, and the pertinence and the effectiveness of fault injection are improved.
The present embodiment provides another fault injection method, and fig. 2 is a schematic flow chart of the fault injection method provided in the embodiment of the present application.
As shown in fig. 2, the method may include the steps of:
step 201, counting event occurrence frequency and event influence range for each fault event in the fault event pool.
Optionally, for any fault event, determining the occurrence frequency of the event according to the occurrence times of the corresponding fault event in at least one statistical period; and/or for any fault event, determining the event influence range according to the micro-service duty ratio of the corresponding fault event in the distributed micro-service system in at least one statistical period.
Step 202, inquiring event introduction cost and introduction complexity of each fault event configuration in the fault event pool, and taking event occurrence frequency, event influence range, event introduction cost and introduction complexity as evaluation values of corresponding dimensions for any fault event.
Alternatively, the following definition may be made:
the value range of the occurrence frequency of the event is 0-100, and the occurrence frequency of the fault event is higher when the value is larger;
the value range of the event influence range is 0-100, and the larger the value is, the larger the fault event influence range is;
the value range of the event introduction cost is-100-0, and the event introduction cost is lower when the value is larger;
complexity is introduced into a complexity value range of-100 to 0, and the larger the value is, the lower the complexity is introduced into the event.
And 203, selecting a target event to be injected from the fault event pool according to the evaluation value of each set dimension.
Optionally, for each fault event, carrying out weighted summation on the evaluation values of a plurality of set dimensions to determine a total evaluation value; ranking according to the total evaluation value of each fault event; based on the ranking, a target event to be injected is selected from the fault event pool.
The fault event prioritization algorithm is Σx as follows k a k K=0, … …, I. Wherein x is k For the weight value of the kth dimension (wherein the event occurrence frequency weight is 40%, the event influence range weight is 40%, the event introducing cost weight is 10%, the complexity introducing complexity weight is 10%, a) k The k-th dimension evaluation value is the sequence number of the dimension, and I is the last sequence number of the dimension. The higher the total evaluation value calculated finally, the higher the priority of the fault event.
For example, the evaluation value of the event occurrence frequency is 20, the evaluation value of the event influence range is 60, the evaluation value of the event introduction cost is (-40), and the evaluation value of the introduction complexity is (-40), and the foregoing formulas of the prioritization algorithm, and the weight values of the dimensions are substituted, and the total evaluation value of the fault event=40% ×20+40% ×60+10% × (-40) +10% ×40.
After being processed by the sorting algorithm, the first 20% of fault events are finally selected as target events in the fault introducing event list.
Before each fault injection, the fault input event parameters are counted once according to an algorithm and output.
In step 204, in response to the application program interface receiving the input parameters of the fault injection, at least obtaining the identification, the explosion radius and the dependency information of the target micro-service in the input parameters, wherein the explosion radius is used for indicating a deployment unit allowing the fault injection in the deployment units of the target micro-service.
Optionally, fault injection is performed for a target event in the fault introduction event list. The tester can package relevant information required by fault injection into input parameters and input the input parameters into an application program interface so as to execute a fault input process.
Step 205, determining a target instance of the target micro-service within the explosion radius based on the explosion radius and the identification of the target micro-service.
Optionally, by specifying the radius of explosion, a target instance for the micro-service within the radius of explosion is determined, facilitating subsequent definition of finer grained injection objects for testing.
Step 206, determining a downstream instance, middleware and/or physical device of the target micro-service dependence according to the dependence information.
Alternatively, failures occurring at different levels based on network failures, OS failures, downstream dependency failures, middleware failures, etc. affect a certain micro service directly or indirectly, so it is necessary to observe the restoration capability of the service itself when each external dependency of the service is abnormal. Fault injection may be performed to these dependent downstream instances, middleware, and/or physical devices.
And step 207, taking at least one of the target instance, the downstream instance, the middleware and the physical equipment as the object needing fault injection.
And step 208, injecting executable instructions corresponding to the target events into the object according to the mapping relation between the events and the executable instructions.
As one possible implementation manner, the object is controlled to install the fault injection agent, and under the condition that the installation is successful, the executable instruction corresponding to the target event is sent to the installed agent according to the mapping relation between the event and the executable instruction, so that the agent runs the executable instruction.
Further, the input parameters further include injection start time and injection end time, based on which, at the injection start time, indication information for starting injection is sent to the agent, so that the agent starts to run the executable instruction; and at the injection ending time, sending instruction information for ending injection to the agent so as to enable the agent to stop running the executable instruction.
In some possible scenarios, the fault center may be defined to execute the steps of determining, according to the input parameters, an object to be subjected to fault injection, and injecting, according to a mapping relationship between an event and an executable instruction, the executable instruction corresponding to the target event into the object, where, as shown in fig. 3, the fault center specifically includes: an application program interface service (API Server), a Scheduler (Scheduler), a control program (Controller) 3 components and a Core Storage database (Core Storage).
The application program interface service is used for packaging and providing a unified fault injection interface A to the outside.
The core storage database is used for storing all fault injection parameters which are transmitted in through the fault injection interface A.
And the dispatcher is used for analyzing the parameters of the input fault injection interface A and continuously searching the instance corresponding to the target micro-service and the downstream instance, middleware and physical equipment corresponding to the dependency.
And the control program is used for analyzing the fault event (Action) parameters of the incoming fault injection interface A into executable instructions, issuing the executable instructions to agents (agents) of corresponding examples, or calling application program interfaces of corresponding middleware.
In some possible embodiments, the fault input parameters entered by the fault input interface a may include: identification of the Target microservice (Target), explosion radius (Scope), dependency information (Dependency), target event (Action), injection Start time (Start), and injection End time (End).
The target micro service identification is used for definitely injecting fault micro service, wherein the micro service is a main observation target, and whether the running condition of the micro service after fault injection is abnormal or not is observed.
Explosion radius, in order to reduce the control influence scope, some deployment units are filtered out, for example, the deployment units can be specifically some clusters or some instances or some traffic, etc.
Depending on the information, the service is affected by the failure, in fact, it depends on the abnormality. Possibly from middleware, downstream services or dependent cpus, disks, networks, etc.
The target event is a fault event to be injected in the fault event pool, and describes what fault occurs in dependence of the micro service, for example, the downstream service returns to refusal, and packet loss or delay occurs; or the disk is abnormal in writing, busy in reading and the like.
The injection start time, i.e., the fault injection start time.
The injection end time, i.e., the fault injection end time.
For example: when a certain target event is injected, the event is only required to be transmitted into the packaging interface according to the set requirement. Multiple sets of input parameters may be entered simultaneously.
For example, 20% of the instances cpu within the cluster1 of micro service a are fully loaded, and the input parameters include:
Target("A").
Scope("cluster1","20%").
Dependency("cpu").
Action("cpu_fullload").
Start("2023-11-16 17:00:00").
End("2023-11-16 18:00:00").
simultaneous injection of multiple target events is performed, for example, 20% of instances cpu in cluster1 of micro service a are fully loaded, while the delay of downstream micro service C on which cluster2 of micro service B depends increases by 100ms, and input parameters include:
Target("A","B").
Scope(["cluster1,20%","cluster 2").
Dependency("cpu","C").
Action(“cpu_fullload”,"delay,100ms").
Start("2023-11-16 17:00:00").
End("2023-11-16 18:00:00").
alternatively, as shown in fig. 3, after the fault input parameters are entered by the fault center through the fault input interface a, the fault input parameters are stored in the core storage database and the search for the fault injected object through the scheduler component is started.
The fault input object can be a downstream instance, middleware and/or physical equipment on which the target micro-service depends, a fault agent is automatically installed aiming at the instance, and then a fault event related instruction is issued to the agent through a control program component based on an event and executable instruction mapping relation, so that the purpose of automatic fault injection is realized. Aiming at the middleware, a fault event related instruction can be issued to the middleware in a mode of calling an application program interface of the middleware, so that the purpose of automatic fault injection is realized.
In this embodiment, by monitoring each fault event in the fault event pool, an evaluation value of each fault event is determined, where the evaluation value is used to indicate a severity of an impact generated by a corresponding fault event. And selecting a target event to be injected from the fault event pool according to the set dimension evaluation value, responding to an input parameter received by an application program interface and carrying out fault injection, determining an object to be subjected to fault injection according to the input parameter, and injecting an executable instruction corresponding to the target event into the object according to the mapping relation between the event and the executable instruction, thereby realizing fault injection. The fault event in the fault event pool is dynamically monitored, and the target event needing to be injected is selected from the fault event pool based on the severity of the influence generated by the fault event, so that manual selection is avoided, the object of fault injection is appointed during injection, the test range is limited, and the pertinence and the effectiveness of fault injection are improved.
In order to achieve the above embodiment, the present application further proposes a fault injection device.
Fig. 4 is a schematic structural diagram of a fault injection apparatus according to an embodiment of the present application.
As shown in fig. 4, the fault injection apparatus includes: a monitoring module 41, a selection module 42, a determination module 43 and an injection module 44.
The monitoring module 41 is configured to monitor each fault event in the fault event pool, and determine an evaluation value of each fault event, where the evaluation value is used to indicate a severity of an impact generated by a corresponding fault event.
And a selection module 42, configured to select a target event to be injected from the fault event pool according to the evaluation value of the set dimension.
The determining module 43 is configured to determine an object to be subjected to fault injection according to an input parameter of fault injection received by the application program interface.
And the injection module 44 is configured to inject the executable instruction corresponding to the target event into the object according to the mapping relationship between the event and the executable instruction.
Optionally, the monitoring module 41 is configured to:
counting event occurrence frequency and event influence range for each fault event in the fault event pool;
querying event introduction cost and introduction complexity of each fault event configuration in the fault event pool;
and aiming at any fault event, the event occurrence frequency, the event influence range, the event introduction cost and the introduction complexity are respectively used as evaluation values of corresponding dimensions.
Optionally, the monitoring module 41 is configured to:
for any fault event, determining the occurrence frequency of the event according to the occurrence times of the corresponding fault event in at least one statistical period; and/or the number of the groups of groups,
and aiming at any fault event, determining the event influence range according to the micro-service duty ratio of the corresponding fault event in the distributed micro-service system in at least one statistical period.
Optionally, the selection module 42 is configured to:
for each fault event, carrying out weighted summation on the evaluation values of a plurality of set dimensions to determine a total evaluation value;
ranking according to the total evaluation value of each fault event;
based on the ranking, a target event to be injected is selected from the fault event pool.
Optionally, the determining module 43 is configured to:
responding to input parameters of fault injection received by an application program interface, and acquiring identification, explosion radius and dependency information of a target micro-service in the input parameters, wherein the explosion radius is used for indicating a deployment unit allowing fault injection in deployment units of the target micro-service;
determining a target instance of the target micro-service that is within the radius of explosion based on the radius of explosion and an identification of the target micro-service;
determining a downstream instance, middleware and/or physical equipment on which the target micro-service depends according to the dependency information;
and taking at least one of the target instance, the downstream instance, the middleware and the physical equipment as the object needing fault injection.
Optionally, an injection module 44 for:
an agent for controlling the object installation fault injection;
and under the condition of successful installation, sending the executable instruction corresponding to the target event to the installed agent according to the mapping relation between the event and the executable instruction, so that the agent runs the executable instruction.
Further, in one possible implementation manner of the embodiment of the present application, the input parameters further include an injection start time and an injection end time, and the injection module 44 is configured to:
at the injection start time, sending instruction information for starting injection to the agent so that the agent starts to run the executable instructions;
and at the injection ending time, sending instruction information for ending injection to the agent so as to enable the agent to stop running the executable instruction.
It should be noted that the foregoing explanation of the embodiment of the fault injection method is also applicable to the fault injection apparatus of this embodiment, and will not be repeated herein.
In the embodiment of the application, by monitoring each fault event in the fault event pool, an evaluation value of each fault event is determined, wherein the evaluation value is used for indicating the severity of the influence generated by the corresponding fault event. And selecting a target event to be injected from the fault event pool according to the set dimension evaluation value, responding to an input parameter received by an application program interface and carrying out fault injection, determining an object to be subjected to fault injection according to the input parameter, and injecting an executable instruction corresponding to the target event into the object according to the mapping relation between the event and the executable instruction, thereby realizing fault injection. The fault event in the fault event pool is dynamically monitored, and the target event needing to be injected is selected from the fault event pool based on the severity of the influence generated by the fault event, so that manual selection is avoided, the object of fault injection is appointed during injection, the test range is limited, and the pertinence and the effectiveness of fault injection are improved.
In order to achieve the above embodiments, the present application further proposes an electronic device including: a processor, and a memory communicatively coupled to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the methods provided by the previous embodiments.
In order to implement the above-mentioned embodiments, the present application also proposes a computer-readable storage medium in which computer-executable instructions are stored, which when executed by a processor are adapted to implement the methods provided by the above-mentioned embodiments.
In order to implement the above embodiments, the present application also proposes a computer program product comprising a computer program which, when executed by a processor, implements the method provided by the above embodiments.
The processes of collecting, storing, using, processing, transmitting, providing, disclosing and the like of the personal information of the user related in the application all accord with the regulations of related laws and regulations, and do not violate the popular public order.
It should be noted that personal information from users should be collected for legitimate and reasonable uses and not shared or sold outside of these legitimate uses. In addition, such collection/sharing should be performed after receiving user informed consent, including but not limited to informing the user to read user agreements/user notifications and signing agreements/authorizations including authorization-related user information before the user uses the functionality. In addition, any necessary steps are taken to safeguard and ensure access to such personal information data and to ensure that other persons having access to the personal information data adhere to their privacy policies and procedures.
The present application contemplates embodiments that may provide a user with selective prevention of use or access to personal information data. That is, the present disclosure contemplates that hardware and/or software may be provided to prevent or block access to such personal information data. Once personal information data is no longer needed, risk can be minimized by limiting data collection and deleting data. In addition, personal identification is removed from such personal information, as applicable, to protect the privacy of the user.
In the foregoing descriptions of embodiments, descriptions of the terms "one embodiment," "some embodiments," "example," "particular example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present application, the meaning of "plurality" is at least two, such as two, three, etc., unless explicitly defined otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and additional implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present application.
Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
It is to be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. As with the other embodiments, if implemented in hardware, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
Those of ordinary skill in the art will appreciate that all or a portion of the steps carried out in the method of the above-described embodiments may be implemented by a program to instruct related hardware, where the program may be stored in a computer readable storage medium, and where the program, when executed, includes one or a combination of the steps of the method embodiments.
In addition, each functional unit in each embodiment of the present application may be integrated in one processing module, or each unit may exist alone physically, or two or more units may be integrated in one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules may also be stored in a computer readable storage medium if implemented in the form of software functional modules and sold or used as a stand-alone product.
The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, or the like. Although embodiments of the present application have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the application, and that variations, modifications, alternatives, and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the application.

Claims (18)

1. A fault injection method, comprising the steps of:
monitoring each fault event in a fault event pool, and determining an evaluation value of each fault event, wherein the evaluation value is used for indicating the severity of influence generated by the corresponding fault event;
selecting a target event to be injected from the fault event pool according to the evaluation value of the set dimension;
responding to input parameters of fault injection received by an application program interface, and determining an object to be subjected to fault injection according to the input parameters;
and injecting the executable instruction corresponding to the target event into the object according to the mapping relation between the event and the executable instruction.
2. The method of claim 1, wherein monitoring each fault event in the fault event pool to determine an evaluation value for each fault event comprises:
counting event occurrence frequency and event influence range for each fault event in the fault event pool;
querying event introduction cost and introduction complexity of each fault event configuration in the fault event pool;
and aiming at any fault event, the event occurrence frequency, the event influence range, the event introduction cost and the introduction complexity are respectively used as evaluation values of corresponding dimensions.
3. The method of claim 2, wherein said counting each fault event in said fault event pool to obtain an event occurrence frequency and an event impact range comprises:
for any fault event, determining the occurrence frequency of the event according to the occurrence times of the corresponding fault event in at least one statistical period; and/or the number of the groups of groups,
and aiming at any fault event, determining the event influence range according to the micro-service duty ratio of the corresponding fault event in the distributed micro-service system in at least one statistical period.
4. The method according to claim 2, wherein selecting the target event to be injected from the fault event pool according to the set dimensional evaluation value comprises:
for each fault event, carrying out weighted summation on the evaluation values of a plurality of set dimensions to determine a total evaluation value;
ranking according to the total evaluation value of each fault event;
based on the ranking, a target event to be injected is selected from the fault event pool.
5. The method according to any one of claims 1-4, wherein the determining an object to be subjected to fault injection according to the input parameters in response to the application program interface receiving the input parameters of the fault injection comprises:
responding to input parameters of fault injection received by an application program interface, and acquiring identification, explosion radius and dependency information of a target micro-service in the input parameters, wherein the explosion radius is used for indicating a deployment unit allowing fault injection in deployment units of the target micro-service;
determining a target instance of the target micro-service that is within the radius of explosion based on the radius of explosion and an identification of the target micro-service;
determining a downstream instance, middleware and/or physical equipment on which the target micro-service depends according to the dependency information;
and taking at least one of the target instance, the downstream instance, the middleware and the physical equipment as the object needing fault injection.
6. The method of claim 5, wherein the deployment unit comprises at least one of a cluster, a server, and an instance;
the physical device includes at least one of a processor, a disk, and a gateway device.
7. The method according to any one of claims 1-4, wherein the injecting the executable instructions corresponding to the target event into the object according to the mapping relationship between the event and the executable instructions comprises:
an agent for controlling the object installation fault injection;
and under the condition of successful installation, sending the executable instruction corresponding to the target event to the installed agent according to the mapping relation between the event and the executable instruction, so that the agent runs the executable instruction.
8. The method of claim 7, wherein the input parameters further include an injection start time and an injection end time, the method further comprising:
at the injection start time, sending instruction information for starting injection to the agent so that the agent starts to run the executable instructions;
and at the injection ending time, sending instruction information for ending injection to the agent so as to enable the agent to stop running the executable instruction.
9. A fault injection apparatus, comprising:
the monitoring module is used for monitoring each fault event in the fault event pool and determining an evaluation value of each fault event, wherein the evaluation value is used for indicating the severity of the influence generated by the corresponding fault event;
the selection module is used for selecting target events to be injected from the fault event pool according to the evaluation value of the set dimension;
the determining module is used for responding to the input parameters of fault injection received by the application program interface and determining an object to be subjected to fault injection according to the input parameters;
and the injection module is used for injecting the executable instruction corresponding to the target event into the object according to the mapping relation between the event and the executable instruction.
10. The apparatus of claim 9, wherein the monitoring module is configured to:
counting event occurrence frequency and event influence range for each fault event in the fault event pool;
querying event introduction cost and introduction complexity of each fault event configuration in the fault event pool;
and aiming at any fault event, the event occurrence frequency, the event influence range, the event introduction cost and the introduction complexity are respectively used as evaluation values of corresponding dimensions.
11. The apparatus of claim 10, wherein the monitoring module is configured to:
for any fault event, determining the occurrence frequency of the event according to the occurrence times of the corresponding fault event in at least one statistical period; and/or the number of the groups of groups,
and aiming at any fault event, determining the event influence range according to the micro-service duty ratio of the corresponding fault event in the distributed micro-service system in at least one statistical period.
12. The apparatus of claim 10, wherein the selection module is configured to:
for each fault event, carrying out weighted summation on the evaluation values of a plurality of set dimensions to determine a total evaluation value;
ranking according to the total evaluation value of each fault event;
based on the ranking, a target event to be injected is selected from the fault event pool.
13. The apparatus according to any one of claims 9-12, wherein the determining module is configured to:
responding to input parameters of fault injection received by an application program interface, and acquiring identification, explosion radius and dependency information of a target micro-service in the input parameters, wherein the explosion radius is used for indicating a deployment unit allowing fault injection in deployment units of the target micro-service;
determining a target instance of the target micro-service that is within the radius of explosion based on the radius of explosion and an identification of the target micro-service;
determining a downstream instance, middleware and/or physical equipment on which the target micro-service depends according to the dependency information;
and taking at least one of the target instance, the downstream instance, the middleware and the physical equipment as the object needing fault injection.
14. The apparatus of any one of claims 9-12, wherein the injection module is configured to:
an agent for controlling the object installation fault injection;
and under the condition of successful installation, sending the executable instruction corresponding to the target event to the installed agent according to the mapping relation between the event and the executable instruction, so that the agent runs the executable instruction.
15. The apparatus of claim 14, wherein the input parameters further include an injection start time and an injection end time, and wherein the injection module is configured to:
at the injection start time, sending instruction information for starting injection to the agent so that the agent starts to run the executable instructions;
and at the injection ending time, sending instruction information for ending injection to the agent so as to enable the agent to stop running the executable instruction.
16. An electronic device, comprising: a processor, and a memory communicatively coupled to the processor;
the memory stores computer-executable instructions;
the processor executes the computer-executable instructions stored by the memory to implement the method of any one of claims 1-8.
17. A computer readable storage medium having stored therein computer executable instructions which when executed by a processor are adapted to carry out the method of any one of claims 1-8.
18. A computer program product comprising a computer program which, when executed by a processor, implements the method of any of claims 1-8.
CN202311640859.4A 2023-12-01 2023-12-01 Fault injection method and device, electronic equipment and storage medium Pending CN117743153A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311640859.4A CN117743153A (en) 2023-12-01 2023-12-01 Fault injection method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311640859.4A CN117743153A (en) 2023-12-01 2023-12-01 Fault injection method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN117743153A true CN117743153A (en) 2024-03-22

Family

ID=90249960

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311640859.4A Pending CN117743153A (en) 2023-12-01 2023-12-01 Fault injection method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN117743153A (en)

Similar Documents

Publication Publication Date Title
CN109586952B (en) Server capacity expansion method and device
US7237266B2 (en) Electronic vulnerability and reliability assessment
JPH10511524A (en) Failure monitoring
CN103401698B (en) For the monitoring system that server health is reported to the police in server set group operatione
US8381036B2 (en) Systems and methods for restoring machine state history related to detected faults in package update process
CN111881014B (en) System test method, device, storage medium and electronic equipment
CN101170447A (en) Service failure diagnosis system based on active probe and its method
CN108710673A (en) Realize database high availability method, system, computer equipment and storage medium
CN110881059B (en) Application deployment system, method, release engine and computer equipment
CN112783725A (en) Index acquisition method and device
CN112532455B (en) Abnormal root cause positioning method and device
CN113468009B (en) Pressure testing method and device, electronic equipment and storage medium
CN112732474A (en) Fault processing method and device, electronic equipment and computer readable storage medium
CN110618853B (en) Detection method, device and equipment for zombie container
CN102045213B (en) Fault positioning method and device
CN117743153A (en) Fault injection method and device, electronic equipment and storage medium
CN112054925A (en) Method and device for deploying background service
CN114706733B (en) Section program abnormity monitoring method and device
EP1146426B1 (en) Dynamic rule sets for generated logs in a network
CN109634814A (en) Fault early warning method, equipment, storage medium and device based on log stream
CN114546717A (en) Method and device for starting android intelligent terminal, intelligent terminal and storage medium
CN101964922B (en) Abnormal condition capturing method and device
CN114564369B (en) Application program abnormity monitoring method and device, electronic equipment and storage medium
CN113518055A (en) Data security protection processing method and device, storage medium and terminal
CN113138872A (en) Abnormal processing device and method for database system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination