CN112433947A - Chaos engineering method and system based on network data - Google Patents

Chaos engineering method and system based on network data Download PDF

Info

Publication number
CN112433947A
CN112433947A CN202011368339.9A CN202011368339A CN112433947A CN 112433947 A CN112433947 A CN 112433947A CN 202011368339 A CN202011368339 A CN 202011368339A CN 112433947 A CN112433947 A CN 112433947A
Authority
CN
China
Prior art keywords
data
network
test
module
transaction
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011368339.9A
Other languages
Chinese (zh)
Inventor
杜匡俊
蔡晓华
杨光辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Netis Technologies Co ltd
Original Assignee
Shanghai Netis Technologies Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Netis Technologies Co ltd filed Critical Shanghai Netis Technologies Co ltd
Priority to CN202011368339.9A priority Critical patent/CN112433947A/en
Publication of CN112433947A publication Critical patent/CN112433947A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/36Preventing errors by testing or debugging software
    • G06F11/3668Software testing
    • G06F11/3672Test management
    • G06F11/3688Test management for test execution, e.g. scheduling of test suites
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/36Preventing errors by testing or debugging software
    • G06F11/3664Environments for testing or debugging software
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/36Preventing errors by testing or debugging software
    • G06F11/3668Software testing
    • G06F11/3672Test management
    • G06F11/3684Test management for test design, e.g. generating new test cases

Abstract

The invention provides a chaotic engineering method and system based on network data, comprising the following steps: step M1: acquiring a network message in a production environment through a network packet capturing technology, analyzing the network message, generating and storing structured transaction data; step M2: preprocessing the structured transaction data to obtain and store the preprocessed transaction data; step M3: initializing a random data generator according to the test definition; step M4: selecting required test case data from the preprocessed transaction data according to test requirements, randomly modifying data in a preset proportion in the test case data by using a random data generator to generate an abnormal data packet, sending the abnormal data packet to a tested software system, monitoring the state of the tested software system, and generating a test report. The invention is a data-driven chaotic engineering method, which is different from other traditional methods for simulating unexpected faults, can detect and cover more real scenes, and improves the fault discovery rate.

Description

Chaos engineering method and system based on network data
Technical Field
The invention relates to the technical field of chaotic engineering, in particular to a chaotic engineering method and system based on network data.
Background
Software testing is an important link in the software product development process and is used for finding software defects and guaranteeing software quality.
The prior testing technique has the following disadvantages
1. The white box test requires a tester to understand and analyze source codes, comprehensively understand the internal logic structure of a program, and test all logic paths. Therefore, the white box has the following disadvantages:
a. the testing cost is high;
b. failure to detect both code and data sensitivity errors;
c. the correctness of the specification is not verified;
2. the black box test refers to a test according to functional specifications without considering the internal structure and internal characteristics of a program at all. The black box test has the following disadvantages:
a. complete, non-exhaustive testing is not possible;
b. the input mainly depends on the specification, but the specification can have omission and errors;
3. the gray box test is a test between the white box test and the black box test. Disadvantages of the Ash-Box test:
a. not applicable to simple systems;
b. the requirement on testers is high;
meanwhile, the methods are all artificial methods, and have the following defects:
1. the method comprises the steps of defining a test case, including preparing a test environment, test data, preconditions, and testing, and judging whether a test result is correct or not; the test does not go beyond human comprehension.
2. If necessary, the script needs to be manually translated or developed into a script capable of being repeatedly operated, and the script is repeatedly operated as an 'automatic' test;
3. both test cases and automation cases require manual maintenance according to functional changes.
In view of the above-mentioned drawbacks of the prior art, the technical problems to be solved by the present invention are as follows:
1) rapidly acquiring transaction data by using a network packet capturing technology;
2) reconstructing the transaction data to obtain test data;
3) and injecting the test data into the system for testing.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a chaotic engineering method and system based on network data.
The chaos engineering method based on network data provided by the invention comprises the following steps:
step M1: acquiring a network message in a production environment through a network packet capturing technology, analyzing the network message, generating and storing structured transaction data;
step M2: preprocessing the structured transaction data to obtain and store the preprocessed transaction data;
step M3: initializing a random data generator according to the test definition;
step M4: selecting required test case data from the preprocessed transaction data according to test requirements, randomly modifying data in a preset proportion in the test case data by using a random data generator to generate an abnormal data packet, sending the abnormal data packet to a tested software system, monitoring the state of the tested software system, and generating a test report.
Preferably, the step M1 includes:
step M1.1: in a transaction environment, network mirroring or network road assisting equipment is adopted, network data is exported as a data message under the condition that transaction is not influenced, and the network message is acquired;
step M1.2: analyzing network data of the network message to obtain flat key value pair information or structured information;
step M1.3: and distinguishing the request and the response of the flat key value pair information or the structured information, associating a plurality of information by adopting a transaction association technology to obtain structured transaction data, and storing the structured transaction data.
Preferably, said step M1.2 comprises: the analysis method comprises data packet decoding, deep packet analysis, regular expression, rule matching and/or character string searching.
Preferably, the M2 preprocessing includes: filtering and matching transaction data meeting preset conditions, filtering sensitive information, calling simulation of an external system and modifying a user name.
Preferably, the step M4 includes:
step M4.1: setting starting parameters according to the test requirements, and starting the chaotic engineering test;
step M4.2: loading test case data required by the test according to the starting parameters;
step M4.3: randomly modifying data in a preset proportion in the test case data by using a random data generator to generate an abnormal data packet;
step M4.4: sending the generated abnormal data packet to a tested software system;
step M4.5: and monitoring the state of the tested software system, ensuring that the tested software system still normally operates under the condition of receiving the abnormal data packet, and generating a test report.
The invention provides a chaotic engineering system based on network data, which comprises:
module M1: acquiring a network message in a production environment through a network packet capturing technology, analyzing the network message, generating and storing structured transaction data;
module M2: preprocessing the structured transaction data to obtain and store the preprocessed transaction data;
module M3: initializing a random data generator according to the test definition;
module M4: selecting required test case data from the preprocessed transaction data according to test requirements, randomly modifying data in a preset proportion in the test case data by using a random data generator to generate an abnormal data packet, sending the abnormal data packet to a tested software system, monitoring the state of the tested software system, and generating a test report.
Preferably, said module M1 comprises:
module M1.1: in a transaction environment, network mirroring or network road assisting equipment is adopted, network data is exported as a data message under the condition that transaction is not influenced, and the network message is acquired;
module M1.2: analyzing network data of the network message to obtain flat key value pair information or structured information;
module M1.3: and distinguishing the request and the response of the flat key value pair information or the structured information, associating a plurality of information by adopting a transaction association technology to obtain structured transaction data, and storing the structured transaction data.
Preferably, said module M1.2 comprises: the analysis method comprises data packet decoding, deep packet analysis, regular expression, rule matching and/or character string searching.
Preferably, the preprocessing in the module M2 includes: filtering and matching transaction data meeting preset conditions, filtering sensitive information, calling simulation of an external system and modifying a user name.
Preferably, said module M4 comprises:
module M4.1: setting starting parameters according to the test requirements, and starting the chaotic engineering test;
module M4.2: loading test case data required by the test according to the starting parameters;
module M4.3: randomly modifying data in a preset proportion in the test case data by using a random data generator to generate an abnormal data packet;
module M4.4: sending the generated abnormal data packet to a tested software system;
module M4.5: and monitoring the state of the tested software system, ensuring that the tested software system still normally operates under the condition of receiving the abnormal data packet, and generating a test report.
Compared with the prior art, the invention has the following beneficial effects:
1. the invention is a data-driven chaotic engineering method, which is different from other traditional methods for simulating unexpected faults, can detect and cover more real scenes, and improves the discovery rate of the faults;
2. the test data of the invention is automatically generated through network packet capturing and data analysis based on the real data of the user, the generation efficiency is high, the simulation actual environment degree is high, and the test effect is obvious;
3. the test cases are automatically generated, so that the case creation cost and the maintenance cost are low;
4. the test is independent of the specification, and the conditions of omission and errors in the specification can be tested.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
fig. 1 is a flow chart of a chaotic engineering method based on network data.
Detailed Description
The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that it would be obvious to those skilled in the art that various changes and modifications can be made without departing from the spirit of the invention. All falling within the scope of the present invention.
Example 1
The chaos engineering method based on network data provided by the invention comprises the following steps:
step M1: acquiring a network message in a production environment through a network packet capturing technology, analyzing the network message, generating and storing structured transaction data;
step M2: preprocessing the structured transaction data to obtain and store the preprocessed transaction data;
step M3: initializing a random data generator according to the test definition;
step M4: selecting required test case data from the preprocessed transaction data according to test requirements, randomly modifying data in a preset proportion in the test case data by using a random data generator to generate an abnormal data packet, sending the abnormal data packet to a tested software system, monitoring the state of the tested software system, and generating a test report.
Specifically, the step M1 includes:
step M1.1: in a transaction environment, network mirroring or network road assisting equipment is adopted, network data is exported as a data message under the condition that transaction is not influenced, and the network message is acquired;
step M1.2: analyzing network data of the network message to obtain flat key value pair information or structured information;
step M1.3: and distinguishing the request and the response of the flat key value pair information or the structured information, associating a plurality of information by adopting a transaction association technology to obtain structured transaction data, and storing the structured transaction data.
In particular, said step M1.2 comprises: the analysis method comprises data packet decoding, deep packet analysis, regular expression, rule matching and/or character string searching.
Specifically, the M2 preprocessing includes: filtering and matching transaction data meeting preset conditions, filtering sensitive information, calling simulation of an external system and modifying a user name.
Specifically, the step M4 includes:
step M4.1: setting starting parameters according to the test requirements, and starting the chaotic engineering test;
step M4.2: loading test case data required by the test according to the starting parameters;
step M4.3: randomly modifying data in a preset proportion in the test case data by using a random data generator to generate an abnormal data packet;
step M4.4: sending the generated abnormal data packet to a tested software system;
step M4.5: and monitoring the state of the tested software system, ensuring that the tested software system still normally operates under the condition of receiving the abnormal data packet, and generating a test report.
The invention provides a chaotic engineering system based on network data, which comprises:
module M1: acquiring a network message in a production environment through a network packet capturing technology, analyzing the network message, generating and storing structured transaction data;
module M2: preprocessing the structured transaction data to obtain and store the preprocessed transaction data;
module M3: initializing a random data generator according to the test definition;
module M4: selecting required test case data from the preprocessed transaction data according to test requirements, randomly modifying data in a preset proportion in the test case data by using a random data generator to generate an abnormal data packet, sending the abnormal data packet to a tested software system, monitoring the state of the tested software system, and generating a test report.
Specifically, the module M1 includes:
module M1.1: in a transaction environment, network mirroring or network road assisting equipment is adopted, network data is exported as a data message under the condition that transaction is not influenced, and the network message is acquired;
module M1.2: analyzing network data of the network message to obtain flat key value pair information or structured information;
module M1.3: and distinguishing the request and the response of the flat key value pair information or the structured information, associating a plurality of information by adopting a transaction association technology to obtain structured transaction data, and storing the structured transaction data.
In particular, said module M1.2 comprises: the analysis method comprises data packet decoding, deep packet analysis, regular expression, rule matching and/or character string searching.
Specifically, the preprocessing in the module M2 includes: filtering and matching transaction data meeting preset conditions, filtering sensitive information, calling simulation of an external system and modifying a user name.
Specifically, the module M4 includes:
module M4.1: setting starting parameters according to the test requirements, and starting the chaotic engineering test;
module M4.2: loading test case data required by the test according to the starting parameters;
module M4.3: randomly modifying data in a preset proportion in the test case data by using a random data generator to generate an abnormal data packet;
module M4.4: sending the generated abnormal data packet to a tested software system;
module M4.5: and monitoring the state of the tested software system, ensuring that the tested software system still normally operates under the condition of receiving the abnormal data packet, and generating a test report.
Example 2
Example 2 is a modification of example 1
The method comprises the following steps: as shown in figure 1 of the drawings, in which,
step 1: and (5) data acquisition process. Collecting transaction data in the environment through network packet capturing;
step 2: and (5) testing and constructing a flow. Automatically generating a test case based on the collected transaction data according to a test strategy;
and step 3: and (5) testing and executing the flow. Arranging according to requirements, injecting test cases into the system, and carrying out automatic test;
wherein, step 1 includes the following steps:
step 1.1: network data is collected. In a real transaction environment, network mirroring (TAP) or network side equipment (SPAN) is adopted, and network data is exported in a data message and collected under the condition of not influencing the transaction. This method is also called network packet grabbing;
step 1.2: network data is analyzed.
Parsing methods include, but are not limited to, packet decoding (decode), deep packet analysis (DPI), Regular expression (Regular expression), rule matching, string search, and the like.
The result of the analysis is flat key-value pair information, or structured information, which constitutes a message. The structuring includes, but is not limited to XML, JSON, Protocol buffer, YAML, TOML, and the like.
In a complex scenario, a transaction management technology is also needed to distinguish a request and a response for a message, and associate a plurality of messages to form a transaction, so as to ensure the validity and accuracy of data.
Step 1.3: the analytical data is stored. This is an optional step. The analysis data is stored, and the test case can be generated repeatedly.
Wherein, the step 2 comprises the following steps:
step 2.1: and defining a test case. And designing a test case according to the test requirement. Selecting a network data packet, a random data generation strategy and the like;
step 2.2: and loading the analysis result. Loading the data stored in the step 1.3 from the persistent storage and matching out a network data packet matching the use case range;
step 2.3: and generating a test case. Initializing a random data generator according to the test definition;
step 2.4: and storing the test case. The test cases are stored. And (5) optional steps. The generated test case is stored, and repeated test execution can be performed.
Wherein, the step 3 comprises the following steps:
step 3.1: and (6) arranging and testing. And starting a chaotic engineering test once according to the test requirement. Starting parameters comprises selecting contained test case sets;
step 3.2: and loading the test case. Loading test case data required by the test according to the starting parameters in the step 3.1;
step 3.3: and generating a random data message. And randomly modifying the data with a certain proportion in the original request message to generate an abnormal data packet sent to the tested system.
Step 3.4: and sending the random data message. And 3, sending the abnormal data packet generated in the step 3.3 to the tested system.
Step 3.5: and monitoring the tested system. Monitoring the state of the system to be tested to ensure that the system to be tested can still normally run under the condition of receiving the abnormal data packet;
step 3.6: and generating a test report. Generating a test report according to the monitoring result of the step 3.5;
those skilled in the art will appreciate that, in addition to implementing the systems, apparatus, and various modules thereof provided by the present invention in purely computer readable program code, the same procedures can be implemented entirely by logically programming method steps such that the systems, apparatus, and various modules thereof are provided in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Therefore, the system, the device and the modules thereof provided by the present invention can be considered as a hardware component, and the modules included in the system, the device and the modules thereof for implementing various programs can also be considered as structures in the hardware component; modules for performing various functions may also be considered to be both software programs for performing the methods and structures within hardware components.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The embodiments and features of the embodiments of the present application may be combined with each other arbitrarily without conflict.

Claims (10)

1. A chaos engineering method based on network data is characterized by comprising the following steps:
step M1: acquiring a network message in a production environment through a network packet capturing technology, analyzing the network message, generating and storing structured transaction data;
step M2: preprocessing the structured transaction data to obtain and store the preprocessed transaction data;
step M3: initializing a random data generator according to the test definition;
step M4: selecting required test case data from the preprocessed transaction data according to test requirements, randomly modifying data in a preset proportion in the test case data by using a random data generator to generate an abnormal data packet, sending the abnormal data packet to a tested software system, monitoring the state of the tested software system, and generating a test report.
2. The chaotic engineering method based on network data according to claim 1, wherein the step M1 comprises:
step M1.1: in a transaction environment, network mirroring or network road assisting equipment is adopted, network data is exported as a data message under the condition that transaction is not influenced, and the network message is acquired;
step M1.2: analyzing network data of the network message to obtain flat key value pair information or structured information;
step M1.3: and distinguishing the request and the response of the flat key value pair information or the structured information, associating a plurality of information by adopting a transaction association technology to obtain structured transaction data, and storing the structured transaction data.
3. The chaotic engineering method based on network data according to claim 2, wherein the step M1.2 comprises: the analysis method comprises data packet decoding, deep packet analysis, regular expression, rule matching and/or character string searching.
4. The chaotic engineering method based on network data according to claim 1, wherein the preprocessing in M2 comprises: filtering and matching transaction data meeting preset conditions, filtering sensitive information, calling simulation of an external system and modifying a user name.
5. The chaotic engineering method based on network data according to claim 1, wherein the step M4 comprises:
step M4.1: setting starting parameters according to the test requirements, and starting the chaotic engineering test;
step M4.2: loading test case data required by the test according to the starting parameters;
step M4.3: randomly modifying data in a preset proportion in the test case data by using a random data generator to generate an abnormal data packet;
step M4.4: sending the generated abnormal data packet to a tested software system;
step M4.5: and monitoring the state of the tested software system, ensuring that the tested software system still normally operates under the condition of receiving the abnormal data packet, and generating a test report.
6. A chaotic engineering system based on network data is characterized by comprising:
module M1: acquiring a network message in a production environment through a network packet capturing technology, analyzing the network message, generating and storing structured transaction data;
module M2: preprocessing the structured transaction data to obtain and store the preprocessed transaction data;
module M3: initializing a random data generator according to the test definition;
module M4: selecting required test case data from the preprocessed transaction data according to test requirements, randomly modifying data in a preset proportion in the test case data by using a random data generator to generate an abnormal data packet, sending the abnormal data packet to a tested software system, monitoring the state of the tested software system, and generating a test report.
7. The chaotic engineering system based on network data as claimed in claim 6, wherein the module M1 comprises:
module M1.1: in a transaction environment, network mirroring or network road assisting equipment is adopted, network data is exported as a data message under the condition that transaction is not influenced, and the network message is acquired;
module M1.2: analyzing network data of the network message to obtain flat key value pair information or structured information;
module M1.3: and distinguishing the request and the response of the flat key value pair information or the structured information, associating a plurality of information by adopting a transaction association technology to obtain structured transaction data, and storing the structured transaction data.
8. The chaotic engineering system based on network data according to claim 7, wherein the module M1.2 comprises: the analysis method comprises data packet decoding, deep packet analysis, regular expression, rule matching and/or character string searching.
9. The chaotic engineering system based on network data as claimed in claim 6, wherein the preprocessing in the module M2 includes: filtering and matching transaction data meeting preset conditions, filtering sensitive information, calling simulation of an external system and modifying a user name.
10. The chaotic engineering system based on network data as claimed in claim 6, wherein the module M4 comprises:
module M4.1: setting starting parameters according to the test requirements, and starting the chaotic engineering test;
module M4.2: loading test case data required by the test according to the starting parameters;
module M4.3: randomly modifying data in a preset proportion in the test case data by using a random data generator to generate an abnormal data packet;
module M4.4: sending the generated abnormal data packet to a tested software system;
module M4.5: and monitoring the state of the tested software system, ensuring that the tested software system still normally operates under the condition of receiving the abnormal data packet, and generating a test report.
CN202011368339.9A 2020-11-30 2020-11-30 Chaos engineering method and system based on network data Pending CN112433947A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011368339.9A CN112433947A (en) 2020-11-30 2020-11-30 Chaos engineering method and system based on network data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011368339.9A CN112433947A (en) 2020-11-30 2020-11-30 Chaos engineering method and system based on network data

Publications (1)

Publication Number Publication Date
CN112433947A true CN112433947A (en) 2021-03-02

Family

ID=74698491

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011368339.9A Pending CN112433947A (en) 2020-11-30 2020-11-30 Chaos engineering method and system based on network data

Country Status (1)

Country Link
CN (1) CN112433947A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113515449A (en) * 2021-05-19 2021-10-19 中国工商银行股份有限公司 Chaos test method, system, electronic equipment and storage medium

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105099833A (en) * 2015-09-14 2015-11-25 北京华青融天技术有限责任公司 Business test method , device and system
CN111679979A (en) * 2020-06-03 2020-09-18 中国银行股份有限公司 Destructive testing method and device

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105099833A (en) * 2015-09-14 2015-11-25 北京华青融天技术有限责任公司 Business test method , device and system
CN111679979A (en) * 2020-06-03 2020-09-18 中国银行股份有限公司 Destructive testing method and device

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113515449A (en) * 2021-05-19 2021-10-19 中国工商银行股份有限公司 Chaos test method, system, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
CN105068925B (en) Software safety defect finds system
CN112052172B (en) Rapid test method and device for third-party channel and electronic equipment
CN111309343B (en) Development deployment method and device
US20070061641A1 (en) Apparatus and method for generating test driver
CN112433948A (en) Simulation test system and method based on network data analysis
CN110851352A (en) Fuzzy test system and terminal equipment
CN116383833A (en) Method and device for testing software program code, electronic equipment and storage medium
CN109815124B (en) MBSE-based interlocking function defect analysis method and device and interlocking system
CN112433947A (en) Chaos engineering method and system based on network data
CN117493188A (en) Interface testing method and device, electronic equipment and storage medium
CN111651342A (en) HTTP interface automation testing framework based on Charles recording session
CN115934559A (en) Testing method of intelligent form testing system
CN111708712A (en) User behavior test case generation method, flow playback method and electronic equipment
CN116431522A (en) Automatic test method and system for low-code object storage gateway
CN111475358A (en) Automatic testing method and device for controller interface
Thooriqoh et al. Selenium Framework for Web Automation Testing: A Systematic Literature Review
CN112464237B (en) Static code security diagnosis method and device
CN113079061B (en) Internet of things performance testing method and system
CN115470141A (en) Fault simulation method, device and related equipment
CN112860587B (en) UI automatic test method and device
CN111767218B (en) Automatic test method, equipment and storage medium for continuous integration
CN114064489A (en) Automatic testing method, device, equipment and readable storage medium
CN111026631B (en) Automatic interface detection method, device and server
CN113342641A (en) Automatic test method and system for HTTP service interface
Allen et al. A model-based approach to the security testing of network protocol implementations

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination