CN113037587A

CN113037587A - TCP/IP protocol stack testing method and device and electronic equipment

Info

Publication number: CN113037587A
Application number: CN202110220893.0A
Authority: CN
Inventors: 白家驹; 谈鉴锋; 周介龙; 邹永浩; 胡事民
Original assignee: Tsinghua University; Alipay Hangzhou Information Technology Co Ltd
Current assignee: Tsinghua University; Alipay Hangzhou Information Technology Co Ltd
Priority date: 2021-02-26
Filing date: 2021-02-26
Publication date: 2021-06-25
Anticipated expiration: 2041-02-26
Also published as: CN113037587B

Abstract

The embodiment of the specification discloses a TCP/IP protocol stack testing method, a device and electronic equipment, and the technical scheme to be protected comprises the steps of utilizing all data items to sequentially run a TCP/IP protocol stack to be tested according to the dependency relationship among all data items contained in a testing sequence, monitoring the running process of the TCP/IP protocol stack to be tested to obtain first monitoring data, and utilizing the first monitoring data to determine the testing result of the TCP/IP protocol stack to be tested.

Description

TCP/IP protocol stack testing method and device and electronic equipment

Technical Field

The embodiment of the specification relates to the technical field of computers, in particular to a method and a device for testing a TCP/IP protocol stack and electronic equipment.

Background

The TCP/IP protocol stack is the sum of a series of network protocols and is the core framework that constitutes network communications. In particular, the TCP/IP protocol stack provides a mechanism for point-to-point links, standardizing how data should be encapsulated, addressed, transmitted, routed, and received at the destination. The software communication process is abstracted into four abstraction layers, different communication protocols are respectively realized in a protocol stack mode, and each layer calls a protocol provided by the next layer to realize the requirement of the layer.

To enhance the test integrity and reliability of the protocol stack, automated testing of the TCP/IP protocol stack is typically required.

Disclosure of Invention

In view of this, embodiments of the present disclosure provide a TCP/IP protocol stack testing method and apparatus for improving TCP/IP protocol stack testing efficiency, and an electronic device.

The embodiment of the specification adopts the following technical scheme:

an embodiment of the present specification provides a TCP/IP protocol stack testing method, including:

according to the dependency relationship among all data items contained in the test sequence, sequentially operating a TCP/IP protocol stack to be tested by using all the data items, wherein all the data items comprise one or two of an input data packet and system call;

monitoring the running process of the TCP/IP protocol stack to be tested to obtain first monitoring data;

and determining a test result of the TCP/IP protocol stack to be tested by using the first monitoring data.

An embodiment of the present specification further provides a TCP/IP protocol stack testing apparatus, including:

the running module is used for running the TCP/IP protocol stack to be tested in sequence by utilizing the data items according to the dependency relationship among the data items contained in the test sequence, wherein the data items comprise one or two of input data packets and system calls;

the monitoring module is used for monitoring the running process of the TCP/IP protocol stack to be detected to obtain first monitoring data;

and the determining module is used for determining the test result of the TCP/IP protocol stack to be tested by using the first monitoring data.

An embodiment of the present specification further provides an electronic device, including:

a processor; and

a memory configured to store a computer program that, when executed, causes the processor to: according to the dependency relationship among all data items contained in the test sequence, sequentially operating a TCP/IP protocol stack to be tested by using all the data items, wherein all the data items comprise one or two of an input data packet and system call;

The embodiment of the specification adopts at least one technical scheme which can achieve the following beneficial effects:

the test sequence in the embodiment of the present specification fully considers the dependency relationship among the data items, and sequentially runs the TCP/IP protocol stack to be tested according to the dependency relationship. The test sequence obtained based on the dependency relationship can be identified as legal by the TCP/IP protocol stack to be tested, so that the stability of the test of the TCP/IP protocol stack is ensured, and the test efficiency is improved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the embodiments of the specification and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the specification and together with the description serve to explain the application and not to limit the application. In the drawings:

fig. 1 is a flowchart of a TCP/IP protocol stack testing method provided in an embodiment of the present specification;

fig. 2 is a flowchart of an application example of a TCP/IP protocol stack testing method provided in an embodiment of the present specification;

fig. 3 is a schematic diagram illustrating states and state changes in a TCP/IP protocol stack according to an embodiment of the present disclosure;

fig. 4 is a flowchart of a TCP/IP protocol stack testing method provided in an embodiment of the present specification;

fig. 5 is a flowchart of a TCP/IP protocol stack testing method proposed in an embodiment of the present specification;

fig. 6 is a flowchart of a TCP/IP protocol stack testing method proposed in an embodiment of the present disclosure;

fig. 7 is a structural diagram of a TCP/IP protocol stack testing apparatus according to an embodiment of the present disclosure;

fig. 8 is a schematic diagram illustrating a more specific hardware structure of a computing device according to an embodiment of the present disclosure.

Detailed Description

The inventor finds that the test efficiency of the existing test scheme is low in the process of testing the TCP/IP protocol stack. However, since the TCP/IP protocol stack testing process includes many links, and specifically, in which link needs to be improved, further research is required.

Firstly, analyzing the existing TCP/IP protocol stack test scheme, and particularly adopting a fuzzy test method. Fuzz testing is an automated or semi-automated testing technique that is often used to discover errors and security issues in the code of software/operating systems/networks. By automatically constructing random or abnormal data input, whether the software or the system has defects when processing different input is detected.

When the test scheme is researched, the phenomenon of packet loss of a TCP/IP protocol stack is found in the test process. The inventors have further found that in a general fuzz testing method, the input is a single data, and the input in the TCP/IP protocol stack is a sequence of data items consisting of a system call and a data packet, which is randomly determined.

Through continuous testing, it is found that, because certain dependency exists among the data items contained in the data item sequence, but because of random input, the dependency among the data items is confused, so that the TCP/IP protocol stack recognizes that the input format of the data item sequence is illegal, and directly discards the data item sequence, thereby affecting the testing efficiency.

Therefore, for the problem of low test efficiency, the inventor of the present application has conducted a great deal of research and summary on each link of the existing test scheme to find the reason of low test efficiency, and then proposes an improvement scheme of the embodiments of the present specification.

In order to make the objects, technical solutions and advantages of the present application more clear, the technical solutions of the present application will be clearly and completely described below with reference to the specific embodiments of the present specification and the accompanying drawings. It is to be understood that the embodiments described are only a few embodiments of the present disclosure, and not all embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments in the present specification without any inventive step are within the scope of the present application.

The technical solutions provided by the embodiments of the present description are described in detail below with reference to the accompanying drawings.

Fig. 1 is a flowchart of a TCP/IP protocol stack testing method provided in an embodiment of the present specification, where the method specifically includes:

step 101: and according to the dependency relationship among the data items contained in the test sequence, sequentially running a TCP/IP protocol stack to be tested by using the data items, wherein the data items comprise one or two of input data packets and system calls.

Step 103: and monitoring the running process of the TCP/IP protocol stack to be tested to obtain first monitoring data.

Step 105: and determining a test result of the TCP/IP protocol stack to be tested by using the first monitoring data.

In the embodiment of the present specification, the TCP/IP protocol stack to be tested is sequentially run by using each data item, specifically, each data item is sequentially input into the TCP/IP protocol stack to be tested according to the dependency relationship between each data item, so as to execute the TCP/IP protocol stack to be tested.

Dependencies, as referred to herein, may include:

input data packets and dependencies between input data packets;

inputting a dependency relationship between a data packet and a system call;

system calls and dependencies between system calls.

Specifically, the input data packet is input to the TCP/IP protocol stack to be tested in a specific format, and one condition of the dependency relationship between the input data packets is that the input data packet needs to be correspondingly increased according to the size of the previous input data packet, so as to avoid the input data packet being discarded by the TCP/IP protocol stack to be tested.

Specifically, the system call referred to in the embodiments of the present specification refers to a system call interface, such as a network-related system call interface of socket, connect, send to, and revfrom, and thus, the system call referred to in step 101 may be a system call interface identifier. The dependency relationship between system calls, for example, when establishing a TCP connection, four system calls of socket, bind, list and accept need to be used in sequence to avoid the problem that the TCP connection cannot be established.

Specifically, the dependency between the system call and the input data packet, for example, after using the system call write, the TCP/IP protocol stack returns the corresponding data packet, and at this time, the size of the ACK field in the next input data packet needs to be smaller than or equal to the total number of data of all output data packets received by the test end, so as to avoid that the TCP/IP protocol stack will always try to return the data packet returned after the previous system call.

It should be noted that, in the above embodiments of the present application, all cases where the dependency relationship does not include a dependency relationship are listed, and in a specific application, the dependency relationship may be set according to a specific case, and is not listed any more.

The dependency relationship in the embodiments of the present specification may be pre-configured according to a specific test case, and corresponding data items are combined into a test sequence according to the dependency relationship, and placed in a seed pool for use in each TCP/IP protocol stack test process.

Thus, in the step 101, according to the dependency relationship among the data items included in the test sequence, before the TCP/IP protocol stack to be tested is sequentially run by using the data items, the dependency relationship of the data items can be obtained; and constructing a test sequence based on the dependency relationship.

Alternatively, the corresponding test sequence may be directly retrieved from the seed pool.

In this embodiment of the present description, if the test sequence includes a system call to run the TCP/IP protocol stack to be tested by using the system call, monitoring a running process of the TCP/IP protocol stack to be tested may specifically include:

acquiring a return value of the system call and an output data packet of the TCP/IP protocol stack to be tested;

and determining the first monitoring data according to the return value and the output data packet.

The result of executing each system call is used as a return value, and the output data packet is output from the TCP/IP protocol stack after a series of system calls are executed. The outgoing data packet is associated with a series of system calls that are performed, and there is no necessary association between the outgoing data packet and the incoming data packet. And the input data packet and the system call are used as input to test whether the behavior of the TCP/IP protocol stack to be tested is correct or not.

In this embodiment of the present description, the running of the TCP/IP protocol stack to be tested is based on state driving, and therefore, when the running process of the TCP/IP protocol stack to be tested is monitored, state coverage data in the TCP/IP protocol stack to be tested is specifically monitored, so that the first monitoring data includes the state coverage data.

The state coverage data can reflect the state covered by the current test process of the TCP/IP protocol stack to be tested, and can be represented by using the state coverage rate, which actually refers to the code coverage rate. By means of the state coverage data it is possible to obtain which states are tested and which states are not covered, so that test integrity can be ensured in dependence of the state coverage data.

In a further embodiment of this specification, as shown in fig. 2, monitoring an operation process of a TCP/IP protocol stack to be tested includes:

step 202: for two adjacent data items based on the dependency relationship, respectively acquiring the state coverage data respectively corresponding to the two data items;

step 204: and differentiating the state coverage data respectively obtained by the two adjacent data items to obtain state change data of the TCP/IP protocol stack to be tested, wherein the first monitoring data comprises the state change data.

And obtaining state change data between two states in the TCP/IP protocol stack to be tested through difference. The state change data characterizes the state changes that the TCP/IP protocol stack under test passes from one state to another, e.g. which state changes are covered and which state changes are uncovered. Therefore, in the embodiment of the present specification, the state change data is used as the test feedback, the test feedback may be used as the first monitoring data, and the state change data may be used to test whether the defect of the TCP/IP protocol stack to be tested occurs in the state change process, so as to achieve the comprehensiveness and accuracy of the test.

For example, referring to FIG. 3, in this figure, the program has four possible states S1, S2, S3, and S4, with five possible state changes, including A1, A2, A3, A5, and A5. During execution from state S1 to state S4, it may be reached from S2 or S3 (i.e., state change A2 or A4). Thus, the state change branches specifically covered can be distinguished by the state change data.

Specifically, for two (packet or system call) adjacent Input data items Input1 and Input2 in the test sequence, state override data Cover1 and Cover2 in the TCP/IP protocol stack after each Input are collected.

Wherein each of Cover1 and Cover2 is a one-dimensional array, and each data item corresponds to a status covering data, the data item being 1 represents that the corresponding status is covered, and the data being 0 represents that the corresponding status is not covered. Then, the two one-dimensional arrays are differentiated to obtain a difference vector CoverVar1:

CoverVar₁＝Cover1–Cover2

if the test sequence has N input items, the method obtains N-1 differential vectors, and the differential vectors can be combined into a differential matrix to reflect the state change caused by the test sequence:

Cover Matrix＝{CoverVar₁，CoverVar₂，……，CoverVar_N-1}

the differential matrix is used as test feedback, so that the test can be effectively guided to cover different state change conditions in a TCP/IP protocol stack to be tested, and deep defects can be found.

Therefore, by monitoring the state change data, the test sequence can be adjusted accordingly, so that different state changes can be covered, and the test comprehensiveness is improved.

Specifically, referring to fig. 4, after obtaining the state change data, the test method provided in the embodiment of the present specification further includes:

step 401: comparing the state change data with state change data prestored in a seed pool;

step 403: and if the comparison results are different, putting the test sequence into the seed pool.

And if the comparison results are the same, the test sequence is discarded.

By comparison, a determination can be made as to whether the current test sequence has tested a new state change.

If the comparison result is different, which indicates that the current test sequence tests a new state change, the test sequence is put into the seed pool, which can be used to determine a new sequence, and it is possible to test more new state changes.

Otherwise, if the comparison result is the same, it indicates that the current test sequence and the previous test sequence are repeated, so that the test is meaningless and can be discarded.

In an embodiment of the present specification, after placing the test sequence in the seed pool, the method further includes:

step 405: carrying out variation on the test sequence to determine a new test sequence;

step 407: and executing the method shown in the figure 1 in the embodiment above by using the new test sequence, and executing a TCP/IP protocol stack test process.

The mutation of the test sequence includes adding a new data item to the test sequence retrieved from the seed pool or replacing the data item in the test sequence, which is not limited herein.

In this embodiment, when the state change data is represented by the above Matrix Cover Matrix, the Matrix Cover Matrix may be compared with the Matrix data stored in the data Pool Sta _ Pool, and if the Matrix Cover Matrix and the matrices have the same condition, the test sequence Seq may be discarded; otherwise, the test sequence Seq would be considered interesting and put into the seed pool.

Further, a test sequence is extracted from the seed pool for mutation, a New test sequence Seq New is determined, and the test scheme shown in FIG. 1 is executed.

In this embodiment of the present disclosure, the running process of the TCP/IP protocol stack to be tested is monitored, so as to obtain first monitoring data, where the first monitoring data may include one or more of a return value of system call, an output data packet of the TCP/IP protocol stack to be tested, state coverage data of the TCP/IP protocol stack to be tested, and state change data of the TCP/IP protocol stack to be tested, where the data reflect whether a function of the TCP/IP protocol stack to be tested is correct.

In this way, the first monitoring data may be used as a basis for determining the test result of the TCP/IP protocol stack under test.

In one embodiment, the first monitoring data may be directly determined to determine whether the first monitoring data meets the expectation. Specifically, it is determined whether a return value of the system call meets an expected value, and/or whether an output data packet is complete. And detecting the state of the TCP/IP protocol stack to be tested, which is not covered by the current testing process, according to the state covering data, and/or monitoring the state change, which is not covered by the current testing process, according to the state change data, and the like. And is not particularly limited herein.

The embodiment of the present specification further provides another testing method, and referring to fig. 5, the method includes:

step 502: and according to the dependency relationship among the data items contained in the test sequence, sequentially running the TCP/IP protocol stack to be tested by using the data items, and sequentially running the reference TCP/IP protocol stack by using the data items.

Step 504: monitoring the running process of the TCP/IP protocol stack to be tested to obtain first monitoring data, and monitoring the running process of the reference TCP/IP protocol stack to obtain second monitoring data;

step 506: and determining a test result of the TCP/IP protocol stack to be tested by using the first monitoring data and the second monitoring data.

Specifically, semantic comparison is carried out on the first monitoring data and the second monitoring data, and a test report of the TCP/IP protocol stack to be tested is generated according to a semantic comparison result.

And if the semantic comparison result shows that the first monitoring data is different from the second monitoring data, determining that the TCP/IP protocol stack to be tested possibly has defects. And if the semantic comparison result shows that the first monitoring data is the same as the second monitoring data, determining that the TCP/IP protocol stack to be tested possibly has no defects. Such information may be included in the test report.

The method uses a semantic defect detection mode based on comparison, not only can determine whether the TCP/IP protocol stack to be tested has defects, but also can generate a test report based on the semantic comparison result, and the test report can record whether the TCP/IP protocol stack to be tested has defects and the actually existing defects. If the semantic comparison result shows that the first monitoring data is different from the second monitoring data, namely the semantic difference exists, the error position of the TCP/IP protocol stack to be tested can be accurately determined, the user is prevented from compiling complex check rules, and therefore usability and universality of the testing method are improved.

In another embodiment of the present specification, a third-party defect detector may also be directly used to monitor the TCP/IP protocol stack to be tested, to obtain first monitoring data, and to determine whether the TCP/IP protocol stack to be tested has a defect by directly using the first monitoring data, to obtain a test result.

The third-party defect detector comprises ASan, MSan, TSan and the like, and detects defects automatically by monitoring memory access information of a tested program in the running process of a TCP/IP protocol stack to be tested. When a tested program accesses a memory, a third-party defect detector records and maintains the basic state information of the memory and stores the basic state information in a special shadow memory (shadow memory); these defect detectors automatically report defects if there is a problem with the basic state information of the memory, e.g. the memory has been released but used.

Specifically, the update and judgment logic for the current memory usage may be added before and after the code of the memory accessed by the program, and if the current memory access is judged to be incorrect, the defect may be reported.

Fig. 6 is a flowchart of a TCP/IP protocol stack testing method provided in an embodiment of the present specification, where the method specifically includes:

step 601: generating an initial input sequence, wherein the initial input sequence is used as a test sequence;

step 603: simultaneously operating a TCP/IP protocol stack to be tested and a reference TCP/IP protocol stack by utilizing an initial input sequence;

step 605: collecting operation information and output data of a TCP/IP protocol stack to be tested to form first monitoring data, wherein the operation information corresponds to the state coverage data mentioned above, and the output data can correspond to the return value of the system call mentioned above and an output data packet;

collecting operation information and output data of a reference TCP/IP protocol stack to form second monitoring data;

step 607: comparing the first monitoring data with the second monitoring data;

step 609: obtaining a state difference matrix according to the operation information, wherein the state difference matrix is a specific form of state change data;

step 611: judging whether the current test exceeds the upper limit of the test time;

if yes, ending;

if not, go to step 613: if the state difference matrix is different from the storage matrix in the data pool, generating a new input sequence by using the initial input matrix;

return to step 603.

Fig. 7 is a structural diagram of a TCP/IP protocol stack testing apparatus provided in an embodiment of the present specification, where the apparatus includes:

the running module 701 is configured to sequentially run a TCP/IP protocol stack to be tested by using each data item according to a dependency relationship between each data item included in the test sequence, where each data item includes one or both of an input data packet and a system call;

the monitoring module 702 is used for monitoring the running process of the TCP/IP protocol stack to be tested to obtain first monitoring data;

the determining module 703 determines a test result of the TCP/IP protocol stack to be tested by using the first monitoring data.

Optionally, according to a dependency relationship between data items included in the test sequence, the running module further sequentially runs a reference TCP/IP protocol stack by using the data items;

the monitoring module also monitors the running process of the reference TCP/IP protocol stack to obtain second monitoring data;

determining a test result of the TCP/IP protocol stack to be tested by using the first monitoring data, wherein the test result comprises the following steps:

and determining a test result of the TCP/IP protocol stack to be tested by using the first monitoring data and the second monitoring data.

Optionally, determining a test result of the TCP/IP protocol stack to be tested by using the first monitoring data and the second monitoring data includes:

performing semantic comparison on the first monitoring data and the second monitoring data;

and generating a test report of the TCP/IP protocol stack to be tested according to the semantic comparison result.

Optionally, if the TCP/IP protocol stack to be tested is run by using the system call, monitoring a running process of the TCP/IP protocol stack to be tested, including:

and determining first monitoring data according to the return value and the output data packet.

Optionally, monitoring the running process of the TCP/IP protocol stack to be tested includes:

and monitoring state coverage data in the TCP/IP protocol stack to be tested, wherein the first monitoring data comprises the state coverage data.

Optionally, monitoring the running process of the TCP/IP protocol stack to be tested further includes:

for two adjacent data items based on the dependency relationship, respectively acquiring the state coverage data respectively corresponding to the two data items;

and differentiating the state coverage data respectively obtained by the two adjacent data items to obtain state change data of the TCP/IP protocol stack to be tested, wherein the first monitoring data comprises the state change data.

Based on the same inventive concept, an embodiment of the present specification further provides an electronic device, including:

a processor; and

a memory configured to store a computer program that, when executed, causes the processor to:

according to the dependency relationship among all data items contained in the test sequence, sequentially operating a TCP/IP protocol stack to be tested by using all data items, wherein all data items comprise one or two of an input data packet and system call;

Based on the same inventive concept, there is also provided in the embodiments of this specification a computer-readable storage medium comprising a computer program for use with an electronic device, the computer program being executable by a processor to perform the steps of:

Fig. 8 is a schematic diagram illustrating a more specific hardware structure of a computing device according to an embodiment of the present disclosure, where the computing device may include: a processor 1010, a memory 1020, an input/output interface 1030, a communication interface 1040, and a bus 1050. Wherein the processor 1010, memory 1020, input/output interface 1030, and communication interface 1040 are communicatively coupled to each other within the device via bus 1050.

The processor 1010 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits, and is configured to execute related programs to implement the technical solutions provided in the embodiments of the present disclosure.

The Memory 1020 may be implemented in the form of a ROM (Read Only Memory), a RAM (Random access Memory), a static storage device, a dynamic storage device, or the like. The memory 1020 may store an operating system and other application programs, and when the technical solution provided by the embodiments of the present specification is implemented by software or firmware, the relevant program codes are stored in the memory 1020 and called to be executed by the processor 1010.

The input/output interface 1030 is used for connecting an input/output module to input and output information. The i/o module may be configured as a component in a device (not shown) or may be external to the device to provide a corresponding function. The input devices may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output devices may include a display, a speaker, a vibrator, an indicator light, etc.

The communication interface 1040 is used for connecting a communication module (not shown in the drawings) to implement communication interaction between the present apparatus and other apparatuses. The communication module can realize communication in a wired mode (such as USB, network cable and the like) and also can realize communication in a wireless mode (such as mobile network, WIFI, Bluetooth and the like).

Bus 1050 includes a path that transfers information between various components of the device, such as processor 1010, memory 1020, input/output interface 1030, and communication interface 1040.

It should be noted that although the above-mentioned device only shows the processor 1010, the memory 1020, the input/output interface 1030, the communication interface 1040 and the bus 1050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, those skilled in the art will appreciate that the above-described apparatus may also include only those components necessary to implement the embodiments of the present description, and not necessarily all of the components shown in the figures.

In the 90 s of the 20 th century, improvements in a technology could clearly distinguish between improvements in hardware (e.g., improvements in circuit structures such as diodes, transistors, switches, etc.) and improvements in software (improvements in process flow). However, as technology advances, many of today's process flow improvements have been seen as direct improvements in hardware circuit architecture. Designers almost always obtain the corresponding hardware circuit structure by programming an improved method flow into the hardware circuit. Thus, it cannot be said that an improvement in the process flow cannot be realized by hardware physical modules. For example, a Programmable Logic Device (PLD), such as a Field Programmable Gate Array (FPGA), is an integrated circuit whose Logic functions are determined by programming the Device by a user. A digital system is "integrated" on a PLD by the designer's own programming without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Furthermore, nowadays, instead of manually making an Integrated Circuit chip, such Programming is often implemented by "logic compiler" software, which is similar to a software compiler used in program development and writing, but the original code before compiling is also written by a specific Programming Language, which is called Hardware Description Language (HDL), and HDL is not only one but many, such as abel (advanced Boolean Expression Language), ahdl (alternate Hardware Description Language), traffic, pl (core universal Programming Language), HDCal (jhdware Description Language), lang, Lola, HDL, laspam, hardward Description Language (vhr Description Language), vhal (Hardware Description Language), and vhigh-Language, which are currently used in most common. It will also be apparent to those skilled in the art that hardware circuitry that implements the logical method flows can be readily obtained by merely slightly programming the method flows into an integrated circuit using the hardware description languages described above.

The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, an Application Specific Integrated Circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic for the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller as pure computer readable program code, the same functionality can be implemented by logically programming method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Such a controller may thus be considered a hardware component, and the means included therein for performing the various functions may also be considered as a structure within the hardware component. Or even means for performing the functions may be regarded as being both a software module for performing the method and a structure within a hardware component.

The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. One typical implementation device is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functionality of the units may be implemented in one or more software and/or hardware when implementing the present application.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A TCP/IP protocol stack testing method comprises the following steps:

2. The method of claim 1, further comprising:

according to the dependency relationship among all data items contained in the test sequence, sequentially operating a reference TCP/IP protocol stack by using all the data items;

monitoring the running process of the reference TCP/IP protocol stack to obtain second monitoring data;

3. The method of claim 2, determining a test result for the TCP/IP protocol stack under test using the first monitoring data and the second monitoring data, comprising:

4. The method of claim 3, determining a test result for the TCP/IP protocol stack to be tested according to the semantic comparison result, comprising:

and if the semantic comparison result shows that the first monitoring data is different from the second monitoring data, determining that the TCP/IP protocol stack to be tested has defects.

5. The method according to claim 1, wherein monitoring the running process of the TCP/IP protocol stack to be tested if the TCP/IP protocol stack to be tested is run by using the system call comprises:

6. The method of claim 1, monitoring the running process of the TCP/IP protocol stack to be tested, comprising:

7. The method of claim 6, monitoring the running process of the TCP/IP protocol stack under test, further comprising:

and differentiating the state coverage data respectively obtained by the two adjacent data items to obtain state change data of the TCP/IP protocol stack to be tested, wherein the first monitoring data also comprises the state change data.

8. The method of claim 7, further comprising:

comparing the state change data with state change data prestored in a seed pool;

and if the comparison results are different, putting the test sequence into the seed pool.

9. The method of claim 8, further comprising:

performing variation on the test sequence to determine a new test sequence;

and returning the dependency relationship among the data items contained in the test sequence by using the new test sequence, and sequentially operating the TCP/IP protocol stack to be tested by using the data items.

10. The method according to claim 1, before sequentially running the TCP/IP protocol stack to be tested using the data items according to the dependency relationship between the data items included in the test sequence, further comprising:

acquiring the dependency relationship among the data items;

and constructing the test sequence based on the dependency relationship.

11. A TCP/IP protocol stack testing device comprises:

12. The apparatus according to claim 11, wherein the running module further runs a reference TCP/IP protocol stack in sequence by using the data items according to the dependency relationship between the data items included in the test sequence;

13. The apparatus of claim 12, determining a test result for the TCP/IP protocol stack under test using the first monitoring data and the second monitoring data, comprising:

14. The apparatus according to claim 11, wherein if the TCP/IP protocol stack to be tested is run by using the system call, monitoring a running process of the TCP/IP protocol stack to be tested includes:

and determining the monitoring data according to the return value and the output data packet.

15. The apparatus of claim 11, monitoring the running process of the TCP/IP stack under test, comprising:

16. The apparatus of claim 15, monitoring the running process of the TCP/IP stack under test, further comprising:

17. An electronic device, comprising:

a processor; and