CN114860572A

CN114860572A - Method and device for generating test case and electronic equipment

Info

Publication number: CN114860572A
Application number: CN202210332836.6A
Authority: CN
Inventors: 刘岳萍; 梁一
Original assignee: Qax Technology Group Inc; Secworld Information Technology Beijing Co Ltd
Current assignee: Qax Technology Group Inc; Secworld Information Technology Beijing Co Ltd
Priority date: 2022-03-31
Filing date: 2022-03-31
Publication date: 2022-08-05

Abstract

The application relates to a method and a device for generating a test case and electronic equipment. The method for generating the test case comprises the following steps: determining at least one test flow model, a test data model and a model drive file corresponding to a protocol to be detected; traversing the spliced at least one test flow model to obtain at least one test path, wherein each test path has a corresponding test operation sequence; for each of at least part of the test paths, generating a test case based on the test operation sequence and the test data corresponding to the test path; the test data comprises data determined from the test data model based on the protocol to be tested and the model driver file, and the test data is used as a test parameter value of the test operation in the test operation sequence. The application can promote the test convenience on the basis of promoting the test accuracy.

Description

Method and device for generating test case and electronic equipment

Technical Field

The present application relates to the field of software testing technologies, and in particular, to a method and an apparatus for generating a test case, and an electronic device.

Background

The safety design and realization of the network protocol are not only related to the privacy and property safety of people, but also related to the interests of the country. Network protocol vulnerabilities have become a research hotspot in the field of information security.

The applicant finds the field of industrial control (industrial control for short) safety control, and relates to control and test of a plurality of industrial protocols. The test of industrial control protocol needs to cover dozens of protocols, each protocol has different characteristics, and the concerned test dimension in each test process is uncertain. Therefore, each time a protocol test is performed, each test dimension and the mutual influence between different test dimensions may need to be sufficiently tested, and the process needs to consume a large amount of labor cost and time cost to supplement the successive regression summary of each protocol so as to obtain an applicable test case.

Disclosure of Invention

In order to at least partially solve the problems in the related art, the application provides a method, a device and electronic equipment for generating a test case, which can effectively improve the convenience of obtaining the test case and the coverage rate of the test case.

A first aspect of the present application provides a method for generating a test case, including: determining at least one test flow model, a test data model and a model drive file corresponding to a protocol to be detected; traversing the spliced at least one test flow model to obtain at least one test path, wherein each test path has a corresponding test operation sequence; for each of at least part of the test paths, generating a test case based on the test operation sequence and the test data corresponding to the test path; the test data comprises data determined from the test data model based on the protocol to be tested and the model driver file, and the test data is used as a test parameter value of the test operation in the test operation sequence.

A second aspect of the present application provides an apparatus for generating a test case, including: the device comprises a model determining module, a model traversing module and a test case generating module. The model determining module is used for determining at least one test process model, a test data model and a model driving file corresponding to the protocol to be detected; the model traversing module is used for traversing the spliced at least one test flow model to obtain at least one test path, and each test path has a corresponding test operation sequence; the test case generation module is used for generating a test case for each of at least part of the test paths based on the test operation sequence and the test data corresponding to the test paths; the test data comprises data determined from the test data model based on the protocol to be tested and the model driver file, and the test data is used as a test parameter value of the test operation in the test operation sequence.

A third aspect of the present application provides an electronic device comprising: a processor; a memory having executable code stored thereon, which when executed by the processor, causes the processor to perform the above method.

The fourth aspect of the present application also provides a computer-readable storage medium having stored thereon executable code, which, when executed by a processor of an electronic device, causes the processor to perform the above-mentioned method.

A fifth aspect of the present application also provides a computer program product comprising executable code which, when executed by a processor, implements the above method.

The method, the device and the electronic equipment for generating the test cases can pre-construct the test flow models, the test data models and the like aiming at different test scenes, the test flow models, the test data models and the like can be protocols which are verified for many times and can be reused, when the test protocols are tested, the test flow models, the test data models and the like can be directly called for testing, the mutual influence among different test dimensions and different test dimensions is comprehensively considered, in addition, a large amount of labor cost and time cost are not required to be consumed, and the test effect is effectively improved.

In addition, in some embodiments, the test data may be a field from the protocol content of the industrial control protocol, which facilitates determining which boundary conditions may need to be tested, so as to improve coverage of the boundary conditions by the test process, and may cope with a test scenario in which the test traffic data is insufficient.

In addition, in some embodiments, the test data may adjust the test flow model according to a target test parameter selected by a user, and only test operations related to the target test parameter, thereby effectively reducing redundancy of the test parameter, improving resource utilization efficiency, and improving test efficiency.

In addition, in some embodiments, a plurality of test paths can be determined in a traversal manner, and a test operation sequence of at least part of the test paths is determined, so that the test flexibility is improved, and various user test requirements are met.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

The present application, as well as other objects, features, and advantages thereof, will become more apparent from the following detailed description of exemplary embodiments when taken in conjunction with the accompanying drawings, wherein like reference numerals generally represent like parts throughout the exemplary embodiments of the present application.

FIG. 1 schematically illustrates an exemplary system architecture to which the method, apparatus, and electronic device for generating test cases may be applied, according to an embodiment of the present application;

FIG. 2 is a flow diagram schematically illustrating a method of generating test cases according to an embodiment of the present application;

FIG. 3 schematically illustrates a diagram of an industrial control network common protocol according to an embodiment of the present application;

FIG. 4 schematically shows a structural schematic diagram of a test flow model according to an embodiment of the present application;

FIG. 5 schematically illustrates a process diagram of a detection protocol according to an embodiment of the application;

FIG. 6 schematically illustrates a structural schematic of a stitched test flow model according to an embodiment of the present application;

FIG. 7 schematically illustrates a structural diagram of a test flow model before tuning according to an embodiment of the present application;

FIG. 8 schematically illustrates a diagram of an adjusted test flow model according to an embodiment of the present application;

FIG. 9 is a flow chart of a method for testing based on test cases according to an embodiment of the present application;

FIG. 10 is a block diagram schematically illustrating an apparatus for generating test cases according to an embodiment of the present application;

FIG. 11 schematically shows a block diagram of an electronic device according to an embodiment of the present application.

Detailed Description

Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. While embodiments of the present application are illustrated in the accompanying drawings, it should be understood that the present application may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The terms "comprises," "comprising," and the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.

All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It is noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification and should not be interpreted in an idealized or overly formal sense.

It should be understood that although the terms "first," "second," "third," etc. may be used herein to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present application, "a plurality" means two or more unless specifically limited otherwise.

In order to facilitate understanding of the technical solutions of the present application, some terms are first described.

An industrial control system is a combination of an Ethernet network and a control network which are urged to be brought forward to the requirements of large data volume and high-rate transmission such as images and voice signals. The networking wave of the industrial control system integrates various current popular technologies such as multi-standard industrial control network interconnection, wireless technology and the like, and the development space of the industrial control field is expanded.

A network protocol is a set of communication rules between network nodes, and specifies the formats that information must take in communication and the meanings of those formats. Network nodes include, but are not limited to: at least one of a network server, a switch and a router.

Finite state machines, mathematical models representing a finite number of states and the behavior of transitions and actions between these states.

In the related art, dozens of protocols need to be covered when protocol testing is carried out, and in order to improve the testing efficiency, the configuration and the function of each industrial protocol of industrial control can be tested by using an automatic script testing method. However, each protocol has different characteristics and the test dimensions are uncertain. Therefore, each dimension and the interaction between different dimensions need to be fully tested, which results in the need to expend a lot of labor cost and time cost to supplement the successive regression summary of each protocol. In addition, based on the application scenario of the industrial protocol, the obtained test flow data is limited, and a deeper test needs to be performed on the scenario, for example, a test for a boundary condition is easily omitted.

After the applicant summarizes the related art, the related art is found to have at least the following defects when the protocol test is performed on the related art.

For example, as industrial control protocols increase, corresponding scripts accumulate more quickly, resulting in increased maintenance costs.

For example, the message structure in the actual network environment is various, and the test depth and the test width are limited by the experience of the testers, so that the protocol test has uncertainty. There are various bus networks for fieldbus systems, which are more influential in the control domain. Although the industry has established a unified standard (FF). But due to commercial profit, technical monopoly and the like, the fieldbus product is still in a situation of hundreds of flowers, so that the requirement on the experience of testers is higher.

For example, different protocol characteristics differ, and the test methods are difficult to multiplex.

For example, test data (e.g., test traffic messages) are limited and cover fewer test scenarios.

In the embodiment of the application, a test flow model, a test data model and the like are pre-constructed for different types of protocols, and the test flow model, the test data model and the like can be a model which is verified for many times and can be reused. When some protocols need to be tested, the test flow model and the test data model corresponding to the protocols can be directly called, different test dimensions and mutual influences among the different test dimensions are comprehensively considered, a large amount of labor cost and time cost are not needed to be consumed to determine the mutual influences, and the test accuracy and convenience are effectively improved.

The technical solution of the present application is exemplarily described below with reference to fig. 1 to 11.

Fig. 1 schematically illustrates an exemplary system architecture to which the method, apparatus and electronic device for generating test cases may be applied according to an embodiment of the present application. It should be noted that fig. 1 is only an example of a scenario in which the embodiment of the present application may be applied to help those skilled in the art understand the technical content of the present application, and does not mean that the embodiment of the present application may not be applied to other devices, systems, environments or scenarios.

Referring to fig. 1, an industrial control network 1 is taken as an example to illustrate, a factory has a plurality of

devices

10, 20, 30, 40, 50, etc., and the

devices

10, 20, 30, 40, 50 can be produced according to preset process parameters, etc. according to scheduling information. In addition, multiple sensors may be present in the plant to monitor environmental information, such as temperature information via temperature sensor 60 and humidity information via humidity sensor 70. The

devices

10, 20, 30, 40, 50, temperature sensor 60, humidity sensor 70, etc. may be connected to a control terminal 80 via a network 90. The information fed back by the

devices

10, 20, 30, 40, 50, etc. includes production-related production factor information. For example, the device status, the temperature information, the humidity information, etc. may be stored in the control terminal 80 and may be updated in real time, so that the control terminal 80 schedules the product to be processed based on the production factors acquired in real time. Further, the control terminal 80 may be connected to the server 100 through the network 90 to receive or transmit information, such as production factor information and the like. It should be noted that the network 90 may include various types of connections, such as wire, wireless communication links, or fiber optic cables, to name a few.

For example, the server 100 or the control terminal 80 or the like generates a test case in response to the test request, and can detect a specific protocol based on the test case.

It should be understood that the number of devices, networks, control terminals and servers are merely illustrative. There may be any number of edge terminals, networks, and servers, as desired for implementation.

FIG. 2 is a flow chart schematically illustrating a method for generating test cases according to an embodiment of the present application.

Referring to fig. 2, the embodiment provides a method for generating a test case, where the method includes operations S210 to S230, which are described in detail below.

In operation S210, at least one test flow model, test data model, and model driver file corresponding to a protocol to be tested are determined.

In this embodiment, the protocol to be detected may be designated by the user, or the system may automatically screen the protocol to be detected that needs to be detected. The protocol to be detected can be various communication protocols such as an industrial control protocol.

Fig. 3 schematically shows a schematic diagram of an industrial control network common protocol according to an embodiment of the present application. Referring to fig. 3, the protocol to be detected may be a conventional control network protocol, a fieldbus protocol, an industrial ethernet protocol, an industrial wireless network protocol, etc.

Conventional control network protocols include, but are not limited to: a centralized digital Control System (CCS), a Distributed Control System (DCS), a Fieldbus Control System (FCS), a reference link, and the like.

Fieldbus protocols include, but are not limited to: a Controller Area Network (CAN), a device Network (DeviceNet), a Control and Communication Link (CC-Link), a program BUS Network (PROFIBUS), and the like.

Industrial ethernet protocols include, but are not limited to: Ethernet/Industrial Protocol (Ethernet/Industrial Protocol, abbreviated Ethernet/IP), Ethernet for Control Automation Technology (Ethernet for Control Automation Technology, abbreviated EtherCAT), High Speed Ethernet (HSE), Industrial Ethernet (Profinet), Ethernet for Plant Automation (EPA), power link (POWERLINK), open field bus (Modbus), Neighbor Discovery Protocol 3(Neighbor Discovery Protocol 3, abbreviated NDP3), object link and embedded Process Control (OLE for Process Control, abbreviated OPC), Tcnet developed by toshiba, wwet, etc.

Industrial wireless network protocols include, but are not limited to: IEEE 802 Standards established by local area network/metropolitan area network Standards Committee (LAN/MAN Standards Committee), R field bus (RFieldbus), ZigBee (ZigBee), and the like.

The protocol development trend of control networks is gradually towards open, transparent communication protocols. Ethernet has advantages in terms of high transmission speed, low power consumption, ease of installation, and good compatibility, and is widely used in commercial systems because it supports almost all popular network protocols. With the development of network technology, ethernet has entered the control field, forming a control network technology based on ethernet. This is due, at least in part, to the trend toward distributed, intelligent control of industrial automation systems, with open, transparent communication protocols being a possible trend. The field buses are not compatible with each other due to various types, and the requirement cannot be met. However, the TCP/IP protocol of ethernet cannot replace the fieldbus protocol in a short time, and both protocols coexist for a long time, which further increases the kind of protocols to be tested and the difficulty of testing.

In this embodiment, a test flow model, a test data model, and the like for different types of protocols may be abstracted from historical test cases based on experience, rules, and the like. The test flow models and the test data models can be reused, and the accuracy and the coverage rate of each model and each file can be ensured. When a certain type of protocol needs to be tested, a user inputs the protocol to be tested, and the server can automatically select a test flow model and a test data model corresponding to the protocol to be tested. The user can also manually select a test flow model and a test data model corresponding to the protocol to be tested. The model driver file may also be a previously edited file, and the user may also edit the driver model file according to the user.

Specifically, determining at least one test flow model, test data model, and model driver file corresponding to the protocol to be tested may include the following operations: and determining at least one test flow model and test data model corresponding to the protocol to be detected based on the mapping relation in the model driving file. The protocol to be detected comprises test parameters, the mapping relation comprises corresponding relations among the test parameters, a test flow model and a test data model, the test flow model comprises at least one node and at least one edge, the node represents the state machine model, the edge represents the test operation, and jump among different state machine models by executing the test operation.

For a network protocol comprising multiple states, it is a protocol that is context information, data information and historical track related. The communication process of a network protocol including a plurality of states is very complicated. For example, the processes of handshaking, authentication, and the like may be included, and when the related art performs protocol testing, the adopted test data may only cover the first interaction state, and it is difficult to cover the subsequent state trace. Therefore, a network protocol including multiple states needs to consider the state characteristics of a stateful network protocol.

The state machine model is suitable for describing the state transition characteristics of the network protocol, and is a formal description that may be employed. The state machine model is generally represented by a directed graph, vertices represent states, and directed edges represent operations by which state transitions can be implemented. The directed edge is marked with inputs and outputs, which are conditions for state transition. The state machine model not only has strong intuition, but also is convenient for automatic realization. For example, the state machine model may be represented as a tuple. The state machine model may be a finite state machine model.

For example, a plurality of states may be included in the state machine model as follows: read coil state, coil state in state 1, coil state in state 2, etc. The test operation may be to receive a message 1 and the corresponding state transition is a state transition from state 1 to state 2 of the coil.

In operation S220, the spliced at least one test flow model is traversed to obtain at least one test path, and each test path has a corresponding test operation sequence.

In this embodiment, since a plurality of test flow models and/or test data models may be involved in the process of testing one protocol, in the process of generating a test case, a splicing operation needs to be performed on the plurality of test flow models to obtain a complete test model, so as to perform traversal.

For example, the test flow model may include a plurality of state machine models and test operations, and the jump between the plurality of state machines may be implemented by different test operations, which facilitates determining whether the test result is correct based on the state.

Fig. 4 schematically shows a structural diagram of a test flow model according to an embodiment of the present application.

Referring to fig. 4, the test flow model may include a state machine model 1 to a state machine model 7, which may be illustrated as nodes, and a connection relationship may exist between a plurality of nodes. The connection relation can represent test operation, and jump among different state machines is realized through the test operation. Wherein at least some of the test flow models each include an inlet and an outlet for facilitating connectivity between different test flow models. In certain embodiments, at least a portion of the test flow models each include an inlet and an outlet.

Referring to fig. 4, the test flow model may include three test paths.

For example, test path 1: the state machine model 1-the state machine model 2-the state machine model 6-the state machine model 7. Accordingly, the sequence of test operations may include: test operation 0-test operation 1-test operation 2-test operation 6-test operation 7. Wherein each test operation may be the same or different.

For example, test path 2: the method comprises the following steps of 1-state machine model 2-state machine model 3-state machine model 4-state machine model 6-state machine model 7. Accordingly, the test operation sequence may include: test operation 0-test operation 1-test operation 2-test operation 3-test operation 4-test operation 6-test operation 7. Wherein each test operation may be the same or different.

For example, test path 3: the state machine model 1-the state machine model 2-the state machine model 3-the state machine model 4-the state machine model 5-the state machine model 6-the state machine model 7. Accordingly, the test operation sequence may include: test operation 0-test operation 1-test operation 2-test operation 3-test operation 4-test operation 5-test operation 6-test operation 7. Wherein each test operation may be the same or different.

It should be appreciated that when multiple test flow models are stitched, more test paths may result.

In operation S230, for each of at least some of the test paths, a test case is generated based on the test operation sequence and the test data corresponding to the test path.

The test data comprises data determined from the test data model based on the protocol to be tested and the model driver file, and the test data is used as a test parameter value of the test operation in the test operation sequence.

The test operation may include at least one parameter for which the test data may be assigned a value.

In some embodiments, generating a test case based on the test operation sequence and the test data corresponding to the test path may include the following operations: and for each test operation in the test operation sequence, assigning the test data to the test parameters corresponding to the test operation to generate executable test operation.

The model driver file serves as a parameter of model instantiation to drive the operation of the whole model. For the industrial control protocol, a test object and a test range of each industrial control protocol are defined in the model driver file. In addition, the model driver file may also specify the validity of the test data.

Fig. 5 schematically shows a process diagram of a detection protocol according to an embodiment of the application.

Referring to FIG. 5, dashed boxes represent test model-related data, which may be data called from a pre-developed library. Such as a database including test flow models and a database including test data models may be developed. In addition, a database including driver files may also be developed. When a user needs to test a certain type of protocol, the user can select at least one of a corresponding test flow model, a corresponding test data model and a corresponding model driver file from the library so as to generate a test case and test based on the test case. The test process model may be obtained by process modeling. The test data model may be constructed based on historical test data. It should be noted that, the user may configure the model driver file according to the needs of the user, for example, at least one of adding, deleting, or modifying the setting information.

For example, protocol-dependent constraints may be provided in the model driver file, as shown in conditions 1 through 4.

Condition 1, log format of protocol records.

And 2, the number of the characteristic parameters to be tested by the protocol.

And 3, the protocol to-be-tested characteristic parameter value range.

And 4, protocol testing of the mapping relation of the characteristic parameters.

The above constraints are merely exemplary and are not to be construed as limiting the present application.

For example, in this embodiment, a model-based test method may be used to model all test flows of the industrial control protocol, and construct a test data model and a test flow model. Then, the test data model and the test flow model can be instantiated by taking the model driving file as input, and a test case is generated. This facilitates the execution of test cases and the detection of protocols based on the results of the examination.

In some embodiments, the method may further include the operations of: and constructing a test data model. At least part of test data in the test data model aiming at the industrial control protocol comes from the field of the protocol content of the industrial control protocol. The fields selected from the protocol content are used as the test number, so that the embodiment can be suitable for coverage test in the scene with limited industrial flow.

Specifically, the test data model may include at least one of a testable field of the protocol, a value range corresponding to the testable field, a condition matching the message feature, and a constraint condition.

For example, the test data of the industrial control protocol can be derived from each field of the protocol content, and for the traffic with a certain characteristic, whether the industrial control device (such as an industrial control firewall) can correctly recognize the behavior of the traffic can be determined, and the corresponding action can be taken.

In some embodiments, building the test data model may include the following operations. Firstly, test sample data is obtained, wherein the test sample data comprises test parameter values. And then, performing reverse matching processing on at least part of dimensions of the test parameter values and the combination of the parameter values to obtain a plurality of test data items. Next, the plurality of test data items are used as test data models.

Specifically, the actual test environment does not cover all scenarios, and a reverse matching test method can be adopted to reduce the redundancy of the test quantity. For example, modeling based on message description is performed on a known industrial protocol traffic message, and All dimensions and combinations of values of test variables of a certain protocol are screened by using a pairing algorithm (such as pair/All-Pairs Testing) or an Orthogonal Array Testing Strategy (OATS for short) and the like, so that All values and combinations of All dimensions are prevented from being exhaustively tested to obtain effective test data, and various possibilities of an analysis engine are subjected to traversal test.

For example, the test parameters include categories, groups of devices, and product test results. Wherein the categories include category 1 and category 2. The device groups include group 1 and group 2. The product test results include pass and fail. Through permutation and combination, 8 sets of test data can be obtained, as shown in table 1.

TABLE 1

Serial number	Categories	Equipment group	Product test results
				1	Class 1	Group 1	Qualified
2	Class 2	Group 1	Qualified
				3	Class 1	Group 2	Qualified
4	Class 2	Group 2	Qualified
				5	Class 1	Group 1	Fail to be qualified
6	Class 2	Group 1	Fail to be qualified
				7	Class 1	Group 2	Fail to be qualified
8	Class 2	Group 2	Fail to be qualified

The reverse matching process may be as follows.

First, the analysis was started from sequence number 8. The number 8 is a combination of "category 2, group 2, and fail", and the combinations of two are "category 2 group 2", "group 2 fail", and "category 2 fail". Checking whether these three combinations appear in the serial numbers 1-7, it can be seen that "category 2 group 2" appears in number 4, "group 2 fails" appears in number 7, and "category 2 fails" appears in number 6. Therefore, number 8 can be left out according to the paired test method concept. At this time, the remaining test data are shown in table 2.

TABLE 2

Subsequently, the serial number 7 was analyzed. Two-by-two combination of serial number 7 "category 1 group 2" appeared in number 3, "category 1 failed" appeared in serial number 5, but "group 2 failed" was only one, so serial number 7 needs to be retained.

Subsequently, the serial number 6 was analyzed. The same analysis can show that the combination of serial number 6, category 2, group 1, appears at number 2, group 1 fails at number 5, category 2 fails at only one, and thus number 6 is reserved.

Subsequently, the serial number 5 was analyzed. The same analysis can show that the combination of 5, category 1, group 1, appeared at 1, group 1 failed appeared at 6, and category 1 failed appeared at 7, so the case 6 can be discarded, and the rest of the test data is shown in table 3.

TABLE 3

Serial number	Categories	Equipment group	Product test results
				1	Class 1	Group 1	Qualified
2	Class 2	Group 1	Qualified
				3	Class 1	Group 2	Qualified
4	Class 2	Group 2	Qualified
				6	Class 2	Group 1	Fail to be qualified
7	Class 1	Group 2	Fail to be qualified

Next, serial No. 4, serial No. 3, serial No. 2, and serial No. 1 were sequentially analyzed in the above manner, and the final retention test data was obtained as shown in table 4.

TABLE 4

It can be seen that the test data after the reverse matching processing is half of the original test data, the test data is effectively reduced, and the test coverage rate is better. It should be noted that the above reverse matching process is only an exemplary process, and is not to be construed as limiting the application.

Testable fields and value ranges of the protocol, conditions matched with message characteristics and some conventional constraints are specified in the test data model, so that the test data is more targeted.

For example, the test data may be part of the test data obtained by pariwise algorithm, the test data covering sets of test data for forward and reverse tests. The test data may be as shown in table 5.

TABLE 5

arg1	arg2	arg3	match
				001 Read Coil (reading Coil state)	-1	0	N
Any	-1	100	Y
				015 Write Multiple Coils	0	65535	N
004 Read Input Registers	0	0	N
				024 Read FIFO Queue	65535	65535	N
003 Read Holding Registers	1	100	N
				006 Write Single Register	65535	65535	N
Any	0	0	N
				016 Write Multiple Registers (prefabricated Multiple Registers)	0	0	N
006 Write Single Register	1	100	N
				002 Read disturb Inputs (Read input status)	-1	65535	N
023 Read Write Register	-1	0	N
				001 Read Coil (reading Coil state)	1	65535	Y
005 Write Single Coil (forced Single Coil)	-1	-1	N
				022 Mask Write Register	1	65535	N

In some embodiments, the method may further include: and (5) constructing a test process model.

Control testing of an industrial protocol may include two parts, one part being configuring policies associated with the industrial protocol (e.g., actions related to the characteristics or control of the protocol). The other part is the actual control check of the flow. In the previous embodiment, test data is obtained, and the process of constructing the test flow model is to obtain test operations in a modeling mode, so that a complete test case is combined.

The modeling of the test step is to analyze the function of the module first, and then analyze the relationship between the test step and the test step, and the relationship between the test step and the state machine model. And (4) performing test design according to the state of the tested system and the previously set constraint conditions, and generating and executing a test case. In addition, a test strategy may also be adopted when generating the test case (see fig. 7 and the description of relevant part of fig. 8).

Specifically, building the test flow model may include the following operations.

First, a node corresponding to a state machine model for performing state transition in accordance with a preset state is drawn, and a correlation node of the node is determined. Then, the nodes and the associated nodes are connected to obtain edges. And then, establishing a mapping relation between the edges and the test operation to obtain a test flow model.

For example, a state machine model and a connection relation are drawn by using a drawing tool (such as graphwalk), each operation step of the test is described, and all operation steps which can generate connection are connected to form a complete model which covers the whole process as far as possible.

In one particular embodiment, nodes and edges are drawn using graphwalk. Wherein each edge represents a transition operation and each node represents a state. It should be noted that, in the multi-model combination, an exit and an entrance are provided in each model, so as to facilitate switching between the models.

Fig. 6 schematically shows a structural schematic diagram of a spliced test flow model according to an embodiment of the present application.

Referring to fig. 6, test flow model 1, test flow model 2, and test flow model 3 are shown. The test flow model 1 comprises state machine models 1-7, the test flow model 2 comprises state machine models 8-10, and the test flow model 3 comprises state machine models 11-13. The state machine models 1-7 may be the same or different from the state machine models 8-10.

In some embodiments, the test cases may be generated automatically or manually.

Specifically, traversing the stitched at least one test flow model may include at least one of the following.

For example, in response to a received operation instruction, at least one test flow model is spliced, and then the spliced at least one test flow model is traversed, wherein the operation instruction is an instruction for the test flow model.

For example, according to the dependency relationship between the respective functions of at least one test flow model, the at least one test flow model is spliced, and then the spliced at least one test flow model is traversed.

A simple process for generating test cases is depicted in FIG. 6. The configuration is firstly carried out, the model is loaded in a manual or automatic mode after being generated, and a sequence of testing steps is generated. The finite-state machine model adopted by the example in the figure is used for generating the test cases according to the characteristics of the state machine and a path traversal algorithm.

The traversal process may take any one or more of the following forms.

A Finite State Machine (FSM for short), which automatically generates a use case principle: different algorithms and strategies are incorporated to traverse the model.

The model enumerates all transitions and states according to a large number of different paths, automatically generating a series of test cases.

The main efficacy of the model-based testing technology is embodied in the algorithm of path traversal. And (3) traversing algorithm: random traversal, weighted traversal, covering all transitions, and so forth.

In some embodiments, stitching the at least one test flow model according to the dependency relationship between the respective functions of the at least one test flow model may include: and connecting the inlet of the test process model with the outlet of the depended test process model according to the dependency relationship between the functions of at least one test process model.

For example, traversing the spliced at least one test flow model can include the following operations.

Firstly, taking a node corresponding to an inlet of a first test flow model as an initial node, starting from an unaccessed connected node of the initial node in sequence, and traversing all nodes with depth first until accessing a node corresponding to an outlet of a last test flow model.

Then, the following operations are repeated until all nodes of all test flow models are accessed: and if the nodes which are not accessed exist, selecting one node from the nodes which are not accessed as an initial node, starting from the connected nodes which are not accessed of the initial node in sequence, and traversing the nodes which are not accessed in a depth-first mode until accessing the node corresponding to the outlet of the last test flow model.

For example, referring to fig. 6, a test path may include: the state machine model 1-the state machine model 2-the state machine model 7-the state machine model 8-the state machine model 10-the state machine model 11-the state machine model 12-the state machine model 13. The other test paths are not shown one by one.

An industrial control protocol can support different operations on different functions and components of a plurality of industrial devices. In this embodiment, the requirements of various test scenarios can be met by splicing a plurality of test flow models.

In some embodiments, the test flow model and/or the test data model may be adjusted for the test parameters of interest to the user to improve test efficiency.

Specifically, the above method may further include the following operations. Firstly, in response to the target test parameters determined from the test parameters, the test flow model is adjusted to obtain a sub-test flow model, and target test data corresponding to the target test parameters is selected from the test data model. The sub-test flow model is then used to replace the test flow model. Wherein the target test parameters may be user specified parameters. Since there is a correspondence between the test parameters and the test data, the required test data can be determined based on the parameters specified by the user.

For example, adjusting the test flow model to obtain the sub-test flow model may include the following operations. First, a state machine associated with a target test parameter is determined. And then, deleting non-associated state machines and non-associated test operations except the associated state machines from the test flow model to obtain a sub-test flow model.

Fig. 7 schematically shows a structural diagram of a test flow model before adjustment according to an embodiment of the present application. FIG. 8 schematically shows a diagram of an adapted test flow model according to an embodiment of the application.

Referring to fig. 7, a test flow model includes a state machine model 1 to a state machine model 12, where the state machine model 1 is connected to a state machine model 2, a state machine model 6, a state machine model 7, a state machine model 9, a state machine model 10, a state machine model 11, and the state machine model 12, respectively. Currently, a test parameter a needs to be tested, and a state machine model associated with the test parameter a includes: the state machine model 6, the state machine model 7, the state machine model 9, the state machine model 10, the state machine model 11 and the state machine model 12 may be simplified into the test flow model shown in fig. 8.

The model design mode has the characteristic of dynamic self-adaptation, nodes or processes of the model can be deleted and modified according to different protocol types or different test requirements in the model drive file, and a model suitable for the current test requirements is automatically generated

FIG. 9 is a flowchart schematically illustrating a method for performing testing based on test cases according to an embodiment of the present application. The test sample data may further include identification information of a correct test result.

Referring to fig. 9, the method may further include operations S940 and S950 after performing operation S230.

In operation S940, the protocol to be detected is tested based on the test case, and a test result for the test case is obtained.

In operation S950, the correct test result corresponding to the correct test result identification information is compared with the test result.

For example, the test may be performed based on the generated plurality of test cases, and the test procedure may be as follows.

For example, the user → the industrial protocol a is selected → the industrial protocol feature configuration (matching) → industrial control flow test → inspection of the control result.

For example, the user → select industrial protocol B → industrial protocol feature configuration (mismatch) → industrial control flow test → inspection of the control result.

For example, the user → the industrial protocol B is selected → the industrial protocol feature configuration (matching) → control action → industrial control flow test → inspection of the control result.

In this embodiment, a test data model is obtained based on the description modeling of the industrial protocol flow, and test data is obtained based on the test data model. And generating a test flow model based on modeling of the test flow of the industrial control equipment, and obtaining a test step sequence based on the test flow model so as to generate an executable test case. The test case generated in the mode has the characteristics of dynamic generation, traceability and no need of maintaining a solid-state case. In addition, the user can self-adaptively generate a specific model through self-defined data driving.

Another aspect of the present application further provides an apparatus for generating a test case.

FIG. 10 is a block diagram schematically illustrating an apparatus for generating test cases according to an embodiment of the present application.

Referring to fig. 10, the apparatus 1000 for generating a test case may include: a model determination module 1010, a model traversal module 1020, and a test case generation module 1030.

The model determining module 1010 is configured to determine at least one test flow model, a test data model, and a model driver file corresponding to a protocol to be tested.

The model traversing module 1020 is configured to traverse the spliced at least one test flow model to obtain at least one test path, where each test path has a corresponding test operation sequence.

The test case generating module 1030 is configured to generate, for each of at least some of the test paths, a test case based on a test operation sequence corresponding to the test path and test data, where the test data includes data determined from a test data model based on a protocol to be tested and a model driver file, and the test data is used as a test parameter value of a test operation in the test operation sequence.

In some embodiments, the model determining module is specifically configured to determine, based on a mapping relationship in the model driver file, at least one test flow model and a test data model corresponding to a protocol to be tested, where the protocol to be tested includes test parameters, the mapping relationship includes a corresponding relationship between the test parameters, the test flow model, and the test data model, the test flow model includes at least one node and at least one edge, the node represents the state machine model, the edge represents a test operation, and a jump is performed between different state machine models by performing the test operation.

In some embodiments, the apparatus 1000 may further include: and the test data model building module is used for building a test data model, wherein at least part of test data in the test data model aiming at the industrial control protocol comes from a field of protocol content of the industrial control protocol.

In some embodiments, the test data model includes at least one of a testable field of the protocol, a value range corresponding to the testable field, a condition matching the message feature, and a constraint.

In some embodiments, the test data model building module comprises: the device comprises a data acquisition unit, an inverse matching unit and a model replacement unit.

The data acquisition unit is used for acquiring test sample data, and the test sample data comprises test parameter values;

the reverse matching unit is used for performing reverse matching processing on at least part of dimensionalities of the test parameter values and the combination of the parameter values to obtain a plurality of test data items;

the model replacement unit is used for taking a plurality of test data items as test data models.

In some embodiments, the test sample data further includes correct test result identification information. Correspondingly, the apparatus 1000 may further include a testing module and an alignment module.

The test module is used for testing the protocol to be detected based on the test case to obtain a test result aiming at the test case;

the comparison module is used for comparing the correct test result and the test result corresponding to the correct test result identification information.

In some embodiments, the apparatus 1000 may further include: the device comprises a data and parameter determining module and a model adjusting module.

The data and parameter determining module is used for responding to the target test parameters determined from the test parameters, adjusting the test flow model to obtain a sub-test flow model, and selecting target test data corresponding to the target test parameters from the test data model.

The model adjusting module is used for replacing the testing process model by the sub-testing process model.

In some embodiments, the data and parameter determining module is specifically configured to determine a state machine associated with the target test parameter, and delete non-associated state machines and non-associated test operations other than the associated state machine from the test flow model to obtain the sub-test flow model.

In some embodiments, the apparatus 1000 may further include: and a test flow model building module.

The test flow model building module comprises a node drawing unit, an edge drawing unit and a mapping building unit.

The node drawing unit is used for drawing nodes corresponding to the state machine model and determining the associated nodes of the nodes, and the state machine model is used for carrying out state transition according to a preset state.

The edge drawing unit is used for connecting the nodes and the associated nodes to obtain edges.

And the establishing mapping unit is used for establishing a mapping relation between the edges and the test operation to obtain a test flow model.

In some embodiments, the model traversal module includes a first traversal unit and a second traversal unit.

The first traversal unit is used for responding to the received operation instruction, splicing at least one test flow model, and traversing the spliced at least one test flow model, wherein the operation instruction is an instruction aiming at the test flow model.

The second traversal unit is used for splicing the at least one test flow model according to the dependency relationship among the functions of the at least one test flow model, and then traversing the spliced at least one test flow model.

In certain embodiments, at least a portion of the test flow models each include an inlet and an outlet. Correspondingly, the second traversal unit is specifically configured to connect the entry of the test flow model and the exit of the depended test flow model according to the dependency relationship between the respective functions of at least one test flow model, so as to obtain the test model.

In some embodiments, the second traversal unit comprises: traversal sub-unit and loop sub-unit.

And the traversal subunit is used for taking the node corresponding to the inlet of the first test flow model as an initial node, starting from the non-accessed connected nodes of the initial node in sequence, and traversing all the nodes with depth priority until accessing the node corresponding to the outlet of the last test flow model.

A loop subunit for repeating the following operations until all nodes of all test flow models have been accessed: and if the nodes which are not accessed exist, selecting one node from the nodes which are not accessed as an initial node, starting from the connected nodes which are not accessed of the initial node in sequence, and traversing the nodes which are not accessed in a depth-first mode until accessing the node corresponding to the outlet of the last test flow model.

In some embodiments, the test case generating module 1030 is specifically configured to assign, to each test operation in the test operation sequence, the test data to the test parameter corresponding to the test operation, and generate an executable test operation.

According to the device for generating the test cases, the test flow model and/or the test data model are high in self-adaptability and flexible in change, fixed cases are not used, and the device is more convenient to maintain. In addition, the coverage test range is wider, and hidden problems and design defects are easier to find. In addition, the method is suitable for coverage test in the scene with limited industrial flow.

With regard to the apparatus 1000 in the above embodiment, the specific manner in which each module, unit, and sub-unit performs operations has been described in detail in the embodiment related to the method, and will not be described in detail herein.

Another aspect of the present application also provides an electronic device.

Referring to fig. 11, an electronic device 1100 includes a memory 1110 and a processor 1120.

The Processor 1120 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 1110 may include various types of storage units such as system memory, Read Only Memory (ROM), and permanent storage. The ROM may store, among other things, static data or instructions for the processor 1120 or other modules of the computer. The persistent storage device may be a read-write storage device. The persistent storage may be a non-volatile storage device that does not lose stored instructions and data even after the computer is powered off. In some embodiments, the persistent storage device employs a mass storage device (e.g., magnetic or optical disk, flash memory) as the persistent storage device. In other embodiments, the permanent storage may be a removable storage device (e.g., floppy disk, optical drive). The system memory may be a read-write memory device or a volatile read-write memory device, such as a dynamic random access memory. The system memory may store instructions and data that some or all of the processors require at runtime. In addition, the memory 1110 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (e.g., DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), magnetic and/or optical disks, as well. In some embodiments, memory 1110 may include a removable storage device that is readable and/or writable, such as a Compact Disc (CD), a digital versatile disc read only (e.g., DVD-ROM, dual layer DVD-ROM), a Blu-ray disc read only, an ultra-dense disc, a flash memory card (e.g., SD card, min SD card, Micro-SD card, etc.), a magnetic floppy disk, or the like. Computer-readable storage media do not contain carrier waves or transitory electronic signals transmitted by wireless or wired means.

The memory 1110 has stored thereon executable code that, when processed by the processor 1120, may cause the processor 1120 to perform some or all of the methods described above.

Furthermore, the method according to the present application may also be implemented as a computer program or computer program product comprising computer program code instructions for performing some or all of the steps of the above-described method of the present application.

Alternatively, the present application may also be embodied as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) having executable code (or a computer program or computer instruction code) stored thereon, which, when executed by a processor of an electronic device (or server, etc.), causes the processor to perform part or all of the various steps of the above-described method according to the present application.

Having described embodiments of the present application, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for generating test cases, comprising:

determining at least one test flow model, a test data model and a model drive file corresponding to a protocol to be detected;

traversing at least one spliced test flow model to obtain at least one test path, wherein each test path has a corresponding test operation sequence;

for each of at least part of the test paths, generating a test case based on the test operation sequence and the test data corresponding to the test path;

2. The method of claim 1, wherein determining at least one test flow model, test data model, and model driver file corresponding to the protocol to be tested comprises:

determining at least one test flow model and a test data model corresponding to the protocol to be tested based on a mapping relation in the model driver file, wherein the protocol to be tested comprises test parameters, the mapping relation comprises the test parameters, the test flow model and the test data model, the test flow model comprises at least one node and at least one edge, the node represents a state machine model, and the edge represents a test operation and jumps among different state machine models by executing the test operation.

3. The method of claim 2, further comprising: and constructing a test data model, wherein at least part of test data in the test data model aiming at the industrial control protocol comes from the field of the protocol content of the industrial control protocol.

4. The method of claim 3, wherein the test data model comprises at least one of a testable field of a protocol, a value range corresponding to the testable field, a condition matching a message characteristic, and a constraint condition.

5. The method of claim 3, wherein constructing the test data model comprises:

obtaining test sample data, wherein the test sample data comprises test parameter values;

performing reverse matching processing on at least part of dimensions of the test parameter values and the combination of the parameter values to obtain a plurality of test data items;

and taking a plurality of test data items as the test data model.

6. The method of claim 5, wherein the test sample data further includes correct test result identification information;

the method further comprises the following steps:

testing a protocol to be detected based on the test case to obtain a test result aiming at the test case;

and comparing the correct test result corresponding to the correct test result identification information with the test result.

7. The method of claim 2, further comprising:

responding to target test parameters determined from the test parameters, adjusting the test flow model to obtain a sub-test flow model, and selecting target test data corresponding to the target test parameters from the test data model;

and replacing the test process model with the sub-test process model.

8. The method of claim 7, wherein said adjusting said test flow model to obtain a sub-test flow model comprises:

determining a state machine associated with the target test parameter;

and deleting non-associated state machines and non-associated test operations except the associated state machines from the test process model to obtain the sub-test process model.

9. The method of claim 2, further comprising: constructing a test process model, comprising:

drawing nodes corresponding to a state machine model, and determining associated nodes of the nodes, wherein the state machine model is used for carrying out state transition according to a preset state;

connecting the node and the associated node to obtain an edge;

and establishing a mapping relation between the edges and the test operation to obtain the test flow model.

10. The method of claim 9, wherein said traversing the stitched at least one of the test flow models comprises:

splicing at least one test flow model in response to a received operation instruction, and traversing the spliced at least one test flow model, wherein the operation instruction is an instruction for the test flow model; or

And splicing at least one test process model according to the dependency relationship among the functions of at least one test process model, and traversing at least one spliced test process model.

11. The method of claim 10, wherein at least some of the test flow models each include an inlet and an outlet;

the splicing at least one test process model according to the dependency relationship between the respective functions of at least one test process model comprises:

and connecting the inlet of the test process model with the outlet of the depended test process model according to the dependency relationship between the functions of at least one test process model.

12. The method of claim 11, wherein traversing the spliced at least one test flow model comprises:

taking a node corresponding to the inlet of the first test flow model as an initial node, starting from the non-accessed connected nodes of the initial node in sequence, and traversing all nodes with depth first until accessing a node corresponding to the outlet of the last test flow model;

repeating the following operations until all nodes of all test flow models have been accessed: and if the nodes which are not accessed exist, selecting one node from the nodes which are not accessed as an initial node, starting from the connected nodes which are not accessed of the initial node in sequence, and traversing the nodes which are not accessed in a depth-first mode until accessing the node corresponding to the outlet of the last test flow model.

13. The method according to any one of claims 1 to 12, wherein generating a test case based on the test operation sequence and the test data corresponding to the test path comprises:

and for each test operation in the test operation sequence, assigning the test data to the test parameter corresponding to the test operation to generate an executable test operation.

14. An apparatus for generating test cases, comprising:

the model determining module is used for determining at least one test flow model, a test data model and a model driving file corresponding to the protocol to be detected;

the model traversing module is used for traversing at least one spliced test flow model to obtain at least one test path, and each test path has a corresponding test operation sequence;

the test case generation module is used for generating a test case based on the test operation sequence and the test data corresponding to at least part of the test paths; the test data comprises data determined from the test data model based on the protocol to be tested and the model driver file, and the test data is used as a test parameter value of the test operation in the test operation sequence.

15. An electronic device, comprising:

a processor; and

a memory having executable code stored thereon, which when executed by the processor, causes the processor to perform the method of any of claims 1-13.

16. A computer-readable storage medium having stored thereon executable code, which when executed by a processor of an electronic device, causes the processor to perform the method of any one of claims 1-13.