CN116419275A

CN116419275A - Test method, test device, test equipment and storage medium

Info

Publication number: CN116419275A
Application number: CN202111672831.XA
Authority: CN
Inventors: 谭雨夕
Original assignee: China Mobile Communications Group Co Ltd; China Mobile Communications Ltd Research Institute
Current assignee: China Mobile Communications Group Co Ltd; China Mobile Communications Ltd Research Institute
Priority date: 2021-12-31
Filing date: 2021-12-31
Publication date: 2023-07-11

Abstract

The invention discloses a testing method, a testing device, testing equipment and a storage medium. Wherein the method comprises the following steps: acquiring M test scenes and total test cost of a Network Slice Management Function (NSMF) slice service; and performs the following operations: determining a test scene with highest priority based on M test scenes of NSMF slicing service; and under the condition that the total test cost is greater than or equal to the first test cost of the test scene with the highest priority, testing the determined test scene with the highest priority. In the invention, NSMF slice service links and link priorities are abstracted into NSMF slice service flow graphs, node, edge and edge weights in the service flow graphs are defined, and the problem of maximizing scene priority is converted into the longest path problem for solving the NSMF slice service flow graphs.

Description

Test method, test device, test equipment and storage medium

Technical Field

The present invention relates to the field of wireless technologies, and in particular, to a testing method, apparatus, device, and storage medium.

Background

At present, the scene test scheme of the multi-service branch of the fifth generation (5G,fitth Generation) network slice arranging system is deficient, with the increase of 5G slice service requirements, as the on-demand arranging characteristics of the 5G slice service can be customized according to the requirements of slice service types, slice capability requirements, slice sharing grades and the like, for the network slice management function (NSMF, network Slice Management Function) system, the characteristics of numerous combined scene branches, long logic links, numerous interface parameters and the like complicate NSMF test work, the consideration of engineering implementation is given, the manual traversing of a large number of huge scene branches is impractical under the premise of limited test cost budget, and the writing and maintenance work of the scene test cases have a large number of repeated operations and the test efficiency is low.

Disclosure of Invention

In view of this, the embodiments of the present invention desire to provide a testing method, apparatus, device and storage medium.

The technical scheme of the embodiment of the invention is realized as follows:

at least one embodiment of the present invention provides a test method applied to NSMF, the method comprising:

obtaining M test scenes and total test cost of NSMF slicing service; m is a positive integer greater than 1;

And performs the following operations:

determining a test scene with highest priority based on M test scenes of NSMF slicing service; determining a first test cost of a test scene with the highest priority;

under the condition that the total test cost is greater than or equal to the first test cost, testing the test scene with the highest determined priority;

the total test cost and the first test cost are subjected to difference to obtain residual test cost; updating the total test cost into the residual test cost to obtain updated total test cost;

based on M test scenes of NSMF slicing service, determining the test scene with the highest priority again;

and the like until the updated total test cost is less than the second test cost of the newly determined highest-priority test scenario.

Furthermore, according to at least one embodiment of the present invention, the determining the test scenario with the highest priority based on the M test scenarios of the NSMF slice service includes:

generating a slice business link flow diagram; the slicing business link flow graph comprises M paths; each path is composed of nodes and edges; vectors between nodes represent links between the nodes; the value of the vector represents the priority of the inter-link; one path corresponds to one test scene; one node corresponds to one service link in the test scene;

When the problem of determining the test scene with the highest priority is converted into the problem of determining the shortest path in the slice service link flow graph, performing inverse operation on the value of the vector between the two corresponding nodes according to any two nodes in the slice service link flow graph to obtain a vector value after the inverse operation; and determining other nodes passing between the corresponding two nodes;

constructing an adjacency matrix by using vector values after performing inverting operation between any two nodes; constructing a path matrix by utilizing other nodes passing between any two nodes;

performing relaxation operation by using the adjacency matrix and the path matrix to obtain a shortest path;

and taking the test scene corresponding to the shortest path as the test scene with the highest priority.

Furthermore, according to at least one embodiment of the present invention, the performing a relaxation operation using the adjacency matrix and the path matrix, to obtain a shortest path, includes:

determining a value of a vector between an ith node and a jth node using the adjacency matrix;

determining a shortest path between the ith node and the jth node by using the value of the vector between the ith node and the jth node and combining the path matrix;

Wherein the value range of i is 1 to N; j has a value ranging from 1 to N.

Further, according to at least one embodiment of the present invention, the determining the shortest path between the i-th node and the j-th node using the values of the vectors between the i-th node and the j-th node in combination with the path matrix includes:

determining at least one intermediate node passing between the ith node and the jth node by using the path matrix;

for each intermediate node, performing a relaxation operation on an edge between the jth node and the corresponding intermediate node to determine whether the ith node, the jth node and the corresponding intermediate node meet a preset condition by using values of vectors between the ith node and the jth node;

and under the condition that the ith node, the jth node and the corresponding intermediate nodes meet the preset conditions, obtaining the shortest path between the ith node and the jth node.

Furthermore, in accordance with at least one embodiment of the present invention, the method further comprises:

determining M paths; each path is composed of nodes and edges; vectors between nodes represent links between the nodes; the value of the vector represents the priority of the inter-link; one path corresponds to one test scene; one node corresponds to one service link in the test scene;

Creating, for each path, a plurality of nodes in the respective path; creating an edge between two adjacent nodes in the corresponding path; the weight of the edge is the priority between two business links corresponding to two adjacent nodes;

and obtaining a slice business link flow graph based on the created nodes and edges.

Furthermore, according to at least one embodiment of the present invention, the testing the determined testing scenario with the highest priority includes:

determining test scripts of a plurality of business links contained in a test scene with highest priority;

cascading the test scripts of the determined multiple service links to obtain a test script corresponding to the test scene with the highest priority;

and testing the determined test scene with the highest priority by using the test script corresponding to the test scene with the highest priority.

At least one embodiment of the present invention provides a test apparatus comprising:

the acquisition unit is used for acquiring M test scenes and total test cost of NSMF slicing service; m is a positive integer greater than 1;

a processing unit for performing the following operations:

At least one embodiment of the present invention provides a network device comprising:

the communication interface is used for acquiring M test scenes and total test cost of NSMF slicing service; m is a positive integer greater than 1;

a processor for performing the operations of:

At least one embodiment of the invention provides a network device comprising a processor and a memory for storing a computer program capable of running on the processor,

the processor is configured to execute the steps of any method on the network device side when running the computer program.

At least one embodiment of the present invention provides a storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of any of the methods described above.

The testing method, the testing device, the testing equipment and the storage medium provided by the embodiment of the invention acquire M testing scenes and total testing cost of NSMF slicing service; m is a positive integer greater than 1; and performs the following operations: determining a test scene with highest priority based on M test scenes of NSMF slicing service; determining a first test cost of a test scene with the highest priority; under the condition that the total test cost is greater than or equal to the first test cost, testing the test scene with the highest determined priority; the total test cost and the first test cost are subjected to difference to obtain residual test cost; updating the total test cost into the residual test cost to obtain updated total test cost; based on M test scenes of NSMF slicing service, determining the test scene with the highest priority again; and the like until the updated total test cost is less than the second test cost of the newly determined highest-priority test scenario. By adopting the technical scheme provided by the embodiment of the invention, the priority of different test scenes is considered, so that the automatic test of the test scene with high priority is preferentially ensured on the premise of limited test cost.

Drawings

FIG. 1 is a diagram of a two-level business orchestration system architecture according to the related art;

FIG. 2 is a schematic diagram of a flow chart of an implementation of a test method according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a generated slice business link flow diagram in accordance with an embodiment of the present invention;

FIG. 4 is a schematic flow chart of a specific implementation of the test method according to the embodiment of the present invention;

FIG. 5 is a schematic diagram of an initialization process of an NSMF slicing service provisioning procedure in an embodiment of the present invention;

FIG. 6 is a schematic diagram of an implementation flow of setting priority of each service link according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of the structure of a testing device according to an embodiment of the present invention;

fig. 8 is a schematic diagram of a composition structure of a network device according to an embodiment of the present invention.

Detailed Description

Prior to introducing the technical solution of the embodiment of the present invention, a description will be given of related technology.

In the related art, with the continuous development and progress of communication technology and mobile internet technology, in order to meet the differentiated network resource requirements of different user groups, a 5G network slicing technology has been developed. The essence of the 5G network slicing technology is to divide a basic network into mutually independent and isolated logic networks according to the service demands of users, and network resources configured by slicing are changed along with the change of the user demands so as to flexibly meet the multi-user differentiated service demands. A 5G network slice is a special purpose logical network that provides specific network capabilities, and users and service providers create service level agreements (SLAs, service Level Agreement) to specify network capabilities required to meet users.

To facilitate management of 5G network slices, NSMF was introduced. NSMF aims to manage the full life cycle of network slice instances, supporting the management of slice life cycles according to traffic scenarios, including: creating, modifying and terminating network slice instances according to network slice templates (NST, network Slice Template), creating and generating network slice instance identifications; the attribute requirements of the network slice instance, the service parameters and the model defined in NST and the slicing subnet templates of each professional domain are decomposed into the slicing subnet attribute requirements and the slicing subnet templates of the three professional domains of the wireless, transmission and core network, and the sub-slice management function (NSSMF, network Slice Subnet Management Function) system of the three subdomains is called to complete the creation of the slicing subnet instances of the three professional domains.

Fig. 1 is a schematic diagram of a two-level service orchestration system according to the related art, as shown in fig. 1, in order to make network orchestration capability more efficient and flexible, operators typically layer NSMF according to network resource nanotube scope, such as a national primary service orchestration center and a provincial secondary service orchestration center, where the two-level service orchestration centers cooperate to complete network slice lifecycle management of each domain. Taking a secondary service arrangement center as an example, a slicing order is derived from a primary service arrangement system, an provincial enterprise opening system, a service designer and a third party application system, and after the order is received, the slicing order cooperates with a network sub-slicing manager of each professional domain, wherein the sub-slicing service opening of each professional domain is completed by the systems including NSSMF of a wireless network, NSSMF of a core network, operation and maintenance centers (OMC, operation and Maintenance Center) of transmission, a resource-saving pipe and the like.

In the related art, the wireless network NSSMF and the core network NSSMF relate to multiple manufacturers and multiple areas, and the peripheral system is complex. In addition, due to the on-demand arrangement characteristic of the 5G slice service, the slice service types such as the excellent sharing, the exclusive sharing, the honored sharing service, the slice capacity demands such as Embb, urllc, miot, v x, the slice sharing grade and the like can be customized according to the requirement, and for an NSMF system, the NSMF test work is complex and time-consuming due to the characteristics of numerous branches of a combined scene, longer logic links, numerous interface parameters and the like. And as the slicing service parameters are increased, the branching of the 5G network slicing service arrangement scene grows exponentially, so that on the premise of limited testing cost budget, how to better ensure the software quality of the 5G network slicing arrangement system becomes a problem to be discussed urgently.

In the related art, a testing scheme of a 5G network slice mainly discusses performance and quality indexes of the 5G network slice, and specifically includes:

firstly, a 5G slice testing method is proposed in a patent with publication number CN113259983A, and testing parameters associated with network slice service are configured to a terminal through an interface; and in the process of implementing the network slicing service by the terminal, executing at least two test flows on the terminal to obtain a test result of the test index of each test flow, wherein the test result is used for testing the test index of the terminal under the concurrent scene of the multi-network slicing service and generating the test result aiming at the terminal.

Secondly, a scheme for realizing 5G transmission sub-slice on-demand selection and guarantee based on a dynamic scheduling algorithm is provided in a patent with publication number CN113453260A, and data of network elements, links, ports, tunnels, segmented routing tunnels, network slices and network element alarms in a network are periodically collected; carrying out data association on the acquired data, matching all routes interconnected among network elements, automatically generating a full-network topology diagram of the slice packet network, and calculating logic configuration information on the network elements, ports and links according to the topology diagram; and verifying each performance index of the 5G slice service by a simulation test of the configuration instruction before the service is opened.

Third, a network slice quality test scheme is described in the patent publication number KR1020200046415a, by providing a 5G slice link through simulation on a virtual network, collecting quality information of a 5G network slice path through test traffic generated by testing a virtual machine, and verifying each traffic index of the 5G network slice based on the collected quality information.

In summary, the patent with publication number CN113259983a and the patent with publication number KR1020200046415A mainly discuss a 5G slice network link testing scheme, the latter discuss a 5G slice network quality testing scheme, the former discusses a method for testing a terminal in a multi-network slice service concurrency scenario, and both the two patents focus on verifying whether the terminal can meet network resource and service configuration requirements through 5G slice simulation link level testing, but do not relate to a scenario testing scheme discussion of multi-service branches in a network slice orchestration system. The patent with publication number CN113453260a mainly describes a scheme of 5G transmission sub-slice on-demand and guarantee, the main discussion is transmission network NSSMF, and NSMF is not involved.

Based on the above analysis, it can be seen that the scenario test scheme of the multi-service branch of the 5G network slice arrangement system is deficient, along with the increase of the requirement of the 5G slice service, because of the on-demand arrangement characteristic of the 5G slice service, slice service types such as excellent sharing, exclusive sharing, honored sharing service, slice capability requirement such as Embb, urllc, miot, v x, slice sharing grade and the like can be customized as required, so for the NSMF system, the characteristics of numerous combined scenario branches, longer logic links, numerous interface parameters and the like make the NSMF test work complex, from the aspect of engineering implementation, under the premise of limited test cost budget, the manual traversing of a large number of scenario branches is impractical, and the writing and maintenance work of the scenario test case has a large number of repeated operations, so the test efficiency is lower, the scenario priority guarantee with higher priority is considered from the service angle, and the scenario test automation becomes necessary from the aspect.

Based on the above, in the embodiment of the invention, M test scenes and total test cost of NSMF slicing service are obtained; m is a positive integer greater than 1; and performs the following operations: determining a test scene with highest priority based on M test scenes of NSMF slicing service; determining a first test cost of a test scene with the highest priority; under the condition that the total test cost is greater than or equal to the first test cost, testing the test scene with the highest determined priority; the total test cost and the first test cost are subjected to difference to obtain residual test cost; updating the total test cost into the residual test cost to obtain updated total test cost; based on M test scenes of NSMF slicing service, determining the test scene with the highest priority again; and the like until the updated total test cost is less than the second test cost of the newly determined highest-priority test scenario.

It should be noted that, in the embodiment of the present invention, the test cost of each link of the NSMF slicing opening procedure is measurable, such as manpower cost, time cost, and the like. The NSMF slicing opening flow comprises a plurality of test scenes, each test scene is formed by combining one or more business links, and the test scenes are combinations of test cases of a plurality of links of the test data of a given scene. The automatic script of each link is used as the flow step cascade combination of the test scene to form the full-automatic script of the test scene. The service priority of each scene of the NSMF slicing opening flow can be measured, and the priority of each test scene can be decomposed into the sum of the priorities of each service link constituting the scene.

Fig. 2 is a schematic flow chart of an implementation of the test method of the embodiment of the present invention, applied to NSMF, as shown in fig. 2, and the method includes steps 201 to 203:

step 201: obtaining M test scenes and total test cost of NSMF slicing service; m is a positive integer greater than 1.

It will be appreciated that NSMF was introduced in order to facilitate management of 5G network slices. NSMF aims to manage the full life cycle of network slice instances, supporting the management of slice life cycles according to test scenarios.

It can be appreciated that after the initialization of the NSMF slice service opening flow, a plurality of test scenarios may be determined for the opened NSMF slice service, where each test scenario may be composed of one service link; alternatively, each test scenario may be formed by combining multiple business links, i.e., the test scenario may be formed by combining test cases of multiple links of test data for a given test scenario.

It will be appreciated that with the growing demand for 5G slice services, the types of services of the slices may include shared, exclusive, honored, etc. due to the on-demand orchestration nature of 5G slice services.

It will be appreciated that the total test cost may be measured, for example, human costs, time costs, etc. may be measured to obtain an estimate, which may be taken as the total test cost.

Step 202: the following operations are performed: determining a test scene with highest priority based on M test scenes of NSMF slicing service; determining a first test cost of a test scene with the highest priority; and under the condition that the total test cost is greater than or equal to the first test cost, testing the test scene with the highest determined priority.

It can be appreciated that, in order to achieve the trade-off between the test cost and the scene priority, so as to cover the test work of the important scenes as much as possible through the limited test cost, the test scene with the highest priority can be determined based on the M test scenes of the NSMF slice service, and the test scene with the highest priority is tested when the total test cost is greater than or equal to the first test cost.

In practical application, the NSMF slicing opening flow can be abstracted into a slicing service link flow graph, so that the problem of selecting the test scene with the highest priority from M test scenes of NSMF slicing service can be converted into the problem of solving the optimal path in the slicing service link flow graph, that is, the optimal path determined from the slicing service link flow graph corresponds to the test scene with the highest priority one by one.

Based on this, in an embodiment, the determining the test scenario with the highest priority based on the M test scenarios of the NSMF slice service includes:

In practical application, the fact that the priority of the test scenes of each NSMF slicing service is differentiated on the premise of limited test cost is considered, so that NSMF slicing service flows can be abstracted into slicing service link flow graphs, and priority parameters related to service links contained in the test scenes are set in the slicing service link flow graphs.

Based on this, in an embodiment, the method further comprises:

It can be appreciated that in the embodiment of the present invention, the priority is defined as a positive integer.

It can be appreciated that the priority between two service links may be obtained by differencing the priorities of the two service links.

It will be appreciated that each traffic link may be a branching link or, alternatively, may be a primary link.

It can be understood that the service link irrelevant to the slicing service parameters is the main link; the main links are not affected by slicing service parameters, and thus, the priority of the main links can be manually set or set to a default value.

It can be understood that the service links related to the slice service parameters are branch links; the priority of the branch links can be determined according to the slicing service requirement, and the higher the priority is, the higher the importance of the service links is.

Further, considering that the slice service parameters are mainly related to slice service requirements, the slice service parameters include parameters of slice service type, slice capability requirements, slice sharing, and the like, so that the priority definition for the slice service requirements can be mapped to a priority setting of a slice service parameter combination. The combination of the values of the slice service parameters directly determines the number of test scenes of the NSMF slice service.

Fig. 3 is a schematic diagram of a generated slice service link flow diagram, as shown in fig. 3, a slice service link flow diagram may be generated according to M test scenarios of NSMF slice service, where each test scenario includes multiple service links and priorities between the service links.

As shown in formula (1), G represents a slice service link flow graph, which is specifically as follows:

G＝(M,P) (1)

as shown in formula (2), M represents a service link set, and is specifically as follows:

M＝(m ₁ ,m ₂ ,...,m _n )；1≤n≤N (2)

wherein m is _i (1.ltoreq.i.ltoreq.N) represents a link with a sequence number i. N represents the total number of sliced traffic links.

As shown in formula (3), R represents a set of edges between two nodes corresponding to any two links, and is specifically as follows:

R＝{r _ij }，|r _ij |＝P _ij ；1≤i,j≤N,P _ij ＞0 (3)

wherein r is _ij (1.ltoreq.i, j.ltoreq.N) represents the element m _i To link m _j Is a directed path of (a); p (P) _ij Defined as the slave link m _i To link m _j Is a priority of (3); n represents the total number of slicing service links; definition |r _ij |＝P _ij I.e. the weight of an edge is the priority between two links.

As shown in formula (4), the set S records the test cost of the directional path between the service links, specifically as follows:

S＝{S _ij }，1≤i,j≤N，|S _ij |＞0 (4)

wherein S is _ij Representing link m _i To link m _j The test cost of the directed path of (a); n represents the total number of slicing service links. S is only calculated in an auxiliary way, so that S is not embodied in the slicing business link flow graph G.

It can be appreciated that the process of setting the priority of the internode links in the slice service link flow graph may specifically include:

first, initializing each link in a slicing service link flow graph G, and traversing a link m _i ∈M i＝1,2，...,N。

For m _j E M j =1, 2,..n, j+.i, if P _ij > 0, description link m _i To link m _j With links between, if P _ij =0, description of link m _i To link m _j There is no link between them.

Second, drawing link m _i To link m _j Is the directed vector r of (2) _ij At the same time, set link m _i To link m _j Edge r of (2) _ij Weight of (2) is P _ij 。

In practical application, the classical dynamic programming algorithm, namely the Floyd algorithm (also called as an insertion point method), is considered to find the shortest path between multiple source points in the directed weighted graph by utilizing the dynamic programming idea, and the Floyd algorithm allows the edge weight value in the directed weighted graph to be negative, so that the edge weight value in the slice service link flow graph generated by the embodiment of the invention can be subjected to the inverse operation, and the problem of solving the longest path in the slice service link flow graph can be converted into the problem of solving the shortest path between the multiple source points.

Based on this, in an embodiment, the performing a relaxation operation using the adjacency matrix and the path matrix, to obtain a shortest path includes:

wherein, the value range of i is 1 to N, and the value range of j is 1 to N.

It will be appreciated that N represents the total number of sliced traffic links.

Here, the determining, by using the values of the vectors between the i node and the j node and in combination with the path matrix, the shortest path between the i node and the j node includes:

It will be appreciated that summing the value of the vector from the i-th node to the corresponding intermediate node with the value of the vector from the corresponding intermediate node to the j-th node, results in a summed value; and if the value of the vector between the ith node and the jth node is larger than the summed value, the ith node, the jth node and the corresponding intermediate nodes meet the preset condition.

It will be appreciated that the process of determining the shortest path between the i-th node to the j-th node may specifically include:

Step S1: initializing an adjacency matrix to execute the inverse operation on the weight values of all the edges in the cut business link flow graph.

As shown in formula (5), the inverse operation is executed on the weights of all the edges in the slice service link flow graph, and the specific steps are as follows:

R＝{r _ij }，|r _ij |＝-P _ij ；1≤i,j≤N,P _ij ＞0 (5)

wherein R represents the set of all edges in the slice business link flow graph, and R _ij (1.ltoreq.i, j.ltoreq.N) represents the element m _i To link m _j Is a directed path of (a); p (P) _ij Defined as the slave link m _i To link m _j Is a priority of (3); n represents the total number of traffic links.

It will be appreciated that the goal of this step is to translate the longest path problem between solution links into a shortest path problem between solution links.

Step S2: at least one intermediate node passing between the ith node and the jth node is determined using the path matrix.

D as shown in formula (6) ^(k) Representing the adjacency matrix used to perform the relaxation operation in the kth round. And the adjacency matrix is used for recording the weight of the edge between any two nodes in the slice business link flow diagram and is also the priority between two business links corresponding to any two nodes.

Wherein matrix element d _ij Is the weight of the edge from node i to node j.

As shown in formula (7), l ^(k) Representing a path matrix. And the path matrix is used for recording the nodes through which the shortest path between any two nodes must pass.

Wherein, the liquid crystal display device comprises a liquid crystal display device,

indicating the nodes traversed by node i through node j when the relax operation is performed in the kth round.

Step S3: and performing a relaxation operation by using the initialized adjacency matrix and the path matrix.

When k=0, i.e., during the first round of iterative selection of the test scenario with the highest priority, the priority d from node i to node j in the adjacency matrix is initialized as shown in formulas (8) and (9) _ij Priority |r for node i to node j not passing through other nodes _ij | a. The invention relates to a method for producing a fibre-reinforced plastic composite. Initializing l in the path matrix due to no intermediate nodes _ij ＝-1。

And in the process of iteratively selecting the test scene with the highest priority in the kth round of loop, performing relaxation operation on the nth edge in the R set, so that the shortest distance estimated value among nodes gradually approaches the shortest distance.

Specifically, for link m _i E M, traversing node k of the path between nodes i and j, checking r _ij ＞r _ik +r _kj Whether or not to establish; if so, the path from node j to node j through node k is proved to be shortest.

It should be noted that, if there are a plurality of nodes k in the path between the nodes i and j, each node k is traversed circularly to obtain the shortest path from the node i to the node j.

As shown in formula (10), at the kth cycle, a plurality of intermediate nodes k of the paths between the nodes i and j are set, and the shortest path is searched, and for the kth cycle, if the intermediate nodes k cause:

d _ij ^(k) ＞d _ik ^(k) +d _kj ^(k) 1≤i,j≤N；k＝1,2，...，N (10)

Wherein d _ij Weights for the edges of node i to node j; d, d _ik Weights for the edges of node i to node k; d, d _kj Is the weight of the edge from node i to node j.

If the intermediate node k satisfies equation (10), then update equations (11), (12) and (13):

d _ij ^(k) ＝d _ik ^(k) +d _kj ^(k) 1≤i,j≤N (11)

wherein d _ij ^(k) ＝d _ik ^(k) +d _kj ^(k) Indicating that the weight of the edge from node i to node j is equal to the sum of the weight of the edge from node i to node k and the weight of the edge from node i to node j when the relaxation operation is performed in the kth round.

l _ij ^(k) ＝k 1≤i,j≤N (12)

Wherein l _ij ^(k) =k means that the node traversed by node i to node j is node k when the relax operation is performed in the kth round. That is, the shortest path between node i and node j is the path from node i to node k to node j.

s _ij ^(k) ＝s _ik ^(k) +s _kj ^(k) 1≤i,j≤N (13)

Step S4: the shortest path between node i and node j is output.

It will be appreciated that during each iteration of the loop, a relaxation operation may be performed to complete the shortest path selection using the Floyd algorithm in combination with the adjacency matrix and the path matrix.

Because NSMF slicing service does not have a circulating link, namely the slicing service link flow diagram does not have a negative weight loop, a shortest path exists for every two nodes in the slicing service link flow diagram.

As shown in the formulas (14) and (15), the shortest path L from the node i to the node j is as follows:

L _ij ∈L 1≤i,j≤N (14)

L _ij ＝{l _ij ^(k) }l _ij ^(k) ≠-1,1≤i,j≤N (15)

Wherein L is _ij To slice the shortest path from node i to node j in the traffic segment flow graph,i.e. the test link corresponding to the highest priority test scenario selected in the kth round.

Step S5: determining the test cost of the test scene with highest priority corresponding to the shortest path between the nodes j and j, namely L _ij The corresponding test cost is S _ij ^(N) 。

Step S6: and comparing the test cost of the test scene with the highest priority selected by the kth round with the total test cost.

Assume that the test cost of the test scenario with highest priority of the kth round of selection is s ^(k) The total test cost is S. If s ^(k) ＜S，

Then the test of the test scenario of the present round is performed and s=s-S is updated ^(k) . If s ^(k) And if the number is more than S, stopping the test of the test scene of the round.

In practical application, in order to realize the test of the test scene with the highest priority, the automatic scripts of each link in the shortest path can be combined according to the determined shortest path, so that the automatic script of the test scene with the highest priority to be tested currently can be obtained.

Based on this, in an embodiment, the testing the determined testing scenario with the highest priority includes:

It can be understood that, because of the repetition of the component links of the test scenarios of the multiple NSMF slicing services, the efficiency of the automatic test scripts of the independent maintenance links is higher, and thus, the test script of the scenario with the highest priority can be a collection of the automatic scripts of each link.

Step 203: the total test cost and the first test cost are subjected to difference to obtain residual test cost; updating the total test cost into the residual test cost to obtain updated total test cost; based on M test scenes of NSMF slicing service, determining the test scene with the highest priority again; and the like until the updated total test cost is less than the second test cost of the newly determined highest-priority test scenario.

In the process of redefining the test scene with the highest priority, determining two new nodes by combining slice service link flow diagrams generated by M test scenes based on NSMF slice service, and calculating the shortest distance between the two new nodes by using a Flod algorithm to obtain the test scene with the highest priority in the cycle. And if the residual test cost is enough to complete the test of the test scene of the round, testing the test scene with the highest priority selected in the round of circulation.

In the embodiment of the invention, the test scene with the highest priority is tested by combining the total test cost, and the method has the following advantages:

(1) In the NSMF test process, the test links are more in branches, the test scene is huge, and in the agile iteration test process, all scenes are completely covered and the reality condition is not met. Considering that the priorities of different test scenes are different, and the regression test range in the software iteration process can be updated according to the research and development change range, so that the automatic test of the key high-priority scene is preferentially ensured on the premise of limited test cost.

(2) After the test scene with the highest optimal test priority is obtained, the automatic test scripts of the test scene with the highest current priority are obtained by cascading and combining the automatic test scripts of each link of the test scene, and the test scene with the highest priority is tested by utilizing the obtained automatic test scripts. And then, the optimal path selection strategy is circularly executed until the total test cost is exhausted, so that the test efficiency is improved, the balance between the test cost and the scene priority is realized, and the test work of the key scene is covered as much as possible with the limited test cost.

Fig. 4 is a schematic flow chart of a specific implementation of the testing method according to the embodiment of the present invention, as shown in fig. 4, the method includes steps 401 to 408:

step 401: obtaining M test scenes and total test cost of NSMF slicing service; m is a positive integer greater than 1.

FIG. 5 is a schematic diagram of an initialization process of an NSMF slice service activation procedure, as shown in FIG. 5, after the NSMF slice service activation procedure is initialized, M test scenarios may be determined for the activated NSMF slice service, each test scenario may be composed of one service link; alternatively, each test scenario may be formed by combining multiple business links, i.e., the test scenario may be formed by combining test cases of multiple links of test data for a given test scenario.

Step 402: generating a slice business link flow diagram; the slicing business link flow graph comprises M paths; each path is composed of nodes and edges; vectors between nodes represent links between the nodes; the value of the vector represents the priority of the inter-link; one path corresponds to one test scene; one node corresponds to one business link in the test scenario.

It can be understood that the slice service link flow graph can be generated according to M test scenarios of the NSMF slice service and priorities among a plurality of service links and each service link included in the test scenario of each NSMF slice service.

Step 403: when the problem of determining the test scene with the highest priority is converted into the problem of determining the shortest path in the slice service link flow graph, performing inverse operation on the value of the vector between the two corresponding nodes according to any two nodes in the slice service link flow graph to obtain a vector value after the inverse operation; and determining other nodes passing between the corresponding two nodes; constructing an adjacency matrix by using vector values after performing inverting operation between any two nodes; and constructing a path matrix by using other nodes passing between any two nodes.

Fig. 6 is a schematic diagram of an implementation flow for setting the priority of each service link, as shown in fig. 6, including:

step 601: and traversing all service links of the NSMF slicing service flow.

Step 602: judging whether the current service link is a branch link or not; if the current service link is not a branch link, the current service link is a main link, and step 603 is executed; otherwise, step 604 is performed.

It can be understood that the service link irrelevant to the slicing service parameters is the main link; the traffic links related to the slice traffic parameters are branching links.

Step 603: and setting the priority of the main link as a default value.

It can be appreciated that, since the main link is not judged by the slicing service parameter branch and is fixedly present in each test scene, the priority of the main link can be set manually, for example, to a default value. The priority is a positive integer, and the higher the priority is, the higher the importance degree of the description link is.

Step 604: and setting the priority of the branch link according to the slice service parameter combination.

It can be appreciated that the priority of the branching links may be determined according to the slicing service requirement, and the greater the priority, the higher the importance of the service links.

It can be appreciated that, because the branching links are strongly associated with slicing service parameters, manual setting can be performed according to service indicators or customer requirements.

Step 404: performing relaxation operation by using the adjacency matrix and the path matrix to obtain a shortest path; and taking the test scene corresponding to the shortest path as the test scene with the highest priority.

Step 405: determining a first test cost of a test scene with the highest priority; judging whether the total test cost is greater than or equal to the first test cost of the test scene with the highest priority; in the event that the total test cost is greater than or equal to the first test cost, performing step 406; otherwise, the selection of the test scene is ended.

It can be understood that, because there are many test scenarios in NSMF slice service, in the embodiment of the present invention, a test scenario with a high priority is selected for testing on the premise of limited test cost.

It will be appreciated that the test cost also needs to be measured, and the test cost of the test scenario for each NSMF slice service depends on all the service links that make up the scenario, so only the test cost of each service link needs to be measured.

It will be appreciated that the measurement of the test cost may specifically include:

first, the traditional manual test scheme measures the test cost by a person x days, and sets the test cost according to the human budget of each link in the historical scene test process.

Second, for an automation scheme, since the program automatically executes each link, the procedure is mainly dependent on the procedure steps of the link.

For example, the interface automation scheme, the test cost of a link can be described by the number of interfaces; alternatively, a User Interface (UI) automates the test scheme, and the test cost for each link may be described by the number of test steps.

Step 406: determining test scripts of a plurality of business links contained in a test scene with highest priority; cascading the test scripts of the determined multiple service links to obtain a test script corresponding to the test scene with the highest priority; and testing the determined test scene with the highest priority by using the test script corresponding to the test scene with the highest priority.

It can be understood that the number of branches of the test flow in the test scenario of the NSMF slice service is huge, and if both manual tests are adopted, the efficiency is low. To enable automated scenario testing, and considering that there is duplication of links contained by each scenario of an NSMF slice service, an automated script of a test scenario of an NSMF slice service may be described as a collection of automated scripts of its constituent links.

That is, first, an automation script of each link is generated, and the automation scripts of a plurality of links corresponding to the test scene with the highest priority are cascaded, so as to obtain the automation test script corresponding to the test scene with the highest priority.

The automation script is not limited to a specific script, and may be an interface automation script, a UI automation script, or the like. The data layers of various types of automation scripts may depend on slice service composition parameters.

Here, after testing the determined test scenario with the highest priority by using the test script corresponding to the test scenario with the highest priority, the total test cost and the first test cost may be differenced to obtain a remaining test cost; updating the total test cost to a residual test cost. Based on M test scenes of NSMF slicing service, determining the test scene with the highest priority again; and steps 404, 405 are performed again. And the like until the updated total test cost is less than the second test cost of the newly determined highest-priority test scenario.

In this example, based on the slice service link flow graph, the test scene with the highest priority is determined, which has the following advantages:

(1) In consideration of the fact that priority differentiation exists in each test scene under the premise that the test cost is limited, an NSMF slicing service opening flow is abstracted into a service flow diagram, priority parameters are set in link paths of the service flow diagram, and the problem of selecting the test scene with the highest priority is converted into the problem of solving the shortest path in the slicing service flow diagram.

(2) The automatic scripts of all links contained in the test scene with the highest priority are combined in a cascading manner, so that the automatic script of the test scene with the highest priority can be obtained, and the automatic test of the test scene can be realized.

In order to realize the test method of the embodiment of the invention, the embodiment of the invention also provides a test device. Fig. 7 is a schematic diagram of the composition structure of a testing device according to an embodiment of the present invention, as shown in fig. 7, the device includes:

an acquiring unit 71, configured to acquire M test scenarios and total test costs of an NSMF slice service; m is a positive integer greater than 1;

a processing unit 72 for performing the following operations:

In one embodiment, the processing unit 72 is specifically configured to:

wherein the value range of i is 1 to N; j has a value ranging from 1 to N.

In one embodiment, the processing unit 72 is specifically configured to:

In an embodiment, the processing unit 72 is further configured to:

In one embodiment, the processing unit 72 is specifically configured to:

In practical application, the acquiring unit 71 may be implemented by a communication interface in the testing device; the processing unit 72 may be implemented by a processor in the testing device.

It should be noted that: in the test device provided in the above embodiment, only the division of each program module is used for illustration, and in practical application, the process allocation may be performed by different program modules according to needs, that is, the internal structure of the device is divided into different program modules, so as to complete all or part of the processes described above. In addition, the test device and the test method provided in the foregoing embodiments belong to the same concept, and specific implementation processes thereof are detailed in the method embodiments and are not described herein again.

The embodiment of the invention also provides a network device, as shown in fig. 8, including:

a communication interface 81 capable of information interaction with other devices;

and the processor 82 is connected with the communication interface 81 and is used for executing the method provided by one or more technical schemes on the network equipment side when running the computer program. And the computer program is stored on the memory 83.

It should be noted that: the specific processing procedures of the processor 82 and the communication interface 81 are detailed in the method embodiment, and are not described herein.

Of course, in actual practice, the various components in network device 80 are coupled together by bus system 84. It is understood that the bus system 84 is used to enable connected communications between these components. The bus system 84 includes a power bus, a control bus, and a status signal bus in addition to the data bus. But for clarity of illustration the various buses are labeled as bus system 84 in fig. 8.

The memory 83 in the present embodiment is used to store various types of data to support the operation of the network device 80. Examples of such data include: any computer program for operation on the network device 80.

The method disclosed in the embodiments of the present application may be applied to the processor 82 or implemented by the processor 82. The processor 82 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuitry of hardware in the processor 82 or by instructions in the form of software. The processor 82 described above may be a general purpose processor, a digital data processor (DSP, digital Signal Processor), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. The processor 82 may implement or perform the methods, steps, and logic blocks disclosed in embodiments of the present application. The general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the method disclosed in the embodiments of the present application may be directly embodied in a hardware decoding processor or implemented by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium located in a memory 83, and the processor 82 reads information from the memory 83 and, in combination with its hardware, performs the steps of the method described above.

In an exemplary embodiment, the network device 80 may be implemented by one or more application specific integrated circuits (ASIC, application Specific Integrated Circuit), DSPs, programmable logic devices (PLD, programmable Logic Device), complex programmable logic devices (CPLD, complex Programmable Logic Device), field programmable gate arrays (FPGA, field-Programmable Gate Array), general purpose processors, controllers, microcontrollers (MCU, micro Controller Unit), microprocessors (Microprocessor), or other electronic components for performing the aforementioned methods.

It is to be understood that the memory (memory 83) of the embodiments of the present application may be either volatile memory or nonvolatile memory, and may include both volatile and nonvolatile memory. Wherein the nonvolatile Memory may be Read Only Memory (ROM), programmable Read Only Memory (PROM, programmable Read-Only Memory), erasable programmable Read Only Memory (EPROM, erasable Programmable Read-Only Memory), electrically erasable programmable Read Only Memory (EEPROM, electrically Erasable Programmable Read-Only Memory), magnetic random access Memory (FRAM, ferromagnetic random access Memory), flash Memory (Flash Memory), magnetic surface Memory, optical disk, or compact disk Read Only Memory (CD-ROM, compact Disc Read-Only Memory); the magnetic surface memory may be a disk memory or a tape memory. The volatile memory may be random access memory (RAM, random Access Memory), which acts as external cache memory. By way of example, and not limitation, many forms of RAM are available, such as static random access memory (SRAM, static Random Access Memory), synchronous static random access memory (SSRAM, synchronous Static Random Access Memory), dynamic random access memory (DRAM, dynamic Random Access Memory), synchronous dynamic random access memory (SDRAM, synchronous Dynamic Random Access Memory), double data rate synchronous dynamic random access memory (ddr SDRAM, double Data Rate Synchronous Dynamic Random Access Memory), enhanced synchronous dynamic random access memory (ESDRAM, enhanced Synchronous Dynamic Random Access Memory), synchronous link dynamic random access memory (SLDRAM, syncLink Dynamic Random Access Memory), direct memory bus random access memory (DRRAM, direct Rambus Random Access Memory). The memory described in the embodiments of the present application is intended to comprise, without being limited to, these and any other suitable types of memory.

In an exemplary embodiment, the present invention also provides a storage medium, i.e. a computer storage medium, in particular a computer readable storage medium, for example comprising a memory storing a computer program executable by the processor 82 of the network device 80 for performing the steps of the network device side method described above. The computer readable storage medium may be FRAM, ROM, PROM, EPROM, EEPROM, flash Memory, magnetic surface Memory, optical disk, or CD-ROM.

It should be noted that: "first," "second," etc. are used to distinguish similar objects and not necessarily to describe a particular order or sequence.

In addition, the embodiments of the present invention may be arbitrarily combined without any collision.

The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the present invention.

Claims

1. A method of testing applied to a slice management function NSMF, the method comprising:

and performs the following operations:

2. The method of claim 1, wherein the determining the highest priority test scenario based on the M test scenarios of NSMF sliced traffic comprises:

3. The method of claim 2, wherein performing a relaxation operation using the adjacency matrix and the path matrix results in a shortest path, comprising:

wherein the value range of i is 1 to N; j is a value ranging from 1 to N.

4. A method according to claim 3, wherein said determining the shortest path between the i-th node and the j-th node using the values of the vectors between the i-th node and the j-th node in combination with the path matrix comprises:

5. The method according to claim 2, wherein the method further comprises:

6. The method of claim 1, wherein testing the determined highest priority test scenario comprises:

7. A test device, comprising:

a processing unit for performing the following operations: determining a test scene with highest priority based on M test scenes of NSMF slicing service; determining a first test cost of a test scene with the highest priority;

8. A network device, comprising:

a processor for performing the operations of: determining a test scene with highest priority based on M test scenes of NSMF slicing service; determining a first test cost of a test scene with the highest priority;

9. A network device comprising a processor and a memory for storing a computer program capable of running on the processor,

Wherein the processor is adapted to perform the steps of the method of any of claims 1 to 6 when the computer program is run.

10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method according to any one of claims 1 to 6.